Gold Penguin Logo with Text

Stable Diffusion vs Midjourney Niji V6: Which Is Better For Anime?

Midjourney Niji 6 has become almost ubiquitous with anime AI art. Can Stable Diffusion take down the king?
Updated May 22, 2024
Two artificial intelligences fighting each other, generated with Midjourney Niji
Two artificial intelligences fighting each other, generated with Midjourney Niji

Anime is an iconic Japanese art style that has legions (yes, legions) of fans all over the globe. Its distinct visuals, gripping tales, and diverse genres have smashed through cultural barriers, making it a worldwide phenomenon.

In recent times, AI image generators have unlocked new possibilities for artists and enthusiasts to explore and recreate the world of anime with ease. As demand for AI-made anime art skyrockets, one model has risen to the top: Midjourney Niji 6. But the burning question is — is it actually the best?

So, let’s go on a quest to decide, once and for all, if Midjourney Niji 6 is the best AI image generator for anime. Today’s comparison will feature Niji in a head-to-head battle for the title against Stable Diffusion: another popular text-to-image model. Can Stable Diffusion make a case for the crown?

What are Stable Diffusion and Midjourney Niji?

Midjourney is an AI image generation model developed by a company called, well, Midjourney. It uses machine learning to create images from text prompts, allowing you to turn ideas to complete artwork. Niji is an anime-optimized model built on top of Midjourney’s base model, which uses an extensive training set of anime images and artwork.

Stable Diffusion is also a text-to-image model like Midjourney, but the key difference is that it’s open-source. While its base model is already capable of creating artwork, being open-source allows developers and artists to fine-tune the model by letting you train the model further with LoRAs or small Stable Diffusion models.

Stable Diffusion vs. Midjourney Niji 6: Features Comparison

Apart from image generation, Stable Diffusion’s features (using Dream Studio) includes negative prompts to exclude certain elements from an image, aspect ratio adjustment, changing of generated image count, and switching between different art styles. You can also upload an image and let Stable Diffusion generate variations of the input.

Midjourney Niji’s features are almost the same as Stable Diffusion (except image count adjustment) along with panning and zooming, region variations, style and character reference, and more.

Stable Diffusion vs. Midjourney Niji 6: Output Comparison

Just a quick note for this comparison, we'll be focusing on the capabilities of the Stable Diffusion XL model from DreamStudio. We'll hold off on using LoRA fine-tuning for now. With that in mind, let's see how Midjourney Niji and Stable Diffusion perform using the same prompts.

Character Design

Prompt: a powerful necromancer, dark fantasy, in the style of junji ito, character design, runes and spellbooks

Superhero Anime

Prompt: still from a superhero anime, a teenage girl controlling a death god, in the style of mappa animation, overcast weather

Horror Anime

Prompt: a woman possessed by the devil, satoshi kon, unsettling, creepy, uncanny

Sci-Fi Anime

Prompt: an astronaut stepping foot into a new planet, triple moons, celestial bodies, pseudo-realistic, light crimson and violet

Slice of Life Anime

Prompt: romance anime, two teenagers sharing an umbrella during a rainstorm, ambient lighting, framing by edward yang

Fantasy Anime

Prompt: fantasy anime, a woman in a kimono talking to animals in the forest, whimsical, earth tones, in the style of hayao miyazaki, studio ghibli

With Text

Prompt: a “CLOSING SALE” sign outside a store

Overall Thoughts

Amongst all of our Midjourney Niji anime comparisons, this head to head feels closest. The fact that Stable Diffusion was able to trade blow for blow with Midjourney Niji 6 is an achievement in itself. However, there’s one significant issue with Stable Diffusion images, but let’s save that for later and focus on the good things first.

One of the minor issues I have with Midjourney Niji 6 is its strong artistic direction. While it can lead to beautiful artwork, the trade-off is that some of its outputs will look too similar to each other. On the other hand, Stable Diffusion’s base model is a lot more flexible when it comes to creativity.

Stable Diffusion did particularly well, in my opinion, with the character design and horror anime prompts. It’s also a good sign that it can (almost) generate accurate text. As for Midjourney Niji, what else did we expect? They’re all beautiful and intricate anime art pieces that follow the prompt to a tee.

Now, for the reason I still prefer Midjourney Niji over Stable Diffusion: I want you to scroll back up and zoom in to all the Stable Diffusion outputs featuring faces. Notice how all of them are still distorted or warped? This is an issue plaguing AI image generators from the beginning. It seems that Stable Diffusion hasn’t gotten over that hump yet.

The Bottom Line

This was a close one, but in the end, I still have to give the win to Midjourney Niji 6 for anime artwork.

That said, I do think that Stable Diffusion shows a lot of promise for anime, and that’s without using any LoRAs. I’ve seen what a fine-tuned version of Stable Diffusion can do, and believe me, it’s on a whole different level. 

Do you want to see more of what Midjourney Niji can do? You can check out some of our comparison articles like this one. Have fun!

Want To Learn Even More?
If you enjoyed this article, subscribe to our free monthly newsletter
where we share tips & tricks on how to use tech & AI to grow and optimize your business, career, and life.
Written by John Angelo Yap
Hi, I'm Angelo. I'm currently an undergraduate student studying Software Engineering. Now, you might be wondering, what is a computer science student doing writing for Gold Penguin? I took up studying computer science because it was practical and because I was good at it. But, if I had the chance, I'd be writing for a career. Building worlds and adjectivizing nouns for no other reason other than they sound good. And that's why I'm here.
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
Join Our Newsletter!
If you enjoyed this article, subscribe to our newsletter where we share tips & tricks on how to make use of some incredible AI tools that you can use to grow and optimize a business
magnifiercross