ML/AI
Art
Stable Diffusion vs Midjourney vs DALL•E 2
Meta Mae
•
Jan 24, 2023
Amidst the ongoing generative AI storm, three text-to-image models have stood out: Stable Diffusion, Midjourney, and DALL-E-2.
Let's keep this super short. You're here to figure out which one is best for your workflow — we'll give you just that.
Stable Diffusion
If you’re looking for an open-source image generator, Stable Diffusion is the only option of the three at this time. You can run SD locally on your computer, meaning you have more control, better customization, and can even build AI tools of your own using their deep learning text-to-image model.
Because it is open source, a lot of tools/companies have emerged that allow you to access Stable Diffusion directly from your web browser. We'll use Lexica in this note because it has a search aggregator and generator built in!
Go to lexica.art and create an account if you don't already have one.
Enter a prompt for what you want to create in the “Describe your image” box.
(optional) Enter negative prompts for things you don't want to see in your image.
Generate.
That's it. Seriously.
Since Stable Diffusion is open source and allows for some pretty intense customization, you can train it with your very own model and push the limits of your imagination. Lexica has done just that with their own 'Aperture' model. If you're curious about how to train your own model, check out this project:
Stable Diffusion is also really great if you're on a price crunch. There are many, many ways to access SD for free (search 'SD Replicate' to get immediate options).
Some of the custom models (like Aperture) also blow away the rest of the competing AI-generation tools since they are specifically trained on humans and stock images. Check out the image below for example:
Manju Warier, An astronaut in a garden on a spring day, by martine johanna and simon stalenhag and chie yoshii and casey weldon and wlop : : ornate, dynamic, particulate, rich colors, intricate, elegant, highly detailed, harper's bazaar art, fashion magazine, smooth, sharp focus, 8 k, octane render
Absolutely insane.
Midjourney
Midjourney isn't open-source like Stable Diffusion, but it is still magical, especially for more artistic images. It shines in generation of images that are similar to those created by an artist.
Join the MJ Discord: https://discord.gg/midjourney
Once you join the discord, it will ask you a few questions to verify yourself.
Read the #getting-started channel if you want to learn more directly from Midjourney about their features.
Visit ANY of the #newbies channels.
Type /imagine in the chat text box at the bottom, followed by your prompt.
Press Enter on your keyboard and watch your prompt come to life!
This is the equivalent of a "free version". You can use it to test out whether the Midjourney style suits your taste. You can also view some of the incredible generations by other people in the community.
Midjourney prompts need a little bit of different engineering than those for Stable Diffusion. The best way to learn this is by just trying a bunch of stuff to see how you can fine-tune images to get exactly what you want. Let's try the same prompt from earlier:
Manju warier , An astronaut in a garden on a spring day, by martine johanna and simon stalenhag and chie yoshii and casey weldon and wlop : : ornate, dynamic, particulate, rich colors, intricate, elegant, highly detailed, harper's bazaar art, fashion magazine, smooth, sharp focus, 8 k, octane render
This may not look that great compared to the Stable Diffusion one above, but you can keep trying different variations and prompt changes till you find something that fits your boat.
All in all, Midjourney is fantastic for beginners because even its most advanced features are very easy to use. The Discord server comes with a lot of instructions, support, and settings that are easy to access and understand. The pricing for more generations is also extremely affordable (starts at $8/month).
Prompt #1
wide shot of a chaotic arcade at night, anime line art by hayao miyazaki, multi-panel comic, a woman wearing streetwear playing an arcade game, sci-fi, manga panels, light novel style, dark red colors
Stable Diffusion:
Midjourney:
Dall-E-2:
Dall-E-2
Like Midjourney, Dall-E-2 isn't open source, but it does have an API. You can immediately start playing around with it and see how the generations differ from Stable Diffusion and Midjourney.
Visit https://labs.openai.com/ and create an account if you don't already have one.
Type your prompt in the text box.
Press “Generate” or optionally, press “Surprise me”.
Let's repeat the same prompt we have used for SD and MJ.
Manju warier , An astronaut in a garden on a spring day, by martine johanna and simon stalenhag and chie yoshii and casey weldon and wlop : : ornate, dynamic, particulate, rich colors, intricate, elegant, highly detailed, harper's bazaar art, fashion magazine, smooth, sharp focus, 8 k, octane render
As you can see, the results aren't that great. But we can finetune the prompt and try again. However, artistic renderings aren't Dall-E's strong suit. It lacks in terms of customization and advanced features but the pricing is very affordable! Although, on a per-image basis it can be slightly more difficult to keep track of.
What it is really, really good at is photorealism. And the prompts tend to not be as complicated 😁. Let's try something extremely simple.
a portrait of a cat in italy
Let's try the same prompt in Midjourney.
And now, Stable Diffusion.
As you can see, Dall-E-2 has the best generation out of the three when it comes to photorealistic images and succinct prompts. Not to say you can't get similar results with Stable Diffusion or MidJourney, you'll just need to learn a bit about prompt engineering. Here's a good note to learn about it:
Prompt #2
multi-panel comic, anime line art by hayao miyazaki, light novel style, a nerdy girl is programming at a computer in a room full of gadgets, art station, outlined silhouettes, dramatic lighting, masterpiece, dark blue colors
Stable Diffusion:
Midjourney:
Dall-E-2:
Prompt #3
comic panel, anime line art, Vintage 90's anime style environmental wide shot of a sketchy alleyway at night; a woman wearing a hood and sitting while reading a book; by Hajime Sorayama, Greg Tocchini, Virgil Finlay, sci-fi. line art. cyberpunk art
Stable Diffusion:
Midjourney:
Dall-E-2:
Prompt #4
ethereal interdimensional portal in the forest by jean giraud + beeple + insanely detailed, illustrated by kentaro miura, poster, peter mohrbacher, pastel color, symmetrical
Stable Diffusion:
Midjourney:
Dall-E-2:
Wrapping it up!
All these Ai-image generators have their strengths and can provide a great experience depending on what you’re looking for. After reading this note, I encourage you to visit all of the sites and try these out for yourself.
Explore their strengths, test out things on your own, and see how things compare in your own experience. Ultimately, with a solid idea of what you want to create and a super clear prompt, you will receive something cool no matter which platform you are using.
On top of all the greatness I just shared with you above, what if I told you that you could use the strengths of all 3 AI tools together to generate one mega image to triumph over them all?
Here’s an example of one I created using all 3 resources:
Try to combine all 3 image generators on your own, post your results on Twitter, and tag @_buildspace. We want to see what beautiful, creative, and whacky things you all come up with! A special note on this topic may very well be underway.
Good luck!
– Meta Mae