Prompt Engineering 101: Generate Sick Stable Diffusion Images
Nov 18, 2022
Anyone can generate sick AI images — you just gotta know how to talk to AI.
If you’ve tried your hand at something like Stable Diffusion and wondered: “How in the hell did they generate that? Why aren’t my prompts working like theirs?!”
Then this is the best place to start.
Even if you’ve generated something spectacular, you may feel like it was dumb luck. The AI feels random. But there’s a method to this madness — we call it prompt engineering.
Give me less than 10 minutes of your life and I’ll teach you how to make something like this:
Or, you can just read the tl;dr
By the way - throughout this note you'll have a chance to generate and share original images. If you like it and want to build more stuff check this out
Join the the world's best builders for a 6-week sprint
Come join the best builders from around the world to build wild ideas in web3, ML/AI, gaming, bio-anything. You've got what it takes - all you need to do is apply
The best way to start is by breaking down the prompt to a good image.
Here’s the prompt from the first image you saw:
illustration of jupiter clouds by dan mumford, alien landscape and vegetation, epic scene, a lot of swirling clouds, high exposure, highly detailed, fantastical, vibrant red tinted colors, uhd
Let’s look at it from the POV of the prompt formula.
illustration (this is my style choice) of
jupiter clouds (the thing I want in the image) +by dan mumford (artist inspo),
alien landscape and vegetation (more stuff I want in the image),
and the rest are all details on how I would like the image to look.
Nice, now you have all that you need to make some amazing images.
It’s time to get you shipping something.
Get wild and put a car on planet Jupiter using Lexica.art. Try using the formula I just showed you.
TL;DR: Prompt Formula
Art Medium + The actual things you want in the image + Art Styles (could be an artist or an actual style like Renaissance) + Modifications (extra details on how the image should look) ⇒ Good Image.
Leveling up your prompt production
I like to imagine that I am sketching on a piece of paper. You add the details step by step to see if it fits the vibes you want.
Here’s my process. Tag along and try it out yourself.
For example, you get this image with just the prompt:
Illustration of Jupiter clouds
Not bad...it might make it on your parent’s fridge.
But what if Dan Mumford drew it?
Illustration of jupiter clouds by dan mumford
Well, here’s an example of Dan’s real art, so it ain’t bad as a start!
What would it look like if a different artist drew it? Let’s try same prompt + different artist.
Illustration of jupiter clouds by aaron horkey
See how it changed the vibes? You can clearly see that the artist’s style makes a big difference.
Let’s add a few more details to our Dan Mumford image so it can have that cool green vibe and a more upscale look.
Heck, let’s add in some more planets too. Yellow on green can look pretty good.
Illustration of jupiter clouds by dan mumford, yellow planets, green tinted colors, uhd
Btw, “uhd” just means ultra-hd lol.
Now this is looking pretty sick and it’s as if we can sketch in Dan Mumford’s style now!
Show me what you got!
Keep in mind, it took about 3-4 tries to get something I liked — the same prompt will actually give you different results each time.
3-4 times of generating images before changing the prompt is my rule of thumb.
After that, I will start to tweak the prompt to make exactly what I want.
By this point, you are ready to go. You can officially generate images that match your imagination.
All you need to do now is practice. This is the best way to build up insane intuition for how the AI generates images, and how to make it an artistic medium for you.
Now, let’s do something more interesting — let’s study some sick art.
A lil “art theory” for builders
Before we add any additional details to our prompt, it’s good to study the techniques of real artists.
I’m gonna keep this discussion brief though. I want you spending more time making art yourself, rather than reading about it.
The more art you make, the more genius you’ll notice in these artists when you come back to them.
What do you notice when you look at Dan Mumford’s art above?
He uses high exposure. There’s always a strong source of light emitting from the background. His color usages are also rich af.
This gives his art a dramatic look.
How do we capture this with AI?
Let’s go back to the first image you saw here that has the Dan Mumford vibes.
Now check out the prompt with a fresh set of eyes now that you’re a prompt engineering pro.
Illustration of jupiter clouds by dan mumford, alien landscape and vegetation, epic scene, a lot of swirling clouds, high exposure, highly detailed, fantastical, vibrant red tinted colors, uhd
By adding in high exposure + vibrant [any color you want], we are able to capture more of how Dan Mumford would have illustrated the image we want.
You are 99% there
Where does the last 1% come from?
Well, we haven’t even covered the technical aspects like weights, step counter, seed, etc.
But if you’re satisfied with just generating images via prompts, then you’re good to go.
All of the images here were created only through prompts. The images on my Twitter account are also made only through prompts — and there are a lot of wild and beautiful creations there.
You just gotta practice. Mix a few style/artists together and see what you get. Add some styles that an artist typically doesn’t do, like the modifier, fantastical, that I included for my Dan Mumford style.
Let’s make some sick art.
ps. Don't stop here - keep shipping your wildest ideas alongside 1000s of other builders during the next season of our 6-week long sprint called Nights & Weekends. You can apply below.