If you're not familiar with DALL-E, it's an artificial intelligence photo generator that can create images from textual descriptions, using a 3.5-billion parameter model to interpret natural language inputs. You can read all about it in my previous article. Long story short, it's an incredibly powerful tool that can be used for all sorts of different things.
If you're obsessed with all these new text-to-image AI generators like I am, you've probably dipped your toes into each of them. After using DALL-E 2 for about 2 months, I feel like it's the best one yet. Other generators like Midjourney and Stable Diffusion still work amazingly well, but they don't seem to recreate the experience I find working with DALL-E.
What makes DALL-E 2 better? For starters, the ability to edit existing photos and add content to certain sections of them. This is a game changer for anyone who wants to create composite images or simply tweak an image they have. DALLE is also trained on millions of stock photo images, so it can do a great job of incorporating real-world objects into your creations. Editing pictures in combination with millions of data sets allows for some really mind-blowing results that I haven't seen elsewhere.
Over the last few days, I've been getting really into inpainting with DALL-E. Inpainting is the formal term describing the process of filling in missing or damaged portions of an image. It's a technique that's been used for years by graphic designers and photoshop artists to fix photos, but it's only recently that AI has become good enough to do it automatically. And DALL-E is really good at it.
Expanding Image Inpainting with DALL-E
This technique was something I thought of when I was scrolling through pictures of a beautiful resort and noticed the picture was great – but would be even better if there was simply more beauty! I've seen people use inpainting to remove things like power lines and telephone poles from pictures, but I wanted to see if I could use it for something a little more creative. So I decided to try and expand the image outward, essentially filling in the new space with an interpolated version of the existing image.
I used Photoshop to increase the image by 200% in both the width & height, leaving me with the original picture in the middle & padding all around it. I then took that image and ran it through DALL-E by uploading, erasing the sides, and letting it do the rest. The results, as you can see below, are pretty unique
This seems to work better with abstract pictures over those with a lot of sharp detail. I imagine it would also be interesting to run an image that has been edited in photoshop first, as the AI would then have more to work with when expanding the image.
Add Object Inpainting with DALL-E
After this, I decided to use the upload feature as it was intended: to fill in an image with an object. I found a picture I took of a house near a lake and figured I could add a garage to it with DALL-E. I took the original image, went to DALL-E, uploaded the new image with the hole in it, typed 'a garage for cars', and waited for the results
The results were mixed, but I think it's definitely possible to get some interesting results with this technique.
Default Image Variations with DALL-E
And lastly, I wanted to see what would happen if I simply took an image, uploaded to DALL-E, and instead of editing, just generated variations on the default output. I took a picture of a road during autumn and let DALL-E do its thing
The results were interesting, to say the least. I got a wide variety of outputs, some of which were completely different versions but still maintained the feel of the original image.
Overall, I'm extremely impressed with DALL-E's ability to edit & expand images. I think it has a lot of potential for creative uses despite it not being perfect. None of these tools are perfect and we shouldn't expect them to be, but it's really fun to experiment with them and see what they're capable of. So if you're looking for a new way to edit pictures or simply want to generate some interesting variations on an image, give DALL-E a try.