AI flavors

Hey guys, so I've had more time to spend not only learning about the different AI flavors but also configuring my own local version.

There are new versions of text to image AIs being worked on every day. The Big Three are Doll-e 2, midjourney and Disco Diffusion. Currently, Doll-e 2 is only accepting signups to be on a "coming soon" list. Midjourney also has a sign up sheet but they are much more available to new users and those wishing to try it out. One reason is that new users can also invite up to five other friends.

And Disco Diffusion is the one that pretty much anybody can jump into and start using. Both Doll-e 2 and midjourney have very simple and sparse user interfaces that don't easily allow for much customization. On the other hand, Disco Diffusion is much more complex with many more different types of settings and properties that go along with the text prompting. It runs using "Colab notebooks" on webpages which allow users to run sequenced groups of segmented python code. The notebook is basically a single web page with a bunch of fields on it where you can enter in variable values and when you run that segment of code those values are automatically updated in the python. While not the best interface in the world, it is one that is easily updatable and you can get used to it pretty fast.

Disco Diffusion also is free to use but if you want any sort of performance gains you'll need to subscribe to either the $10 or $50 per month Colab plans. The difference between the two is that with the more expensive plan, you can run batches and not have to be at your computer working with the program. So in other words, you can run 50 or 100 renders overnight and do it using two different Colabs if you wish. Also, with the most expensive plan called Pro+, you tend to get access to faster GPU computers.

From what I can gather, each one of these three AIs use different data sets, and the resulting images, even from the exact same text prompt, are mostly never similar. Frankly, I'm still learning about all the nuances in not only the different Clip and Diffusion settings, not to mention the different branded versions. Here's a neat video that explains a bit how these AIs work. https://youtu.be/SVcsDDABEkM

Also, I should mention that both Doll-e 2 and midjourney have much more powerful online computers and render significantly faster than Disco Diffusion with the Colab Google servers.

My only experience is with midjourney and Disco Diffusion-- in that order.

Midjourney has a very simplistic interface, (see pictue) which only runs inside of a Discord channel. You enter in a text prompt and press return and very shortly the AI sends back four small images. At this point you can choose to refine/iterate on any of the images, or you can choose to upscale any of the images-- which create larger resolutions and more detail. And, you can also choose to refresh and yet a new batch of four images.

Midjourney has a few different pricing plans. The first is $10 per month and this gets you 200 GPU minutes which is probably about 20 to 30 full sessions where you do a number of iterations as well as a number of upscaling. At any time you can check and see how many minutes you have left on your monthly allotment. At any time, you can also purchase directly more minutes if need be.

And for $30 a month you get unlimited usage but after a set period of usage it will put you in "relax" mode slows down significantly to make it almost unusable. At least that's some of the complaints I heard as I've only subscribed to the $10 per month plan.

Shown above is the current plan matrix, I suspect it will change.

There is no real batch mode for midjourney. You typically work with one prompt at a time and keep refining it. You can of course upscale more than one version as well as get different iterations in a batch like manner. One nice thing about it, is that you can do all of this using Discord on your smartphone. Unfortunately, Disco Diffusion Colab does not work on smartphones, at least not on mine (Samsung Fold 3).

When you are first invited to midjourney, they give you a few minutes of free use where you get to play around before they ask you to subscribe.

Pretty soon after playing around in midjourney, I started to explore Disco Diffusion. I think both can create great images. Unfortunately, Disco Diffusion is pretty hard to get started with and AFAICT the online tutorials are pretty crappy. I think I can do a better job and will do so soon.

One of the nice things about Disco Diffusion and the collab notebooks, is I can run them from a Chromebook which I often use when not at my desk. My workflow is fairly straightforward, as I first use many iterations of text prompts and other settings to create very small batches of previews and when I finally find ones that I like I then create larger resolutions with higher quality settings and let them run for a few hours.

This can become tedious and take quite a bit of time, as the servers are slower than midjourney, so I thought it might be a good idea to try and get a local version of Disco Diffusion as I have a Nvidia RTX 3090 GPU on my computer-- which runs 3-4X the Google Colab servers. My goal was to create Disco Diffusion on my hard disk and then use a Colab notebook stored in the cloud on my Google drive. By doing this, I can rapidly do the small preview iteration step and then duplicate the same notebook, upscale the settings, and batch run it on Google's servers. This way I'm not stressing out my computer and can use it for other things like Blender or making videos.

It turns out this was a very difficult thing to do, but I took lots of notes and I'll probably eventually create a video tutorial on how all that works as well.

I also have a lot of thoughts on what this new technology means for artists and designers but I'll leave that to another post. For one, I'm interested in using this tool to augment some of my existing 3D renders. It'll be interesting to see how that goes.