Diffusion Models and AI Art
In prior chapters, we’ve looked at examples of how generative models can be used to create novel images; we’ve also seen how language models can be used to author answers to questions or create entirely new creative text like poems. In this chapter, we bring together these two concepts by showing how user prompts can be translated into images, allowing you to author “AI art” using natural language. In addition to creating novel images, we can perform some useful functions like extending an image beyond its current boundaries (“outfilling”) and defining features for safety screening in our results. We’ll also look at one of the foundational ideas underlying this image generation methodology, the diffusion model, which uses the concept of heat transfer to represent how an input of random numbers is “decoded” into an image. To illustrate these ideas, we’ll primarily work with Stable Diffusion...
