OPINION
AI is outrageous — and wonderful. It’s also prompting a new art form.
AI-GENERATED IMAGES
"Dragon Cave" by Grzegorz "Greg" Rutkowski.
SCROLL TO CONTINUE
Tens of billions of images made by artificial intelligence have been posted online in the past two years. The images you see here were created by Stable Diffusion, a popular AI image generator.
Artificial intelligence creates pictures from users’ word prompts. The images highlighted now resulted from prompts using the name Greg Rutkowski.
Rutkowski is a 35-year-old Polish artist known for drawing fantastic worlds. His name has become a common prompt because many users want to mimic his style. Here’s one of his original artworks.
The many knockoffs of his work have Rutkowski worrying about his career. “Some of the clients that I was working with are using AI in the designing-early production phase,” he wrote in an email.
Artists such as Rutkowski often feel threatened by artificial intelligence. But some are starting to find new creative potential in the technology.
All kinds of people in the creative community are at odds with artificial intelligence. Some are suing top AI companies for feeding their work to computer models without permission, compensation or even notification. Others criticize the quality of the material it produces and worry that it could be used to create knockoffs, threatening millions of jobs.
These are serious concerns that need to be addressed. AI companies should give artists the option to opt out of AI training, and artists whose work is used to feed computer models should get appropriate compensation. But artificial intelligence also has an upside: The technology is opening new pathways for creativity. Consider visual arts. Just as photography changed the course of art in the 19th century, AI image generators now stand to revolutionize how humans create.
The key to this creation is the interaction between human and machine — the “prompts” users enter to generate images. Here’s an example of how it works, using Dall-E, a popular AI image generator developed by OpenAI.
PROMPT:
Artist holding brush drawing an oil painting
AI-GENERATED IMAGE
PROMPT:
Artist holding brush drawing an oil painting
AI-GENERATED IMAGE
PROMPT:
Artist holding brush drawing an oil painting
AI-GENERATED IMAGE
Most models are able to generate images with a few keywords, but these images can be biased or unappealing. For example, unless other details are provided, the word “artist” is likely to generate images of White men.
A refined prompt would generate a more customized picture. The new image might introduce different biases — an “Asian artist” represented as a petite man, for instance. But users can keep fine-tuning.
Image generators are evolving to optimize users’ prompts automatically, but a clear visual description is essential. Users can evoke specific styles, objects or camera angles. Here, “cyberpunk” and “Blade Runner” contribute to a futuristic scene.
Advanced image generators also enable users to upload images to help control the outcome. The cat photo below shows how style and subject references can come together.
Mixing word and image prompts allows users to reshape a picture’s style, subject or emotion to achieve a more original result with personal touches.
With prompts, language itself becomes the creative medium. Here’s another analogy with photography: Though anyone can snap a photo, not every photo qualifies as meaningful art. Similarly, not all prompts yield compelling visuals.
AI enthusiasts have noticed this. Researchers from Georgia Tech and IBM recently analyzed 14 million pairs of image prompts and found that mentioning artistic styles, artist names, genres, media or specific painting techniques significantly influences the result.
Developing effective prompts requires trial and error, and it can often feel random and unprincipled. The chart below shows some popular terms that users have been testing to try to better control the outcome:
Source: DiffusionDB by Georgia Tech and IBM Research
The words “photo” and “photography” are among the most widely used terms.
But words such as “cinematic lighting,” “volumetric shadows” and “deep focus” are less common. This gives people with more knowledge of photography and art an edge.
Other specialized artistic terms, such as “chiaroscuro lighting” and “occlusion shadow,” usually help achieve a more predictable style.
Phillip Toledano, a photographer with about 20 years of experience and an early AI adopter, shared his perspective on AI’s creative potential. When making an image, Toledano thinks about light, shadow, details of the scene, emotions, narrative and how to convey these ideas to AI. “If you haven’t spent so much time being a photographer, you may not think about that,” he said.
The photographer uses Midjourney, a popular AI image generator, to create his pieces. “AI is often not about the initial product, but it’s about finessing that image over and over again,” he said. Below are some of Toledano’s concepts.
“AI will produce as good an idea as I can produce. It’s only limited by my level of creativity, by my level of taste.”
– Phillip Toledano
The artist politely declined The Post’s request to list some of his prompts, saying that they are part of his “magic spells.” His reservation is further proof that AI is a creative tool. Prompts have become products that can be traded at online marketplaces such as PromptBase.
Even New York’s Museum of Modern Art has started to embrace AI-generated artwork. The image below is part of the show “Unsupervised,” by Refik Anadol, displayed at the museum last year. Anadol used AI to interpret and transform more than 200 years of art in MoMA’s collection. He wrote in an email to The Post that creating the artwork is “a process requiring meticulous adjustments and a deep understanding of how technology can expand the boundaries of our interactions with art.”
A view of "Refik Anadol: Unsupervised" at MoMA, in New York, in 2023. (Robert Gerhardt/MoMA)
“It is a living, alternate universe of AI-driven creativity, inspired by MoMA’s vast collection, but not confined by it.”
– Refik Anadol
As these examples show, creative professionals might still have an advantage in the world of AI art. Aesthetic taste, culture and skills honed over years can substantially influence the quality of AI-generated images. “If AI is not for you, that’s fine. But shouting about it is like shouting at the sea,” Toledano said. “It’s here. Be curious.”
When photography came along, it inspired artists to dive into new styles like expressionism and surrealism. Now, AI is doing something similar. We might still be in the AI-equivalent of the daguerreotype, the first photographic camera developed for commercial manufacturing. As this new way of creating matures, more unique, groundbreaking forms of art will certainly evolve.