These Abstract Portraits Were Painted By An Artificial Intelligence Program

This past July, at the Galerie Oberkampf in Paris, visitors to the “You Can’t Know My Mind” exhibition were confronted with a laptop and an empty chair.

If they chose, they could sit down and have their portrait painted. The catch, though, was that it’d be planned and executed entirely by an artificial intelligence program called The Painting Fool.

“I’m interested in the idea that software itself can be creative,” says Simon Colton, the British computer scientist behind the program. “I want to drag software into new territory—by getting it to write music, or compose poems or paint pictures in a creative way.”

The Painting Fool was created in 2001, when Colton, who was then working on a dissertation involving artificial intelligence, became obsessed with using photoshop to alter his photography. “I realized photoshop wasn’t doing what I wanted it to do, and I started programming, trying to get the graphics to work how I wanted,” he says. “Eventually, I realized I could bring this computer graphics work into the fold of computational creativity.”

In the years since, his software has created thousands of paintings and graphics, and he’s continually improved the algorithm to come ever-closer to meeting what he sees as seven key criteria for creativity: skill, appreciation, imagination, learning, intentionality, reflection and invention. “Appreciation is what sets the program apart from Photoshop, which has no appreciation of what it’s doing, or what it’s produced, or what materials it’s working with,” Colton says. “In terms of imagination—if the software doesn’t do fun, surprising things, that you wouldn’t have thought of, then it’s not truly creative.”

He and colleagues have developed a number of different applications for the Painting Fool, but for the July exhibition, the program’s approach began with a seemingly unrelated task: reading the newspaper. They want to make the algorithm’s products unpredictable and surprising—hallmarks of creativity—but not merely the result of randomness, so reading the news and analyzing keywords in hundreds of articles is a means of putting the Painting Fool into different moods that inform its work.

At times, reading the news puts the program into such a bad mood that it doesn’t want to paint at all. “I was in a particularly negative mood, because I was reading an article entitled: ‘Aftershocks rock Italy earthquake zone‘ in the world section of the Guardian newspaper, which was really sad, because it spoke of ‘terrified residents.’ So, I decided not to paint a portrait,” the Painting Fool wrote in response to one exhibition-goer.

Most of the time, though, the articles put the program into other moods (experimental, reflective or happy) that dictate one of roughly 30 qualities—bright, colorful, vivid, cold, bleary or crazy, among others—that it seeks to convey with a painting. With this in mind, when a subject sits down for a portrait, the Painting Fool starts issuing instructions. “You never feel like you’re using it, you feel like it’s using you, and you’re the model,” Colton says. “It says, ‘Thanks for being my model.’ Then, maybe ‘I want you to smile right now.’”

After taking a photo, the program isolates the subject’s face and places it within one of roughly 1000 abstract templates, then uses one of an additional 1000 image filters to manipulate the template and face further, searching for a combination likely to produce a portrait with the quality it originally chose. Finally, it splits the image into segments and fills each of these with a different color and texture, using virtual tools such as pencil, pastel or watercolors.

Afterward, the Painting Fool assesses its product and decides whether it achieved the desired look, comparing it to thousands of other works of art in a database with characteristics commonly associated with the artistic quality that it sought to convey. Like a human, it’s sometimes pleased with its work and sometimes disappointed. “I was in a positive mood. So I wanted to paint a patterned portrait,” it wrote in response to the portrait above. “This is a miserable failure—I’m very unhappy about that. And I’m also annoyed that the portrait is bleached, because that does not suit my mood.”

This sort of intentionality and reflection, Colton says, are crucial elements of creativity. “It’s very easy to say, ‘You wrote the program, you tell it what to do, so it’s really just an extension of you. So we tried to get the software to aim to do something on its own, and then realize whether it has or hasn’t achieved it in the end,” he explains.

Colton’s aware that there are lots of people out there who don’t see real creativity in the program—and he sees their criticisms as essential to the Painting Fool’s success. “I’m always looking for people who say to me, ‘I don’t think it’s creative for this reason,’” he says. “That drives me on, and I’ll come back a year later with a few thousand lines of code to begin addressing that issue.”

Like Colton, the Painting Fool’s greatest strength is the fact that it can learn and improve—each time it fails to meet its own expectations, it assesses what went wrong and uses that knowledge in future creative decisions. “It did about 100 portraits, and by the end of the week, it knew, for instance, that pencils are not good for vibrant paintings, but they are good for making bleak and dreary ones,” Colton says. “It reflected, it learned, and by the end, it was doing things that I hadn’t programmed it to do.”

Joseph Stromberg | | READ MORE

Joseph Stromberg was previously a digital reporter for Smithsonian.