New A.I. Tool Makes Historic Photos Move, Blink and Smile

Almost like animated, moving portraits in the Harry Potter franchise, photos once frozen in time are being brought to life with an artificial intelligence (A.I.) program called Deep Nostalgia.

The technology, which was released on February 25 by the genealogy website MyHeritage, has since gone viral. Social media users have created lifelike moving portraits of mathematician Alan Turing, abolitionist Frederick Douglass and physicist Marie Curie, reports Mindy Weisberger for Live Science. The historical figures can blink, move their heads side-to-side, and even smile. The tech is also being used to animate artwork, statues and photos of ancestors.

A.I. can create video footage or "deepfakes" using generative adversarial networks or GANs, Live Science reports. A GAN is composed of two A.I. systems working together to create content that appears eerily similar to the real thing. One system, the generator, creates a copy of an image using new data. Another system, the discriminator, determines if the data passes as real or fake. Over time, both tools will learn to create incredibly lifelike images that are difficult to distinguish from the originals, as reported by Adam Mann for Live Science in 2019.

MyHeritage introduced Deep Nostalgia to allow users to see their ancestors or late relatives in motion. To create lifelike movements, the facial recognition company D-ID, which specializes in deep learning technology, uses GANs to animate the uploaded images. Footage of real people displaying different facial movements were used to train Deep Nostalgia's algorithm to apply motion to still photos, reports Ed Browne for Newsweek.

The algorithm can recognize which pre-recorded video best fits with the photo depending on the subject's pose and applies natural facial movements to images that didn't previously move, reports Kim Lyons for Verge.

"Seeing our beloved ancestors' faces come to life in a video simulation lets us imagine how they might have been in reality, and provides a profound new way of connecting to our family history," Gilad Japhet, CEO and founder of MyHeritage, tells USA Today's Coral Murphy Marcos.

Deep fakes have drawn controversy in recent years for their uncanny ability to create alterred videos or images of political figures and celebrities that can't be distinguished from the real thing. Deep fakes can be used for malicious purposes, such as splicing words into someone's speech, making it appear that they are saying something they never said. In other instances, the faces of Hollywood celebrities were edited onto the bodies of actresses in pornographic videos, reports AJ Willingham for CNN.

Other critics note that historical figures were often very intentional about their posture and expression in photos, like Frederick Douglass, who has been called "the most photographed American of the 19th century." He purposely chose photography as a medium for changing how the world viewed black men at the time. Images of black people existed, but often in subservient, violent or compromised positions in the context of slavery. In the 2015 book Picturing Frederick Douglass, Harvard historian John Stauffer explains that Douglass, as a former enslaved individual, believed photography "highlighted the essential humanity of its subjects," reported Renee Graham for WBUR in 2016. Using Deep Nostaligia to make Douglass move and emote changes the radical message he was trying to convey.

"Douglass specifically—in print, he said that he did not want—he did not want to be portrayed as a happy slave," Stauffer told NPR in 2015. "The smiling black was to play into the racist caricature. And his cause of ending slavery and ending racism had the gravity that required a stern look. And so he tended to confront the viewer, look directly into the camera."

By March 4, the website already had ten million photos uploaded and animated, Newsweek reports.

"Indeed, the results can be controversial, and it's hard to stay indifferent to this technology. This feature is intended for nostalgic use, that is, to bring beloved ancestors back to life. Our driver videos don't include speech in order to prevent abuse of this, such as the creation of 'deep fake' videos of living people," MyHeritage states.

Create your own 'Deep Nostalgia' portraits using the Smithsonian collections through the Institution's Open Access portal.

Elizabeth Gamillo | | READ MORE

Elizabeth Gamillo is a correspondent for Smithsonian and a science journalist based in Milwaukee, Wisconsin. She has written for Science magazine as their 2018 AAAS Diverse Voices in Science Journalism Intern.