Facial Recognition Software Is Helping Identify Unknown Figures in Civil War Photographs

A new facial recognition application pioneered by computer scientist and historian Kurt Luther peers into the past—specifically the American Civil War—to identify anonymous portrait sitters captured in thousands of photographs taken over the course of the bloody four-year conflict.

As Erica X. Eisen reports for Slate, Civil War Photo Sleuth (CWPS) is a three-pronged collaboration launched in August by Luther and his Virginia Tech students; editor Ron Coddington of Military Images; and Paul Quigley, director of the Virginia Center for Civil War Studies. The project, as Luther detailed in a 2017 article for Military Images, features a digital photo archive, research tools and a thriving online community.

Users can contribute their own images from personal collections or upload snapshots spotted in books, museums, cultural institutions, shops and miscellaneous sites across the world. These photographs then join thousands held in national and state archives accessible to the public, enabling CWPS to work toward its goal of becoming the world’s largest, most complete digital archive of identified and unidentified Civil War-era portraits.

According to Slate’s Eisen, CWPS’ software identifies up to 27 “facial landmarks” in every uploaded photograph. If participating sleuths want to learn more about a certain mystery figure, they can narrow down their search by filtering images for details such as unit rank and insignia (colonels fighting for the Union side, for example, wore distinctive shoulder straps with an eagle), photographer details, and inscriptions. Once the system gathers all known information, it cross-references the image with all of the photos in CWPS’ database (which includes 15,000 reference images already identified) to present potential facial matches and, if known, names.

Writing for Military Images, Luther says that the array of facial landmarks used to compare photographs enables CWPS to find matches even if a soldier’s facial hair changes or an existing snapshot captures him from a different angle. This feat is made all the more impressive by the limitations of Civil War images. By the start of the war, photographers were beginning to develop prints from negatives, a delicate process that nevertheless opened up the possibilities of the nascent medium. As Eisen of Slate notes, in addition to the quality and coloring of these images, there were an array of limitations that make it a challenge to identify historical photographs today. Take, for instance, the prevalence of thick beards and mustaches, which could obscure vital facial features.

CWPS has already identified more than 75 photographs and has hundreds more catalogued for eventual identification. The process of identifying unknown figures in Civil War-era photographs requires amateur detectives to draw on an arsenal of tools and skills: As Luther writes in a separate Military Images piece, researchers often augment print resources with a growing body of online data, including genealogical charts, military records and photographic archives, as well as tips offered by burgeoning communities of sleuthing enthusiasts.

Luther has set the highly ambitious goal of identifying every photo in the project’s database. While there are numerous difficulties associated with meeting such a goal, Luther embraces the challenge.

In 2013, he successfully tracked down a portrait of Oliver Croxton, his own great-great-great grand uncle. Describing the search in a 2015 column for Military Images, he summed up the mission driving CWPS, saying, “Every discovery has an impact.”

Correction, 11/21/18: This story has been edited to reflect the correct number of photographs that Civil War Photo Sleuth has identified.

Meilan Solly | | READ MORE

Meilan Solly is Smithsonian magazine's senior associate digital editor, history.