The Cleveland Museum of Art Wants You To Play With Its Art

It wasn't that it was the first museum to attempt to “go digital.” It was that the Cleveland Museum of Art did it in a meaningful way that went beyond augmented reality phone apps that, when you point your phone toward an object you like, superimpose an informative fact over the piece. CMA worked with creative firms, such as Design I/O, to create ARTLENS, a gallery that incorporates physical artwork with their digital representations in interactive, gesture-based games.

In a game called “Line Shape,” for instance, you create a squiggle with your hand movements, and ARTLENS will automatically locate similarly shaped details within the objects of CMA's collection. Say you draw a little corkscrew in the air, and Line Shape will find and display sculptures, paintings and more that feature a corkscrew as a design element, even if it’s impossibly small and unnoticeable.

“We find that play is a very uninhibiting way to engage people to talk about art and learn about art,” says Lori Wienke, CMA's associate director of interpretation and one of the creators of the ARTLENS gallery that debuted in September 2017.

ARTLENS' predecessor was 2013's prototype Gallery One, in which more than 4,000 pieces of CMA's artwork then on display were also digital thumbnails on a 5x40-foot touchscreen of more than 23 million pixels. Patrons touched a thumbnail and it blew up large with a description, or searched by theme (melancholy, or shades of blue, etc…) to find related artwork. Nearly every major museum in the world visited CMA to study how the future of their museums would incorporate digital.

Gallery One was always meant to be a prototype. Its touchscreen wall was carried over into ARTLENS, a permanent gallery that replaced it; only the radio-frequency connection to CMA's smartphone app, now called ARTLENS app, was replaced by Bluetooth. The big change was the new ARTLENS exhibition, where 20 pieces of artwork are physically on display. Every piece has two interactive games. In a game called “Zoom Wall,” your body becomes the pinch-and-zoom you use to manipulate the artwork. Walk to one side, and it enlarges the piece. Backtrack, and it zooms out. When you bump into another visitor, two bodies become one control and it enlarges the artwork together.

Jane Alexander, CMA's chief information officer who oversaw ARTLENS' technologies, talks about patrons who'd come as strangers and often end up laughing together and sharing control in the interactive games. “People are truly engaging together, of all ages, without knowing each other,” she says, “and that's not what happens at museums.” Exhibitions in ARTLENS will rotate every 18 months to keep up a steady flow of new pieces. The back-end software that runs the games was built from the start to allow creation of new games for incoming artwork without the downtime typical to exhibit changes.

/

/

/

/

/

/

/

/

/

/

/

“What we're trying to do with this current iteration is to shift people’s thinking about the idea of play, to take play seriously,” Wienke says. “The way the games are structured is to allow people to absorb information more intuitively, without being hit over the head that they're learning something specific.” The 4,445 pieces of artwork currently on display throughout the museum also exist digitally within the ARTLENS app. A few light-sensitive pieces of art, kept in storage, are also featured. “Everything's out there, but we don't use technology for technology's sake,” says Alexander. “It's always about, 'Is this a tool that should really be digital?” Sometimes an analog tool is going to be much more helpful to an audience.”

CMA learned from Gallery One that visitors would access the touchscreen and peruse the physical pieces, but seeing one didn't lead them to the other. Alexander and Wienke wanted the two to complement each other, for notice of one to lead the visitor directly to the other. “For this space to be successful, people weren't going to say 'Wow, this is a cool touchscreen,' or 'This is a cool piece of technology,” says Alexander. “They were going to say 'Wow, this is an amazing collection.’”

For ARTLENS, everything was reorganized from Gallery One so that pieces are exhibited in the center of the gallery, and all the digital interaction technologies surround them on the periphery. When a visitor sees a piece projected three times its size, its real counterpart is close by. It's reflected in how CMA set up its eye-gazing station, in which you look at a piece of art for 15 seconds as scanners track your gaze. Afterward it tells you information about which parts of the piece you looked at, and compares it to how other visitors viewed the same piece. “It really slows down the time of looking at art,” Wienke says, “and makes you understand this experience of looking can be much more intentional and revealing about what sorts of parts of artwork are drawing visitors in.”

All 14 games at play and the app collect this sort of analytical information on how people digest the artwork. “The average time people looked at our artwork was two to three seconds,” says Alexander. “People are now looking at it for 15 seconds, and I'm really excited to see when they go into the gallery how long they look at the physical [counterpart to the digital representation they just saw].” CMA is only just beginning to dig into those analytical numbers to evaluate ARTLENS, and it'll be around September 2018 before Wienke feels there's enough data to begin drawing conclusions about the effect ARTLENS has had on keeping visitors engaged. As of yet, early indications are that ARTLENS is succeeding in creating long-term patrons who return regularly. “I'd say 50 to 60 percent of our visitors are repeat visitors,” she says.

Ninety percent of CMA's collection is in storage or on loan at any given time; there simply isn't the space to display it all at once. Museums typically have 10 times, or even 20 times, as many pieces as they have room to display, so the vast majority of the world's art sits in storage and waits for rotation. Sometimes it takes years to bring a piece into the light. Evolving displays and scanners could let museums in the near-future share digital scans of art and share them with other museums simultaneously, so that a Picasso physically in Paris' Louvre could be seen at the same time digitally in Philadelphia, Brisbane and Cape Town museums. Too, it could be a boon to small-town art museums that have neither the space nor the money to receive great works or display gigantic collections. “We're trying out new tool sets so museums are not all recreating the wheel,” says Alexander.

“It is vital that museums enter the digital age,” says Julian Spalding, an art critic and former director of Glasgow's Kelvingrove Art Gallery and Museum. “If they don't, they will become beached whales. There are, however, real challenges in introducing digital technology into museums.” He goes on to explain that “parceling up aesthetic experiences into manageable, teachable units, such as composition, gesture, emotion, purpose and symbolism, can fall short of the wholeness of impact that is the unique attribute of any great work of art—the kick in the stomach that takes you by surprise.”

The decision to pursue gesture-based interaction was, in part, to keep visitors' attention on the physical objects. “Projection provides us with a way to intentionally bring the focus back to the actual artwork,” says Alexander. Still, she doesn't agree with Spalding's assessment that parceling up an artwork into its elements robs the viewer of the chance to create an intimate relationship with it. “However, in order to create these relationships, we needed to keep the spotlight both literally and figuratively on the art,” she says.

CMA is looking at sharing the ARTLENS app with museums across the world so that each museum would have their own module within the same app. All would scan their pieces into their modules to recreate what CMA has done in ARTLENS, but there are no concrete plans to digitally display artwork a museum doesn't possess. Still it's a peek into how ARTLENS' interactive games, often the first gesture-based technologies CMA's visitors have ever used, will complement the physical artwork we've collected in our time, rather than replace it outright as so many once feared.