Listen to This Holly, Jolly (and a Little Creepy) A.I.-Penned Christmas Song
A neural network at the University of Toronto wrote a holiday ditty based on an image of a Christmas tree
It may seem like some of the songs on Christmas-themed radio stations were written by robots, but at least one new holiday song actually was. An artificial intelligence program developed by computer scientists at the University of Toronto recently penned its own Christmas ditty based on a digital photograph of a Christmas tree.
"White Christmas" it is not. The computer-generated female voice sings about flowers on the Christmas tree and a blessing being the best gift while backed up by a repetitive piano soundtrack. There is also a disturbing moment of semi-self awareness (“I can hear the music coming from the hall.”) The song would be considered a good effort for a first or second grader, and it has a surprisingly stirring coda (“There are lots and lots and lots of flowers!”).
But the most important thing about the song is that it shows new potentials for artificial intelligence. “We are used to thinking about A.I. for robotics and things like that. The question now is what can A.I. do for us?” Raquel Urtasun, an associate professor in machine learning and computer vision the University of Toronto tells Ian Sample at The Guardian.
“You can imagine having an A.I. channel on Pandora or Spotify that generates music, or takes people’s pictures and sings about them,” Sanja Fidler, who also worked on the project says. “It’s about what can deep learning do these days to make life more fun?”
While the song may be rudimentary, the research behind it is not. It’s part of larger project teaching neural networks to create. For this basic holiday tune, Ph.D. student Hang Chu trained a neural network on 100 hours of digital music. The program was then able to come up with some rules and create its own beats and melodies, layering drums and chords overtop, Sample reports. He details the project in a paper on the preprint server arXiv.org.
Chu also trained the network on footage from the video game Just Dance, which taught it how to connect the movements of a dancing stick figure to its songs. Chu then fed the network 50 hours of song lyrics, which helped it build up a vocabulary of 3,390 words, reports Sample. In the final step, the network trained on a selection of pictures and their captions, learning to associate words with images. When fed a generic photo of a Christmas tree, it was able to create music and generate a song based on the image, an ability the researchers have dubbed “neural karaoke.”
It’s not the first time A.I. has helped produce songs. Olivia Goldhill at Quartz reports that in September Sony’s A.I. system Flow Machines gobbled up a database of pop music, then created a song called “Daddy’s Car” reminiscent of The Beatles and another called “Mr. Shadow” that sounds like Bing Crosby on acid. Unlike Chu’s A.I., however, the Sony compositions had a little help from a human composer who arranged the music and wrote some of the lyrics.
While the Christmas song probably won’t end up on the radio anytime soon, the tech behind it may one day end up under the Christmas tree. “Instead of buying a karaoke machine with certain tracks on it, you can create your own karaoke at home by throwing in some interesting photos and inviting the machine to generate music for you,” Fidler tells Sample. “I think it has endless possibilities.”