This Device Translates Text To Braille in Real Time

In the wee hours of Valentine's day last year, a team of six women, all MIT engineering undergraduates, sat exhausted but exhilarated. Their table strewn with colorful wires, post it notes, food wrappers, scraps of papers, shapes cut from cardboard. This was no craft project gone awry. The team had just competed in MakeMIT’s hackathon—a competition in which teams of students spend 15 hours designing, coding, constructing, testing and debugging ambitious projects.

The women, competing under the team name 100% Enthusiasm, had set out to tackle a big challenge: accessibility for the blind. Their idea: a portable, inexpensive device that could scan text and convert it to braille in real time. It was something with potential to transform the lives of some of the 1.3 million Americans who are legally blind.

This first iteration was rough. Nearly the size of an adult’s hand, the mechanics of the device were sandwiched between two panes of plastic—wires and circuit boards exposed. Six pins poked up through the top of the device to display a single braille character (letter, number or punctuation mark). It imaged each character of text using an external computer’s webcam, rather than an internal camera as the team had hoped, explains Chen “Bonnie” Wang, one of the team members who is currently a senior majoring in material science and engineering. It was slow and not particularly portable. But it worked, translating text to braille. Team 100% Enthusiasm won.

The win was just the beginning of their work with the device, which they dubbed Tactile. Now, many prototypes later, the team has received another accolade. Tactile is one of nine winners for this year’s Lemelson-MIT Student Prize, which celebrates the translation of “ideas into inventions that improve the world in which we live,” according to the contest’s website. The winning inventions—a folding electric drone, proteins to fight superbugs, and a solar-powered desalination system for off-grid water production, to name a few—tackle a wide range of problems.

“We were super honored to be chosen as one of the winners of the award,” says Wang. The title came with a $10,000 prize that they are hoping to put back into the project to continue to improve how the device works.

The team’s latest prototype, about the size of a candy bar, can display six characters at a time (the average English word is roughly five characters long) and has a built in camera. Users can place it down on a line of text and with a push of a button, the device takes an image. Optical character recognition then takes over, identifying the characters on the page using Microsoft’s Computer Vision API. Then the team’s software translates each character into braille and subsequently triggers the mechanical system in the box to raise and lower the pins. They have applied for a patent for the integration of the system through Microsoft’s #MakeWhatsNext patent program, which supports women inventors.

“Currently the camera only takes a picture of its field of view,” Chandani Doshi, one of the team members who is majoring in electrical engineering and computer science, explains via email. “We are aiming to make the device similar to a handheld scanner that allows the user to scan the entire page in one go.” The idea is to make it as easy as possible to operate, preventing the user from needing to keep track of where they are on the page.

Though this is not the first real-time text to braille device, most products are based on digital text, like ebooks or pdfs—and they are extremely expensive. For example, the HumanWare Brailliant can connect to mobile devices and computers, allowing the user to type on the six-keyed braille keyboard and read using the one-line display of 32 characters. Prices for the device start at over $2,500. Also popular are what’s known as braille note-takers. These are like mini-computers, allowing word processing, the use of excel and powerpoint, and internet browsing. But these, too, retail in the thousands.

And a lot of text is not readily available in electronic format—menus, brochures, receipts, business cards, class handouts and more. Tactile would raise the text of these inaccessible documents right off the page. The team hopes to eventually sell the device for a maximum cost of $200.

One of the many challenges in development, however, is figuring out a better way to raise and lower the pins. In similar devices on the market, this has long been done using piezoelectronics—an expensive method that harnesses the properties of crystal structures. The team hopes to use microfluidics (differences in either liquid or air pressure) or electromagnetism (interactions of electric currents and magnetic fields) to move the pins. They are now testing both systems to figure out which is the least expensive, but most responsive and shrinkable for their final prototype.

Ultimately the team hopes that the final product will be slightly smaller than their current prototype and display two lines of 18 characters each. They hope to get it to market within two years.

“This opens up the world, really. What limitation is there if you have a device that would transcribe any document into braille?” the team’s adviser Paul Parravano, who has been visually impaired since he was three, inquires in a video about the device. “Suddenly the library is open.”

The question, however, is how many people will be waiting and ready to read the library. A commonly cited statistic is that less than 10 percent of people who are legally blind can actually read braille. Many people prefer to use text-to-speech technology and other audio-based programs, says Marion Hersh, a researcher who specializes in assistive technology at the University of Glasgow. Braille is challenging to learn and given the option, she says, many instead choose audio or even magnification (if they have limited eyesight).

It is important to note, however, that the braille literacy numbers are based on an outdated mode of measurement: supply of braille books from American Printing House for the Blind, explains Ike Presley, National Project Manager for the American Foundation for the Blind. “We definitely want to stifle that misconception that braille is dead and technology is putting braille out of business,” he says. “If anything, technology is making braille more accessible.”

The women of team Tactile are well aware of the statistic, but believe that part of the problem is the lack of inexpensive devices to make braille more available. The market for such devices is small, so few companies venture in with innovative ideas. “We don't have a Microsoft or an Apple ... the tech companies that make the tools for people who are blind or visually impaired are relatively small,” says Presley.

This means less competition, less innovation and higher prices. “It really drives up the cost, which limits access to braille even more. It's just a bad cycle,” says Wang.

“Whether this could encourage people who don’t already know braille to use it is open to questions,” says Hersh. But she notes that any new accessibility technology that combines low cost with ease of use could be extremely helpful in the market.

Learning braille means literacy for the blind community, says Presley, who helps train service providers so they can more effectively work with the visually impaired. Audio systems don’t provide the same understanding of language. “Auditory is great...but it doesn’t give you the literacy,” he says. “When you listen to [text read aloud], you don't know how to spell the words, you don't see the grammar, you don't see how text is formatted ...But when you read it in braille, you do.”

Studies also suggest that braille literacy increases both likelihood of being employed and an overall higher earnings potential for the blind and visually impaired—a group that has historically suffered high rates of unemployment.

These factors have only made team Tactile more determined to keep working on their product. All six engineers will graduate this June. But that isn’t going to slow them down. Three plan to continue working on Tactile, says Wang, and the others will continue part time.

“These women are on a great path, and as young as they are, if they can devote the next 20 years of their career to this, wow,” says Presley. “There's no telling what they might come up with.”

Maya Wei-Haas | | READ MORE

Maya Wei-Haas is a freelance science writer who specializes in geology of Earth and beyond. Her work has been featured in National Geographic, News from Science, and AGU’s EOS.