Engineering Students Develop Apps For Visually Impaired

Drexel computer science senior Nate Vecchiarelli (right) was part of a senior design group that developed and implemented smartphone apps to aid visually impaired students at Overbook School for the Blind.

For people with visual impairments, daily activities such as reading a newspaper, texting or checking Facebook often require the extra step of sitting down at a computer or pulling out a text magnifier - a step that instantly signals their difference from those around them. A group of undergraduate computer science students at Drexel are now hoping to make these tasks as easy and common as using a smartphone.

VisAssist, a set of applications created by seniors Trevor Adams, Nate Bomberger, Tom Burdak, Shawn Busolits, Andrew Scott, Matt Staniewicz, Nate Vecchiarelli as part of the school’s capstone senior design program, introduces preexisting vision-assistance technology and improved social media interfaces to the mobile devices.

The computer science seniors, charged with solving an open-ended task, met with students, teachers and administrators at Overbook School for the Blind to learn about the current technology and gain insight about the shortcomings of standard smartphone interface for people with visual impairments.  

What the computer science students found was that, in addition to the rigors of daily reading, even the accessing popular social media applications via smartphones, can also be frustrating for individuals with visual impairments.

“From meeting with the students we realized that current accessibility for websites was inadequate,” said Nate Vecchiarrelli, a member of the app design group. “Many of the complaints centered around changing interfaces that were poorly designed for current screen readers. We decided to take an entirely new approach that focused around the paradigm of one view, one purpose. This enabled our user interface to be simple, intuitive and not overload the user with too much information.”

The group started by simplifying the keyboard to make it easier for users to type and send text messages. The program they developed, dubbed “Binoboard,” is a customized keyboard that arranges the alphabet in a binary tree format. Users can simply swipe left or right to navigate through the alphabet and symbols, rather than searching a full keyboard. It also gives auditory feedback, supports voice recognition and stores passwords and phrases to increase the efficiency of the keyboard.

From there, the VisAssist team set off to produce better social media smartphone interfaces for visually impaired users. The students targeted Facebook, Twitter and Wikimedia, three of the most frequently used social media sites, for visual tweaking.

One application, called “Constrastinator,” functions similarly to the current “reader,” a magnifying device widely used by people with visual impairments. The Contrastinator lets users to snap a picture or scan their mobile device over text, such as a newspaper or magazine, while the display alters the text to more contrasting colors and enlarges the type, making it easier to read.

“The reader is big, clunky, red and yellow and looks like a child’s toy and they don’t want to be out in the community using these devices,” said Dael Cohen, transition services coordinator at Overbook School for the Blind. “Everyone has a phone, so snapping a picture of something to view it in the Contrastinator app doesn’t look all that out of the ordinary.”

Students at Overbrook are now becoming the first group to put the apps to use, after lending their input to their design. The Drexel team returned to the school in May to help students install and use the software, as well as learning how it can continue to be improved.

“A lot of attention has been paid to accessibility to academic-related tools for typing, reading and using a computer, so typical phone use is not often considered when it comes to helping people with visual impairments,” said Stephanie Hays, a teacher at Overbrook. “Students tend to shy away from using anything that makes them look different than others.”

The work of this group of computer science students was recently recognized as Drexel’s top senior design project. The senior design program at Drexel is a three-course sequence intended to give students the experience of working in a professional setting. Groups of students work to solve an open-ended problem, while gathering information and learning via interactions with faculty, community members and representatives from industry and government who serve as team advisors. The sequence is open to seniors in any discipline, more than 500 seniors participated this year. 

Judges selected VisAssist as one of eight finalists from the field of 135 entries. After the group made a final presentation of the work to a panel of representatives from local industry, VisAssist was dubbed the overall winner.

“Our approach is a great starting point in helping visually impaired individuals be able to experience technology many people take for granted,” VisAssist group member Nate Bomberger said.  “We believe our software can be used to make other applications more accessible, and while there is a lot more to do in the accessibility domain, our team has demonstrated that engineers can make a huge difference for people with disabilities.”

The VisAssist suite of applications is available for free on Android devices via the Google Play market. The team is also working on moving to other mobile platforms, and have working prototypes for the Contrastinator application on Windows Mobile and iOS, Apple’s flagship mobile platform.