GroZi aims to develop computer vision-based technology to assist the blind with grocery shopping. Currently, blind patrons require a clerk to escort them around the store and pull items from the shelves. This can lead to a number of problems: for example, a shopper may intend to purchase shrimp flavored ramen, but instead the clerk may accidentally select shrimp picante ramen. While this example is trivial, it extends to other, more serious situations, such as diabetics or other customers with serious dietary constraints. This quarter we are working using image sonification and hand tracking to guide the user's hand to the product.
The Multi-touch prototype is based on FTIR (Frustrated Total Internal Reflection) technology. Lining a sheet of acrylic with 850nm infrared LED lights, we create a light plane. When the user touches the surface, the contact scatters the infrared light, creating very defined finger blobs for our IR specific PS3 Eye camera to recognize.
Subteams
- Image Processing: The Image Processing team converts raw camera data into segmented objects within the image via LabelMe, the open annotation tool. The Distance Transform algorithm is then performed on the segmented image to find the center of the object. The team is now in the process of modulating this procedure.
- Image Sonification: The Sonification team translates what the computer vision system sees into sounds that help describe the image and define its bounds. The team focuses on creating a comprehensive sound database in ChucK, a multimedia programming language.
- User Interface: The User Interface team aims to implement multi-touch gestures for effective and efficient object identification. The team is also constructing a new, low budget, multi-touch prototype that will produce more accurate finger blob recognitions.
Client
The National Federation
for the Blind's mission is to achieve widespread emotional acceptance and intellectual understanding that the real problem of blindness is not the loss of eyesight but the misconceptions and lack of information which exist. They do so by bringing blind people together to share successes, to support each other in times of failure, and to create imaginative solutions.