Fast and Discreet access to web services for the Blind through Screenless Browsing

Abstract

Web services on our smartphones have become an integral part of our daily lives. Services like Google Maps and Yelp have helped us explore the world better. However, the blind and visually impaired (BVI) spend unnecessary cognitive and mechanical effort navigating complex menus displayed on a mobile device before they can locate and access the content of their interest. More direct access may happen via voice-based services (e.g., S iri ), but at the cost of breaking privacy and social boundaries. To combat this issue, we propose Screenless Browsing : combining hand gesture recognition with aural navigation patterns that enable the BVI to quickly navigate aural menus through nimble, discrete hand movements. We propose to decouple the friction-prone mechanical interaction with a mobile display from the navigation experience. We demonstrate our approach by: (1) Introducing novel aural browsing menus that combine web content with binary splitting, dynamic sorting and playlists to accelerate navigation across collections; (2) Mapping aural menu navigation to the robust and simple vocabulary of hand movements enabled by M yo , an off-the-shelf muscle-controlled armband; (3) Reify our approach by iteratively prototyping Screenless Browsing of mobile applications for the BVI; (4) Conduct a user study to assess the limits and potential of our approach with participants from the Indiana School for the Blind and Visually Impaired (ISBVI). We believe that the ability to access web services on the move without taking the phone out of the pocket will empower the BVI to navigate and explore places effectively. Our work exemplifies a novel way to to reduce unwanted friction with the device and maximize the content experience for the BVI.