photo of two users' hands interacting with a touch-sensitive table using the Access Overlays tool to gain understanding of a large map accessible without sight

project areas

touchscreen accessibility

friendsourcing Q&A for the blind

image captioning

social tech for parents of children with special needs

social tech for autism

health info seeking

crowdworkers with disabilities

improving communication via AAC

accessibility & social technologies

overview

My research explores the relationship between accessibility and social technologies. In particular, I am interested in how interaction design can have a positive impact on people with disabilities by making it easier for them to connect with others in social and professional contexts. I am also interested in how interaction traces from social technologies (e.g., "big data") can provide insight into the activities, challenges, and successes surrounding technology use by people with disabilities, and/or provide early warning signs of health concerns. I participate in the MSR Ability virtual team within Microsoft Research, a set of people from across groups that work on projects related to accessibility. I also collaborate with MSR NExT's Enable group.

teaching & service

Together with Richard Ladner, I co-organized a joint University of Washington + Microsoft Research seminar series on the topic of "Computers & Disabilities" (CS590W) during the Fall 2010 academic quarter, and we also co-taught the same course at UW in Winter 2017. I served as a faculty panelist for the 2012 and 2017 ACM ASSETS doctoral symposium, and I was the local arrangements chair for the 2013 ASSETS conference. Richard Ladner and I are co-organized the 2016 UW-MSR Summer Institute on the topic of Expanding Accessibility Research. I am the papers co-chair for the 2017 ASSETS conference, and the papers co-chair for the "Health, Accessibility, and Aging" subcommittee for CHI 2018.

research projects

Touchscreen Accessibility for the blind

Friendsourcing Q&A for the blind

  • survey of blind people's use of social networking sites, and experiment comparing their use of friendsourcing vs. crowdsourcing for Q&A [ CSCW 2013 paper ]
  • analysis of visual challenges facing blind people based on questions posted to a friend/crowdsourcing service [ CHI 2013 paper ]
  • proposing and studying the concept of social microvolunteering as a mechanism for friendsourcing on behalf of altruistic causes [ CHI 2015 paper ]

Image Captioning

  • study of blind Twitter users' needs regarding image captioning, and recommendations on how to make visual media in social media more accessible to blind users [ CHI 2016 paper ]
  • A study of how automated image captions in social media are interpreted by blind users and how to better support appropriate user skepticism of incorrect captions. [ CHI 2017 paper ]
  • A study of how human-in-the-loop AI can improve the utility of automatically-generated image captions for users with visual impairment. [ HCOMP 2017 paper and appendix ]
  • introducing proxemic audio interfaces as an interaction technique for immersive and varied rich description of artwork for people with visual impairment. [ IMWUT 2017 paper & video ]

Social technologies for parents of children with special needs

  • study of Facebook and Twitter use by mothers of children aged 3 and under, including those with special needs/developmental delays [ CSCW 2014 paper ]
  • study of ways in which parents of children with special needs use social media, including suggestions for changes to social platforms to better support this demographic [ ICWSM 2014 paper ]

Social technologies for people on the autism spectrum

  • SIDES: a tabletop game for social skills development aimed at youth with Asperger's syndrome [ CSCW 2006 paper ]
  • SWARM: a wearable device to facilitate awareness of one's own or others' emotional states [ TEI 2015 paper ]
  • Supporting Neurodiverse Software Engineers in Technical Workplace Settings [ ASSETS 2015 paper ]
  • A study of how search engines and Twitter are used to seek and share health information, including disability issues [ CHI 2014 paper ]

Crowdsourcing employment and disability

  • A study indicating that people with disabilties view online crowdsourcing work as a potentially important source of employment, and identifying the technical obstacles to full participation in crowdwork by people with disabilities. [ CSCW 2015 paper ]
  • A study exploring senior citizens' understanding of and interest in crowd work, and reflecting on how crowd work systems might evolve to engage this audience. [ CHI 2016 paper ] [ HCOMP 2015 poster abstract ]
  • A framework for subcontracting microtasks that may benefit workers with disabilities. [ CHI 2017 paper ]

Improving communication via AAC devices

  • Reframing AAC as groupware and supplementing AAC with a mobile phone app (AACrobat) to facilitate communication partner awareness and engagement [ CSCW 2017 paper | Video Figure ]
  • Interview study identifying limitations of current AAC devices for personality expression and presenting design guidelines and evaluation metrics for more expressive AAC [ CSCW 2017 paper ]
  • Expanding the expressiveness of AAC via supplementary "awareness displays" for conveying emotion and/or communication logistics. [ CHI 2017 paper ]
  • Automating e-Tran board style input via a smartphone app for interpreting simple gaze gestures and translating them into text. [ CHI 2017 paper ]
  • Analyzing the limitations of eye gaze sensing for interactive applications and considering how applications can be re-designed to mitigate these challenges. [ CHI 2017 paper ]
  • Introducing the dynamic, cascading dwell technique for error-reduction and improved throughput in eye typing. [ CHI 2017 paper ]
  • Using computer vision to provide word and phrase suggestions to AAC systems that faciltiate discussing co-located objects. [ DIS 2017 paper ]