I just submitted short descriptions of three research projects that undergraduates can get involved in -- I thought I'd post them here so you get an idea of some of the cool work we're doing at ROC HCI.
Real-time Human Computation
Over the past few years, human computation -- integrating the intelligence and decision-making skills of people in computational processes -- has been shown a practical means to add true intelligence to computer programs today. As an example, computer vision is difficult, and so it can make sense to have a computer program query humans out on the web when it needs information about an image, instead of trying to do this automatically. Research goals include (i) developing methods for quickly integrating the input of dynamic pools of workers into actionable decisions, (ii) designing and implementing toolkits that enable developers to easily include real-time human computation as part of their own programs, and (iii) devising methods for estimating the expected latency for answers from different sources of human computation from past experience. Students working on this project will participate in the design of methods for achieving effective real-time computation and contribute to an open source toolkit allowing others to use real-time human computation in their own projects.
Human-Backed Access Technology
The past few decades have seen the development of wonderful new computing technology that serves as sensors onto an inaccessible world for disabled people - as examples, optical character recognition (OCR) makes printed text available to blind people, speech recognition makes spoken language available to deaf people, and way-finding systems help keep people with cognitive impairments on track. Despite advances, this technology remains both too prone to errors and too limited in the scope of problems it can reliably solve to address the problems faced by disabled people in their everyday lives. A promising approach for enabling people with disabilities to take advantage of this technology now is to let the error-prone technology fall back to human-powered services when it fails. For instance, if an OCR program is unable to recognize text, it may query always-available workers on services like Amazon's Mechanical Turk. In this project, students will extend an iPhone application that we have created called VizWiz that lets blind users take a picture, speak a question, and receive answers back in less than 30 seconds from workers on the web. Students will add in new automatic services, such as OCR and simple computer vision components (color detection, darkness detection, etc), and enable questions to be sent to social networks like Facebook and Twitter. Students will need to address the research and design challenge of helping users decide where to send their questions based on dimensions such as latency, accuracy, privacy, and anonymity.
Cloud-Based Assistive Technology in the Classroom
Millions of students with disabilities in the United States use assistive technology programs to help them use computers and learn classroom material. These programs range from screen reader programs that convert the visual information on a computer screen to audible speech for blind people, to speech recognition programs that enable people with physical disabilities to control their computers, to reading programs that speak and highlight words as students read. A primary problem with this technology is that it is not available on every computer that students access, and, even when the technology is there, the specific settings and preferences of the students must be repeated. A promising solution to these problems is to host assistive technology in the cloud so it can be accessed from anywhere and from any device with a web browser. In this project, students will design and build web applications that can replicate the complex, multimodal transformations of traditional assistive technologies within the restrictive web sandbox, and investigate the potential of these web applications by disabled students in local schools.
Real-time Human Computation
Over the past few years, human computation -- integrating the intelligence and decision-making skills of people in computational processes -- has been shown a practical means to add true intelligence to computer programs today. As an example, computer vision is difficult, and so it can make sense to have a computer program query humans out on the web when it needs information about an image, instead of trying to do this automatically. Research goals include (i) developing methods for quickly integrating the input of dynamic pools of workers into actionable decisions, (ii) designing and implementing toolkits that enable developers to easily include real-time human computation as part of their own programs, and (iii) devising methods for estimating the expected latency for answers from different sources of human computation from past experience. Students working on this project will participate in the design of methods for achieving effective real-time computation and contribute to an open source toolkit allowing others to use real-time human computation in their own projects.
Human-Backed Access Technology
The past few decades have seen the development of wonderful new computing technology that serves as sensors onto an inaccessible world for disabled people - as examples, optical character recognition (OCR) makes printed text available to blind people, speech recognition makes spoken language available to deaf people, and way-finding systems help keep people with cognitive impairments on track. Despite advances, this technology remains both too prone to errors and too limited in the scope of problems it can reliably solve to address the problems faced by disabled people in their everyday lives. A promising approach for enabling people with disabilities to take advantage of this technology now is to let the error-prone technology fall back to human-powered services when it fails. For instance, if an OCR program is unable to recognize text, it may query always-available workers on services like Amazon's Mechanical Turk. In this project, students will extend an iPhone application that we have created called VizWiz that lets blind users take a picture, speak a question, and receive answers back in less than 30 seconds from workers on the web. Students will add in new automatic services, such as OCR and simple computer vision components (color detection, darkness detection, etc), and enable questions to be sent to social networks like Facebook and Twitter. Students will need to address the research and design challenge of helping users decide where to send their questions based on dimensions such as latency, accuracy, privacy, and anonymity.
Cloud-Based Assistive Technology in the Classroom
Millions of students with disabilities in the United States use assistive technology programs to help them use computers and learn classroom material. These programs range from screen reader programs that convert the visual information on a computer screen to audible speech for blind people, to speech recognition programs that enable people with physical disabilities to control their computers, to reading programs that speak and highlight words as students read. A primary problem with this technology is that it is not available on every computer that students access, and, even when the technology is there, the specific settings and preferences of the students must be repeated. A promising solution to these problems is to host assistive technology in the cloud so it can be accessed from anywhere and from any device with a web browser. In this project, students will design and build web applications that can replicate the complex, multimodal transformations of traditional assistive technologies within the restrictive web sandbox, and investigate the potential of these web applications by disabled students in local schools.
Leave a comment