January 2011 Archives

I just submitted short descriptions of three research projects that undergraduates can get involved in -- I thought I'd post them here so you get an idea of some of the cool work we're doing at ROC HCI.

Real-time Human Computation
Over the past few years, human computation -- integrating the intelligence and decision-making skills of people in computational processes -- has been shown a practical means to add true intelligence to computer programs today.  As an example, computer vision is difficult, and so it can make sense to have a computer program query humans out on the web when it needs information about an image, instead of trying to do this automatically. Research goals include (i) developing methods for quickly integrating the input of dynamic pools of workers into actionable decisions, (ii) designing and implementing toolkits that enable developers to easily include real-time human computation as part of their own programs, and (iii) devising methods for estimating the expected latency for answers from different sources of human computation from past experience. Students working on this project will participate in the design of methods for achieving effective real-time computation and contribute to an open source toolkit allowing others to use real-time human computation in their own projects.

Human-Backed Access Technology
The past few decades have seen the development of wonderful new computing technology that serves as sensors onto an inaccessible world for disabled people - as examples, optical character recognition (OCR) makes printed text available to blind people, speech recognition makes spoken language available to deaf people, and way-finding systems help keep people with cognitive impairments on track. Despite advances, this technology remains both too prone to errors and too limited in the scope of problems it can reliably solve to address the problems faced by disabled people in their everyday lives. A promising approach for enabling people with disabilities to take advantage of this technology now is to let the error-prone technology fall back to human-powered services when it fails. For instance, if an OCR program is unable to recognize text, it may query always-available workers on services like Amazon's Mechanical Turk. In this project, students will extend an iPhone application that we have created called VizWiz that lets blind users take a picture, speak a question, and receive answers back in less than 30 seconds from workers on the web. Students will add in new automatic services, such as OCR and simple computer vision components (color detection, darkness detection, etc), and enable questions to be sent to social networks like Facebook and Twitter. Students will need to address the research and design challenge of helping users decide where to send their questions based on dimensions such as latency, accuracy, privacy, and anonymity.

Cloud-Based Assistive Technology in the Classroom
Millions of students with disabilities in the United States use assistive technology programs to help them use computers and learn classroom material. These programs range from screen reader programs that convert the visual information on a computer screen to audible speech for blind people, to speech recognition programs that enable people with physical disabilities to control their computers, to reading programs that speak and highlight words as students read. A primary problem with this technology is that it is not available on every computer that students access, and, even when the technology is there, the specific settings and preferences of the students must be repeated.  A promising solution to these problems is to host assistive technology in the cloud so it can be accessed from anywhere and from any device with a web browser.  In this project, students will design and build web applications that can replicate the complex, multimodal transformations of traditional assistive technologies within the restrictive web sandbox, and investigate the potential of these web applications by disabled students in local schools.

Reviews coming back from papers and grant proposals of mine that use (or propose to use) Mechanical Turk have included a common sentiment:  Neat ideas, but what happens if Mechanical Turk goes away or if wages don't stay as cheap?

I've been railing for weeks against the short-sighted and downright ignorant attack on science funding from House Majority Whip Eric Cantor -- check out the website where he's inviting the public to help slash projects they don't understand.

Not understanding science leads to silly recommendations, often directly opposing your own goals. For instance, the following examples of wasteful spending provided by Republican Eric Cantor apply directly to defense: (i) computer models to analyze the on-field contributions of soccer players, and (ii) modeling the sound of objects breaking. Automatically determining contributions of soccer players is directly applicable to analyzing and understanding troop movements and battlefield dynamics. A member of my department at the University of Rochester has DARPA funding to understand player coordination in games of Capture the Flag, so the Department of Defense obviously gets the connection.  Similar technology is all the rage in prison technology to help understand inmate movements, prevent violence before it happens, and protect guards. I'll leave it to the reader to figure out why scientists are modeling games first before moving on to real battlefield situations.

As for modeling the sound of objects breaking -- that's directly applicable to automatically understanding what's happening during battles, especially the small urban battles characteristic of today's military engagements. The wall above me was hit by a bullet, wouldn't it be great if I knew from the sound of the brick breaking what kind of bullet it was and where it came from?

What makes all of this even more ridiculous is the scale of the funding.  Sure, a grant worth several hundeds of thousands of dollars seems like a lot.  But, it's likely spread over several years, and provides funding and education to at least 3-4 people over that period.  Most of the funding goes to students, who in exchange for their education do advanced work in the national interest for far below the going rate.  The annual funding of NSF is less than $8 billion, which amounts to less than a month in Iraq, even by conservative estimates.  But then, we got such a good return on investment for the $1 trillion we've spent there and are continuing to spend.

It seems that this backlash on science is part of a broader trend. Somehow the conventional wisdom has become that scientists are out of touch, flitting away tax dollars on meaningless projects.  We as scientists may be partly to blame for this and should do more to connect with the broader community, but it doesn't lessen the importance of scientific research. The reality is that science is what made and maintains the dominance of the United States across everything from economics to medicine, and materials to the military.  Without research funding for science, the Internet and Google may not exist.  But, more likely, they would have been invented slightly later in Europe or China. For more of the historical importance of revving the science engine of the states, see George Will's Op-Ed in the Washington Post.

In closing, my statement to Congress:
"Keep science strong. Keep the U.S. strong. Don't succumb to short-sighted, ignorant pressure to gut innovation by eliminating science funding."