Jeff's Research Blog

Pay Enough or Don't Pay at All - Jeff's Research Blog

2012-07-04T15:45:11Z

I love social media. Not only is it a great place to rant about politics and share cute baby photos, but it's also great for answering questions.

Yesterday, I asked the following to Twitter and Facebook:

"looking for reference for 'ppl will do more for free as volunteers than they will if you pay them a low wage' -- know where this came from?"

I got a lot responses, indicating that this is a pretty well-studied question with a lot of different results pointing to the general idea. Thought it might be useful to list them here for other interested folks.

For my purposes, the following seem to be the best:

"Pay Enough or Don't Pay At All"

Uri Gneezy and Aldo Rustichini

(found by Kyle Murray and Eytan Adar)

"Financial Incentives and the 'Performance of Crowds''"

Winter Mason and Duncan Watts

(found by Clemens Drews, Walter Lasecki)

Other interesting pointers:

"For Best Results, Forget the Bonus"

Alfie Kohn

(found by Daniel Lowd)

Examples from Freakonomics, specifically the chapter about the Israeli daycare's experience with charging parents for late pickup of children causing more to be late.

(found by Dan Weld @dsweld, Matt Post)

Dan Ariely's work, for instance http://web.mit.edu/ariely/www/MIT/Papers/zero.pdf"How Small is Zero Price? The True Value of Free Products"

(found by Bertil Hatt @bertil_hatt)

Crowdsourcing in the Real-World - Jeff's Research Blog

2011-04-25T17:51:27Z

Sam White, an undergrad in my group, created the turk-powered bartender that you see below. The gist is that it's a machine that makes rum and Cokes, and Mechanical Turk workers get to decide how strong you'd like your drink. It's really well done (especially for a weekend project) and I'm sure it's a great conversation starter at parties. It also brings some of the unaddressed issues with human computation into the fore.

For instance: What happens if turkers serve drinks that are a bit too strong - are they culpable for bad things that happen? Do turk bartenders need to be licensed? If so, where -- does or should a turker in India serving real-world drinks need to comply with liquor laws in New York State? Do workers understand that the abstract notion of choosing an alcohol level has real-world impact -- aka, they are *really* mixing a drink that someone will consume? If they knew it was real, would they behave the same way?

I don't have answers to these questions, but asking them is becoming increasingly important as the crowd makes it into our everyday lives. In the meantime, please don't let turkers mix your drinks :)

My April Fool's Email to UR CS Department - Jeff's Research Blog

2011-04-01T23:51:02Z

Dear Colleagues,

After months of high-level discussion with university administrators,
I am happy to report that effective Fall 2011, all curriculum in
computer science (courses with CSC designation) will be required to
incorporate components on Human-Computer Interaction, hereafter
referred to as HCI. This decision was reached after extensive research
and discussion, which concluded that people (aka, humans) are in fact
one of the primary end users of all computing systems [1].

Although this decision will affect our broad offering of courses in
different ways, some examples include the following:

- Artificial Intelligence (CSC 242) will be required to include explicit modules on Intelligent User Interfaces and mixed-initiative interaction.
- Operating Systems (CSC 256) will devote at least 51% of classroom time to discussing Windows XP (the most popular human OS [2]).

- Advanced Algorithms (CSC 284) will include RTF (run-time feeling) analysis as a first-class dimension on which algorithms are analyzed.
- Human-Computer Interaction (CSC 212) will remain unchanged.

If you have any questions, please direct them to either Dean Clark (CC'd here) or me.

I look forward to working closely with all of you to implement these changes over the coming months and years.

Thanks!
Jeff

[1] UR Technical Report #04012011: http://bit.ly/ur-tr-04012011
[2] Web Statistics and Trends: http://www.w3schools.com/browsers/browsers_os.asp

Response to VizWiz Comments - Jeff's Research Blog

2011-02-07T21:45:56Z

Image by Witer via Flickr

Bjoern Hartmann's Crowdsourcing Seminar at Berkeley read my VizWiz paper today, and his students provided written comments. He asked me to share a response, which I did and which I have included below.

Looking back on VizWiz, I think what was interesting was that it showed (i) you can do crowdsourcing in something close to real-time (regardless of how you do it) and (ii) you can use resulting "deployable Wizard-of-Oz" prototypes to learn more about your population. Through the VizWiz prototype, we found and verified (Soylent ref?) a number of visual questions that blind people might want answered, and isolated problems that effective tools will need to address to make them feasible.

I think nearly all of Bjoern's students found such problems in reading through the VizWiz paper, which is great! I hope they'll be inspired to go solve them to make a VizWiz-like tool even better and more useful.

==Enabling Blind People to Take Pictures==

Blind people don't have nearly the trouble taking pictures that one might imagine. Think about all of the great non-visual clues that are available!

Nevertheless, here are a number of interesting approaches one might take to either help blind people take better pictures, or lessen the impact of poor-quality pictures. We explored some simple approaches in the paper (darkness/blur detection), and have expanded the capabilities a lot since then while working with some local blind photographers (who already take some pretty great pictures). Our current version gives users the option to record a video instead of still photos, although at the cost of latency to send a larger file. The best improvement so far came by simply upgrading to the iPhone 4 and its better camera and flash! To me, what I think is interesting about the VizWiz study is that it showed how far you can get with low-fidelity input and crowdsourcing, especially when the capture of that input is mediated by human intelligence (the blind user). Generally, what happens when a question comes back with an answer like "the picture is bad" is that the user will take another picture and ask again.

==Answer Quality==

Our mechanism for dealing with answer quality was to present multiple answers to users. Most strategies you might consider to ensure answer quality end up delaying the answer -- for instance, waiting for other users to verify. We decided to rely on the user to make sense of the answers, especially given that answers were correct the majority of the time. We actually saw zero malicious answers.

Depending on how you look at it, a correct (and quick) "there's nothing in this image" is a great answer because it signals that the person should try again. But, users wanted more interactivity. VizWiz highlights challenges that research could address in facilitating such interactivity -- you can get very interactive responses by pairing a user with a particular worker, but how do you keep that worker around for the whole interaction? What if that worker ends up not being a "good" worker? How do you pair a user with a group of workers and have the interactions still make sense? All great questions.

==Other Latency-Reducing Strategies==

What I like about this paper is that (I think) it introduces the idea that crowdsourcing could happen in something like real-time. I agree that strategies like signaling to workers when work is available may be more cost effective than keeping them busy-waiting, but busy-waiting ends up being cost effective if you have enough users. The additional complexity of the signaling system may not be worth it in the end. Yet another great problem to explore more.

Is Bing Copying Google? - Jeff's Research Blog

2011-02-02T16:37:27Z

Reports have surfaced alleging that Bing has been copying Google's search results. Google claims to have verified this with a "sting" operation in which they rigged their search engine to return a bogus page for terms for which no one would normally search, had some employees search for those terms using Internet Explorer with the Bing toolbar from home, and lo and behold Bing started returning the same bogus pages for the same meaningless keywords.

You can see the original article and Bing's response. Bing essentially admits that for users who opt-in, it records usage data (like clicks on Google) and uses that as one of many factors determining search engine results. Search engines for a long time have used people's observed behavior on search results pages to influence ranking -- if people keep clicking (and staying) on the result the search engine is returning 3rd for a query, then it might get promoted to 1st. This would be the first time that a search engine (has been caught) doing this with another search engine's results.

So, what's the difference? Google would have you believe that Bing is stealing their search results, and that's partially true. Search result quality can be measured along two axes, precision and recall. Roughly, higher precision mean better search results up top, and higher recall means the page you're looking for is more likely to appear in the search results at all.

In my opinion, what Bing is doing to up their precision is mostly legitimate. They're presenting users with a bunch of choices of links to click (which happen to come from Google), and using their behavior to influence Bing's sense of what makes a good search result for a query. But, a less morally clear side effect is to up their recall -- if a user clicks on a link that Google includes in their search results that Bing does not, then the sting would have us believe that Bing will add that URL to its results, thereby increasing its recall on Google's back. But, both Bing and Google include most popular web pages in their search results already, so what we're really arguing about is recall at the long tail of the web.

The long tail of the web are the massive number of web pages that are highly relevant to a small number of infrequently search phrases. Since these aren't popular web pages, many try to argue that they don't matter, but, in fact, these are precisely the pages over which the next search engine battle is likely to be fought. Bing and Google both have money, they can pay an army to crowdsource good results for the top 10k, 100k, 1m results. But, they can't pay for good results for the next 100m or 1b queries, so it might make sense to seed Bing's view of the long tail on Google's back - even if they get caught in a sting.

It will be interesting to see what happens here. Is your usage data on Google the de facto property of Google, or can you choose to donate (or sell) it to the likes of Microsoft? Does this come down to Microsoft attempting to use its dominance in the browser market to hedge in on Google's dominance in web search? Will Danny Sullivan discover that Steve Balmer is his long lost step brother in law? Only time will tell.

Summary of Current Research - Jeff's Research Blog

2011-01-03T20:22:47Z

I just submitted short descriptions of three research projects that undergraduates can get involved in -- I thought I'd post them here so you get an idea of some of the cool work we're doing at ROC HCI.

Real-time Human Computation
Over the past few years, human computation -- integrating the intelligence and decision-making skills of people in computational processes -- has been shown a practical means to add true intelligence to computer programs today. As an example, computer vision is difficult, and so it can make sense to have a computer program query humans out on the web when it needs information about an image, instead of trying to do this automatically. Research goals include (i) developing methods for quickly integrating the input of dynamic pools of workers into actionable decisions, (ii) designing and implementing toolkits that enable developers to easily include real-time human computation as part of their own programs, and (iii) devising methods for estimating the expected latency for answers from different sources of human computation from past experience. Students working on this project will participate in the design of methods for achieving effective real-time computation and contribute to an open source toolkit allowing others to use real-time human computation in their own projects.

Human-Backed Access Technology
The past few decades have seen the development of wonderful new computing technology that serves as sensors onto an inaccessible world for disabled people - as examples, optical character recognition (OCR) makes printed text available to blind people, speech recognition makes spoken language available to deaf people, and way-finding systems help keep people with cognitive impairments on track. Despite advances, this technology remains both too prone to errors and too limited in the scope of problems it can reliably solve to address the problems faced by disabled people in their everyday lives. A promising approach for enabling people with disabilities to take advantage of this technology now is to let the error-prone technology fall back to human-powered services when it fails. For instance, if an OCR program is unable to recognize text, it may query always-available workers on services like Amazon's Mechanical Turk. In this project, students will extend an iPhone application that we have created called VizWiz that lets blind users take a picture, speak a question, and receive answers back in less than 30 seconds from workers on the web. Students will add in new automatic services, such as OCR and simple computer vision components (color detection, darkness detection, etc), and enable questions to be sent to social networks like Facebook and Twitter. Students will need to address the research and design challenge of helping users decide where to send their questions based on dimensions such as latency, accuracy, privacy, and anonymity.

Cloud-Based Assistive Technology in the Classroom
Millions of students with disabilities in the United States use assistive technology programs to help them use computers and learn classroom material. These programs range from screen reader programs that convert the visual information on a computer screen to audible speech for blind people, to speech recognition programs that enable people with physical disabilities to control their computers, to reading programs that speak and highlight words as students read. A primary problem with this technology is that it is not available on every computer that students access, and, even when the technology is there, the specific settings and preferences of the students must be repeated. A promising solution to these problems is to host assistive technology in the cloud so it can be accessed from anywhere and from any device with a web browser. In this project, students will design and build web applications that can replicate the complex, multimodal transformations of traditional assistive technologies within the restrictive web sandbox, and investigate the potential of these web applications by disabled students in local schools.

The GOP's Science Witch-Hunt - Jeff's Research Blog

2011-01-02T18:57:41Z

I've been railing for weeks against the short-sighted and downright ignorant attack on science funding from House Majority Whip Eric Cantor -- check out the website where he's inviting the public to help slash projects they don't understand.

The man trying to undercut the USA's dominance in science and engineering.

Image via Wikipedia

Not understanding science leads to silly recommendations, often directly opposing your own goals. For instance, the following examples of wasteful spending provided by Republican Eric Cantor apply directly to defense: (i) computer models to analyze the on-field contributions of soccer players, and (ii) modeling the sound of objects breaking. Automatically determining contributions of soccer players is directly applicable to analyzing and understanding troop movements and battlefield dynamics. A member of my department at the University of Rochester has DARPA funding to understand player coordination in games of Capture the Flag, so the Department of Defense obviously gets the connection. Similar technology is all the rage in prison technology to help understand inmate movements, prevent violence before it happens, and protect guards. I'll leave it to the reader to figure out why scientists are modeling games first before moving on to real battlefield situations.

As for modeling the sound of objects breaking -- that's directly applicable to automatically understanding what's happening during battles, especially the small urban battles characteristic of today's military engagements. The wall above me was hit by a bullet, wouldn't it be great if I knew from the sound of the brick breaking what kind of bullet it was and where it came from?

What makes all of this even more ridiculous is the scale of the funding. Sure, a grant worth several hundeds of thousands of dollars seems like a lot. But, it's likely spread over several years, and provides funding and education to at least 3-4 people over that period. Most of the funding goes to students, who in exchange for their education do advanced work in the national interest for far below the going rate. The annual funding of NSF is less than $8 billion, which amounts to less than a month in Iraq, even by conservative estimates. But then, we got such a good return on investment for the $1 trillion we've spent there and are continuing to spend.

It seems that this backlash on science is part of a broader trend. Somehow the conventional wisdom has become that scientists are out of touch, flitting away tax dollars on meaningless projects. We as scientists may be partly to blame for this and should do more to connect with the broader community, but it doesn't lessen the importance of scientific research. The reality is that science is what made and maintains the dominance of the United States across everything from economics to medicine, and materials to the military. Without research funding for science, the Internet and Google may not exist. But, more likely, they would have been invented slightly later in Europe or China. For more of the historical importance of revving the science engine of the states, see George Will's Op-Ed in the Washington Post.

In closing, my statement to Congress:
"Keep science strong. Keep the U.S. strong. Don't succumb to short-sighted, ignorant pressure to gut innovation by eliminating science funding."

NFB Files Complaint Against Penn State University - Jeff's Research Blog

2010-11-17T16:18:08Z

The National Federation of the Blind (NFB) filed a complaint with the Department of Ed against Penn State University last Friday due to the inaccessibility of its technology properties (primarily its web site(s), speaker podiums, and debit cards). This is worth paying attention to for a lot of reasons. Foremost in most peoples' minds is that the NFB recently won

Image via Wikipedia

a large settlement (multiple millions of $$) against Target Corporation for the inaccessibility of their web site, but to me it's more illustrative of a broader trend that reflects growing frustration among people with disabilities on the lack of improvement in accessibility. For years, it seemed that education was the answer for making web sites accessible -- most of the accessibility problems are really easy to fix if they're done at the time the site is created, but developers just don't know that they're important. But, after years of this not working, and, after high-profile successes like the Target case, I think we'll increasingly see more cases like this.

The NFB states in its complaint that this problem isn't unique to Penn State, and having been at several universities myself over the past few years I can certainly confirm that. If the NFB is successful in its complaint (even in part), it will be interesting to see what changes occur at universities. There are a number of legitimately difficult problems to solve to make university web pages accessible -- first, they have a ton of legacy content that would be incredibly costly to fix up if it wasn't created correctly in the first place. Secondly, universities often use complicated (and universally reviled) third-party software packages to which they are all but locked-in -- the NFB lists ANGEL at Penn State; at the University of Rochester we use Blackboard (which was recently certified by the NFB - we joke it is now equally horrible for everyone); at UW they used an in-house project (that was actually accessible, and decently usable). Finally, there is not currently one (or even several) central authorities for university web pages -- at a minimum, each department and organization in the university has its own web presence. These are managed by companies, contractors, in-house people, and students, and so it's not always clear who to blame. Course homepages are even worse as many of these are created by the faculty themselves!

Increasingly, we're seeing people with disabilities resort to legal challenges to accessibility problems - I think this is a natural response to the dismal job we've all done in giving our best effort to create accessible web content. Universities are an interesting case because of the distributed nature of web production, and so it will be especially interesting to see the outcome of this case and what universities do in response.

Legitimately Pushing Google Voice Text Messages on iPhone - Jeff's Research Blog

2010-01-24T22:55:49Z

I really like Google Voice, but Apple refuses to accept it in the iPhone's App Store. Google provides a web-based Google Voice - the version just released today (1/26) looks pretty much like the Google Voice iPhone application that was rejected from the store. The problem is that the web page is pull-only - and without push you don't get notified of new text messages or voicemails - you have to actually visit the site. I set out to try to fix this, and came up with a solution that works really well and is completely legitimate -- no Apple approval required!

The summary is this - my Google App Engine application polls Google Voice every 15-20 seconds looking for new text messages. If a new text message is found, then the application sends a direct message containing the sender of the text message and the message itself to my Twitter account from a Twitter account that I set up for this purpose. I have EchoFon running on my phone, which does Push notifications (I needed the $4.99 Pro version of EchoFon to do this), so that when a new DM arrives my phone vibrates and rings.

The end effect is that when I get texts on my Google Voice account, my phone acts almost exactly like it would if I had received a text message from AT&T. I've timed the round trip and it's less than 90 seconds on average. I can respond to the message using the web-based version of Google Voice by clicking a link included with the message.

Because this uses Twitter, there's nothing Apple or AT&T can deny from the App Store unless they agree to deny all Push applications (instead of Twitter, these could be Facebook messages, emails, fake point-of-interest near your location) - so, you don't need approval to run it!

I'll update this post with details of this solution in a few days so you can get free Google Voice push on your phone too!

What if Science were like Banking? - Jeff's Research Blog

2010-01-16T18:00:38Z

Science Editor: Hey boss, we just got an incredible submission! It's titled "Merry-Go-Round Warp Drive" and it explains how to exceed the speed of light! See, all you gotta do is get the merry-go-round going up to like 99.9% of the speed of light and then make it really big. The outside part will then be going way faster than the speed of light.

Science Senior Editor: Who is this guy? What lab's he at?

Science Editor: That's the thing - this is a 12-year-old genius savant. He turned it in for his sixth-grade science homework.

Science Senior Editor: Buy the rights. In fact, buy the rights to the whole damn class' homework assignments - we'll bundle 'em all up into a special issue, maybe sell it to Nature!

Better yet, let's bundle 'em, then auction them off paragraph by paragraph!

Grad Student: But the articles won't make any sense then.

Dr. Bigwig, Ph.D.: Exactly! They'll be difficult to understand, causing everyone to believe they must be incredibly ground-breaking, driving up prices (and our reputations).

...a few months later in a lab near you

Grad Student: I've got this great idea for how to cure cancer. If we can get people moving up to the speed of light then they'll become super-conductors and we can zap them and all the free radicals in their bodies without hurting them. It could stave off or even cure existing cancers!

Dr. Bigwig, Ph.D.: I'm a bit skeptical - how would you get them up to the speed of light, isn't that impossible?

Image via Wikipedia

Grad Student: No! There's this new result out of McKinley Middle School that shows how to exceed the speed of light - we'd just have to spin them really, really fast and then zap them.

Dr. Bigwig, Ph.D.: You know, I've heard that there's a new result showing how if you spin something fast enough it can be made to move faster than the speed of light. We could do that...and then zap them.

Grad Student: Right that's what I ...

Dr. Bigwig, Ph.D.: Ok, then we're all set! What a great idea I had! I'm going to be famous!

Grad Student: But can we get IRB approve it?

Dr. Bigwig, Ph.D.: Ha! IRB's are a worry of the past - the past few administrations have realized that the IRB was stifling creativity - they still exist, but their responsibilities have been reduced to getting coffee for one another and hassling HCI students wanting to do surveys. They've realized that we wouldn't dare do something dangerous - it would hurt our reputations, so subject protection is self-moderating.

Grad Student: Great! So I'll schedule the first couple of participants and see how they react.

Dr. Bigwig, Ph.D.: That's so slow, I need pay-off now. Bring in 100 people, do it all at once, you'll save time and money.

Grad Student: You're so smart Dr. Bigwig, Ph.D. I'll schedule them all as soon as I can attach this platform to this blender motor and rig up this wire for them to hold to an electrical outlet.

...a few days later at a hospital near you

Resident: Doctor! We have just received 100 patients all with electric shocks, burns, and dizziness. I've quickly surveyed them and it appears the patient John Q. is in the worst shape - we should get him to surgery stat!

Doctor: Damn residents! Get this jerk out of here, bring in C. E. Oppenheimer, and tell that jackass to keep up with the latest medical research before gambling with people's lives.

Now C., I'm glad I was able to intervene and see you first - your chart indicates your high levels of cash - is there anything I can get you? Coffee, juice?

...at a conference on a small island paradise off Morocco

Dr. Bigwig, Ph.D.: "...and so after 2 years, of the 100 subjects in the experimental condition, all of whom were previously diagnosed with stage 4 cancer, NONE had died from cancer."

... later that evening

Dr. Bigwig, Ph.D. (on phone with dean): "The university has onlly awarded me 20 x current tuition for my annual bonus - this is simply unacceptable. My contract clearly states that I am to be awarded 30 x annual undergrad tuition. You know the Ivies have been calling me - if you want to remain keep the top talent, you need to reward me for my stellar performance."

Dean: "But Dr. Bigwig, you killed 100 people...I know you're worth 30 x annual undergraduate tuition, but you know the parents and alumni - they just don't understand. How about I give you 10 full undergraduate tuition tickets that you could sell to future undergraduates in a year after these crazy politics have died down?"

Stay tuned for next week's article --- what if banking were like science?

WebAnywhere Gets A New Voice - Jeff's Research Blog

2010-01-10T16:18:29Z

Check out the new voice on WebAnywhere. The voice currently being used is "Jennifer" from Ivona. The inclusion of this voice was made possible by a very generous donation from Ivona. It's incredibly pleasant, a real treat to listen to! If you'd like to use it for other things, Ivona sells a SAPI compliant version.

Look for great things in the future for WebAnywhere's voice. Ivona's donated a few other languages that are just as high quality, so we'll be rolling those out as quickly as possible. Also, we're still looking for a free, high-quality voice that we can use that can be sped up.

Apple Tablet Could Be A Kindle Killer - Jeff's Research Blog

2010-01-06T18:13:47Z

A lot of people have been talking about the rumoured Apple Tablet - a new large-screen, multi-touch device that Apple will introduce priced in between its iPhone 3GS and Mac Books. What I haven't heard is about how this might just be the perfect reading device, and one that can more than compete with the Kindle (whether or not it includes the fancy digital paper.) And the reason is accessibility.

Apple designs great products, and actually includes good access technology on them *for free.* Because the iPhone includes VoiceOver it's quickly becoming the most sought-after phone for blind people. OS X includes a version of VoiceOver that's a fully-functioning screen reader, as opposed to Window's Narrator which true to its name kind of just tells you the high-level of what's going on without letting you actually get much of anything done.

Wisconsin already dropped the Kindle due to its inaccessibility, and despite some rather vague claims of how they're going to hack in some basic accessibility features, Amazon has already shown that accessibility will take a back seat. Schools (secondary and post-secondary) don't want to deal with inaccessible products unless there's a clear advantage - I think this is one case where if you can read books on Apple's tablet they win. Granted there are cost considerations - but the Kindle's already pretty darn expensive for something that pretty much only reads books (most of which you have to buy for $10-20 a pop!).

We in the accessibility field often say that accessibility can differentiate products - I'm really interested to see if this will turn out to be a clear example of that!

Why Comcast Should Have Held Firm Against Fox - Jeff's Research Blog

2010-01-02T17:21:22Z

Image via Wikipedia

In the last few days, Fox was able to strong-arm a better deal with Comcast, likely upping the price paid to the broadcaster for each subscriber (likely in the middle of the $1 Fox demanded and the $0.30 Comcast offered). On the surface this seems ridiculous for a number of reasons - namely, (i) most subscribers should be able to get Fox for free over the air if they have the antenna and converter box and (ii) Fox somehow makes enough ad money to support local broadcasting stations all over the country, which can't be cheap. There's a good case to be made that by "standing firm" Fox is actually putting another nail in its coffin - or at least in the coffin of its current business model.

(This posts gets a little away from the accessibility core of this blog, but I'll wrap up with little blurb connecting it in.)

A building tsunami is set to wreck the current business models of the networks. Since the dawn of television, the networks have enjoyed a psuedo-monopoly on content delivery - it was just too darn expensive for small folks to deliver their own content. Cable has been ever-so-gradually chipping away at that advantage with the introduction of new channels, but they've locked us all in with packages that still only deliver less than 100 channels for a reasonable price. Those channels are also still burdened with having to deliver 24 hours of content every single day. What happens when there are millions of channels available, that may pop into or out of existence as they have content to deliver, for lower cost that the expensive packages we have now?

Before addressing that question, lets first ask what exactly Fox (and the other networks) offer us? On the one hand, they're the only ones big enough to afford to buy the rights to broadcast big events - sports, the Olympics, etc, and to some extent they're the only ones who can develop high budget sitcoms and dramas. But because of how networks work, those shows need to target a really broad audience and still make a strong showing among certain age categories that really drive ad revenues). How can the junk on YouTube compete? But then, if people were free to watch whatever they wanted, and not limited by the networks - how long would it be before the NFL decides they'd rather take the bigger piece of the pie by broadcasting games online on their own?

There will always be people, groups and organizations willing to support the creation of content - sports leagues of course, as well as someone to make the Bachelor and even (regretfully) the Real Housewives of Wherever Is Next. The biggest question remaining is whether the networks are needed in the middle. There's definitely a role for something like networks, but it's not clear you need the big dogs that the networks have become. You need someone to produce, market, and bankroll -- but you no longer need the final channel out to people.

That channel is quickly becoming the Internet - with MythTV, AppleTV, SlingBox, Boxee, and Netflix it's not clear you need the networks or cable television. Hulu is the network's attempt to remain in control, but it's simultaneously wets the appetite of more people for free, instantly available content and remains remarkably inadequate - too few shows and no live events. If the other networks follow suit, working out stronger deals with the cable companies, more people will disconnect their cable.

While the networks will remain, their importance will diminish. What will replace them are thousands of smaller "networks" that will help people make the sense of all the new content that they now have access to - something the networks used to do and something that non-networks can and will eagerly do. Putting content up is one thing, but the main functionality still lacking is a "playlist," something to help folks who just want to turn on the tube find interesting shows and not have to get up every 30 minutes to find something else. That is, I hypothesize that channels won't go away just because the networks have.

In summary, as the networks seek to capitalize on their own sense of self-importance, they are dooming themselves. Until a viable alternative to cable Internet comes along, the cable companies themselves have little to worry about - I have no doubt that they will find a way to offset declining revenues from the droves dropping cable television with new fees or services to help people watch Internet television. The networks will be the losers, and will be losers of their own creations.

Of course, all of this is speculation. I'm hoping to begin investigating the hypotheses suggested in this article soon.

Finally, I promised to connect this back to access. History has shown that large shifts in popular technology used to consume information often comes with access problems. The networks represented a small set of companies on which to focus complaints, and as powerful corporations public opinion could easily help bring legislation offering requirements. If millions of smaller entities replace the networks, it will be harder to regulate - who will provide captions? This is already a problem on YouTube and other video sharing sites.

But, the earlier we think about such issues, the better chance we have of heading them off before they become too big of problems to readily address.

ASSETS 2009 Round-Up - Jeff's Research Blog

2009-11-02T21:16:23Z

ASSETS 2009 happened just a few days ago, and it was once again a great showing of the exciting research happening at the intersection of computer science, human computer interaction, and access technology.

Some highlights (from my perspective, of course), were the best paper awards, who went to two really great papers. The best student paper (that I must admit I was a co-author on) was entitled "ClassInFocus: Enabling Improved Visual Attention Strategies for Deaf and Hard of Hearing Students" and was written by Anna Cavender, Jeffrey P. Bigham, and Richard E. Ladner. It introduces a really great system for enabling deaf and hard of hearing students to combine multiple sources of information all on a single screen in the classroom, and some novel notification strategies to help them know what to pay attention to when.

The best paper award went to a great paper entitled "Collaborative Web Accessibility Improvement: Challenges and Possibilities", that came out of the IBM Tokyo Research Lab and was written by Hironobu Takagi, Shinya Kawanaka, Masatomo Kobayashi, Daisuke Sato, and Chieko Asakawa. This paper highlights some introducing qualities of the Social Accessibility Framework that they've been developing and overviews their initial challenges and successes.

Response Time over 24 Hours - Jeff's Research Blog

2009-08-12T13:56:36Z

In my continuing effort to leverage Mechanical Turk to help with accessibility tasks, I've done another experiment on Mechanical Turk. The full results of the experiment are posted on MIT's Deneme blog.

The main results are that turkers seem to be around 24 hours a day and are relatively quick - took less than 100 seconds on average to get an answer back. Now, we'll have to see if we can play some tricks to make this even faster.

In related news, I heard a rumor from a reliable source that WebVisum might be responsible for at least some of those CAPTCHAs that you see posted on Mechanical Turk. This is somewhat surprising to me, since I would have expected them to use a free volunteer-based model. In any case, it's still not clear to me how it works.