Eye-Tracking Tablets and the Promise of Text 2.0
- By Eliot Van Buskirk
- March 25, 2010 |
- 3:57 pm |
- Categories: Media
The best thing about reading a book on a tablet (so far) is how closely it approximates reading a “real” book — which is why the Kindle’s screen is matte like paper rather than luminescent like a laptop. Some (not all) fear for the demise of real reading and writing, but it’s more likely we’re really at the leading edge of an innovation curve that could breathe new life into the written word.
For example: What if those written words were watching you reading them and making adjustments accordingly? Eye-tracking technology and processor-packed tablets promise to react, based on how you’re looking at text — where you pause, how you stare, where you stop reading altogether — in a friction-reducing implementation of the Observer Effect. The act of reading will change what you are reading.
We noticed something similar happening with televisions, which will soon be capable of adjusting playback based on how viewers are moving, and it looks like text is headed in the same direction. In the above video a team of scientists at the German Research Center for Artificial Intelligence (DFKI in German) leash eye-tracking technology from the Swedish firm Tobii Technology to HTML, CSS and JavaScript code, creating what they call “Text 2.0.”
Their technology is capable of monitoring your eyes in order to define words if you stare at them puzzled, eliminating non-essential information when you’re skimming, helping you pick up exactly where you left off, swapping images based on what you’re reading, surfacing relevant reference materials and more, as reported by h+ magazine.
The above video was posted in October, more than three months before the iPad was announced, and since then more people began re-examining the tablet’s potential to enhance the reading experience beyond the Kindle’s implementation of the “book” experience. The first iPad nearly had a camera, according to some reports, and later models almost certainly will, given the tablet’s potential as a videoconferencing device. Reader-facing cameras, attached to large screens and coupled to processors capable of running eye-tracking software, now seem all but inevitable.
Apple reportedly purchased eye-tracking unit(s) from Tobii Technology, which also makes them for paraplegics so that they can control things they otherwise can’t. As for the software part of the equation, Apple has already filed a patent for “gaze vectors,” which arrange and modify content on a screen based on where a user is looking:
Enter the iPad.
With eye-tracking technologies already in Apple’s possession, e-readers more popular than ever before, and Apple set to unleash an e-reader that’s more computer than it is book, the stage is set for eye-tracking technology to appear on next-generation tablets. This could present incalculable opportunities to reinvigorate the written word, and become a key differentiator from print — a positive one.
Whether it catches on is another story.
Text 2.0 will no doubt draw its share of naysayers — some of the same people who decry electronic text as inferior even when it contains the same information as the dead-tree version. Granted, it might be a bit over-the-top to add contextual information about whales’ feeding habits to Moby Dick when your eye lingers on a certain passage for too long, and Text 2.0’s potential for abuse could be greater than its potential utility. But these are the same kind of best-practices challenges that taught us not to hyperlink every word on a web page, or send someone to a corporate site at the mention of any company name.
Produced in a tasteful, helpful way, Text 2.0 promises to resurrect book-reading in a new form. Before you take these as the self-serving written words of an e-reader junkie addicted to whatever new way there is to read, I should admit that I have yet to make the leap to an e-reader. Having grown up a bookworm in the ’70s and ’80s, I prefer the feel and even the smell of paper books — and the fact that I can just chuck one in my bag for whenever I have a spare moment, without worrying about batteries, theft or breakage.
If Text 2.0 becomes a reality, whether on the iPad or somewhere else, count me in. I’m not reading as many books as I used to, and publishing figures suggest I’m not the only one. To compete for our attention with other methods of storytelling, books need to evolve — especially for readers born more recently — and Text 2.0 is one way forward that appears to respect the written word, enhancing our ability to consume it rather than subjugating it to multimedia fireworks.
These are still early days for Text 2.0. The hardware is still bulky and expensive (the Tobii x120 pictured here costs tens of thousands of dollars, with the price depending on volume), but hardware prices drop precipitously when technology migrates from specialized fields, like helping paraplegics control things, to general fields, like reading.
To encourage software development in this area, DFKI posted a plug-in that e-book developers can use to experiment with the new format’s possibilities using eye-tracking hardware or a mouse pointer simulating the reader’s gaze (see video).
Traditionalists will surely wonder why words need to see us, and they have a point. But if one believes in the magic of prose, one shouldn’t have a problem with it becoming more magical, and that’s precisely what Text 2.0 promises to do.
See Also:
- Television Will Soon Watch You (for Instructions)
- Why E-Books Look So Ugly
- Panacea or Poison Pill: Who Gets to Decide About $10 E-Books
- E-Books Have a Future in iTunes
- Barnes & Noble’s Shiny, Sharing-Friendly ‘Nook’ eBook Reader
- TED 2010: Wired for the iPad to Launch by Summer
- Can Apple’s iPad Save the Media After All?
- The Key to Apple’s iPad? Uh-Oh, It’s Magic
I am serious that we are on the edge of such technology that the eye scanning for identity will really come about soon just like in the movies…this is just only a start. eh
Oh man, and you thought auto-formatting in Microsoft Word was annoying…
It’s one thing for corporations/government to know what books/mags/etc. you are reading, but for them to start getting metrics on EXACTLY where your eyes are looking and for how long?
Let the thought police games begin!
If this is possible then why not just use it for controlling the mouse cursor and make web browsing a lot easier? No need to call it text 2.0 or anything fancy. It’s basically a dynamic web page, with a mouse cursor controlled by eye tracking.
If and when there is something disturbing, in a book that I read; the last thing that I would like it for the book to go back and make me re-read it. It may seem like something that’s new and will revolutionize everything; then it just gets so annoying that you just turn it off and never bother with it ever again.
The future of Tablets?
1st Bet: Microsoft Courier Tablet
2nd Bet: Notion Ink Adam Tablet
3rd… iPad?..
But believe me, these tablets won’t replace your desktop nor your laptop… that’s for sure..
And of course your eye reaction, dilation, dwell, etc., to page features will be sent upstream to the ad people, and advertisements and various other flavors of political propaganda will be fine-tuned to squeeze just a bit more cash and political favor out of the people. Eyes glazing? Better auto-augment the visuals just a tad, maybe with a millisecond image flash of something provocative, maybe a favorite body part morphing into a coupe.
Very Nice!!
I think it is great!!!!! Especially for ADD or ADHD. I know from my own experience that bouncing dots and markers enhance the attention on a sentence. This could be a good thing to break the routine of reading a paragraph!!! I wish i had that movie text 2.0 for real in my browser (no need for eye tracker we can use a mouse), that could really help.
technology gets smarter and more active so humans can be dumber and more lazy
“Produced in a tasteful, helpful way, Text 2.0 promises to resurrect book-reading in a new form.”
-Yeah, that’s the caveat. The ratio of tasteful, helpful to obscene, unnecessary today is 1:100…
The most interesting application is get rid of the mouse, touch pads, and even multitouch and have a quicker more intuitive interface.
This could lead to a more interactive process of writing. Writers will be able to know what paragraphs their readers need to read more than once because they are too difficult or poorly written. Hey, the tablet could also read our faces as in Lie to Me! That will give writters a lot of feedback to enhace their books until it reaches perfection. I envision a future where ebooks will be updated regularly, as any other application.
I remember when Minority Report was a futuristic movie. Now it seems so old! Why is John Anderton waving his hands so much? He should just move his eyeballs to scroll through the screen, or focus on a windows (maybe frowning his forehead) to maximize it.
“I’m not reading as many books as I used to, and publishing figures suggest I’m not the only one. To compete for our attention with other methods of storytelling, books need to evolve”
You know, I stopped reading basically when I finished high-school & didn’t start again until a 3 month holiday in India with too much time on trains & buses. That was two years ago & ever since, I’ve plowed through about two books a month. There is nothing wrong with books in the first place we just need to make time for them. I love the idea of ‘text 2.0′ but ultimately, if you’re not reading books now I don’t think on-the-fly translation & place keeping will make you start tomorrow.
>>> “That will give writters a lot of feedback to
>>> enhace their books until it reaches perfection.”
Or the lowest common denominator.
$10 says it just ends up being another avenue for advertising, like those stupid webpages now that have random words highlighted that will send you to their advertisers. Pretty soon if your eyes linger on the word “socks” too long while reading Huck Finn, you’ll be quickly redirected to sockswarehouse.com which will meet all your sock needs!
.
Seriously though, I could see this being really useful in academia. Start reading a textbook, get stuck on a really tough paragraph, get some sort of pop-up help. It could even direct you to a Wiki-like service generated by other users who also got stuck in the same place.
.
@TheQuickBrownFox - We pretty much can do that already, but the bottom line is it’s not as good as you think. Our body has evolved to make our fingers pretty much the fastest, most sensitive, most precise, and shortest delay parts of our body. The neural pathways are such that your fingers can respond to something faster than you can realize what you’re actually responding to. You’re not going to get that same level of control from your eyes alone, and any attempt to make it the primary mode of interaction will quickly lead to frustration.
I see absolutely no reason for a healthy individual to need to use software that permits you to operate you computer hands free. I see it as a great opportunity for quadriplegics however. This would permit them to access technology in ways that are much more easy than what they have now. While the exercise of creating new ways to interact with your PC is a good thing and innovation ought to be encouraged, sometimes the result isn’t all that great for most people. This will be, forgive me, just like Wall-E with the obscenely obese people in floating chairs do nothing but increase in size.
Tyrell: Capillary dilation of the so-called blush response…fluctuation of the pupil…involuntary dilation of the iris.
Deckard: We call it Voight-Kampf for short.
Rachael: Mr. Deckard, Dr. Elden Tyrell.
Yeah, first used on those floating ads that will overlay wherever your eyes are looking !
I hope somebody develops a phrase-reading feature that lets me read a book phrase by phrase without having to move my eyes. That would increase people’s reading speed quite a lot. Phrase Flash, Phrase Streaming or something.
Besides AD-tracking, the obvious use of this is synching of soundtrack and ambient sounds to the reading. It would make the experience much more immersive.