Imaginary Interfaces
Edited by Andy Ross
Researchers are experimenting with a new interface system for mobile devices
to replace screen and keyboard with hand gestures.
The
Hasso Plattner Institute (HPI) in Potsdam, Germany, is hosting a project
called Imaginary Interfaces (II) that uses a chest-mounted computer and
camera to detect hand movements. Thus users conjure up their own imaginary
set of graphical interfaces.
An II could allow people to use gestures
during phone calls, much as they do in face-to-face conversations, without
the need for clunky hardware.
"We definitely envision a system like
this replacing all input for mobile devices," said Sean Gustafson, a
research student at the HPI and lead author of an upcoming study on the II.
Many attempts to advance beyond keyboards and mice have focused on
gestures. But previous gesture-based interfaces have relied on a physical
screen as visual reference.
With II, there is nothing to see;
short-term visual memory instead serves as the reference. Users rely on
memory to "touch" these virtual elements.
"People are able to
interact spatially without seeing what it is that they create," said Patrick
Baudisch, a professor of computer science at the HPI and Gustafson's
teacher.
In generating the VR interface, the II device combines a
camera and a computer to see and interpret gestures.
At present, the
device is about 5 cm square and attaches to clothing on the chest. Its
makers envision shrinking it down to the size of a button.
An LED
ring around the camera beams out IR light. The camera sees the light
reflected by nearby hands but not the distant background.
To operate
an II, a user has two basic commands. Making out an L shape opens up a 2D
interaction plane. A pinching gesture at a point in the plane can then press
a virtual button at that point.
Other more sophisticated II
techniques are in the works. "We are exploring how users can sketch
interfaces, then use them," said Baudisch.
The II study will be
presented at the Symposium on User Interface Software and Technology, ACM,
New York, October 2010.
AR Wow, great! I didn't know about this, but it's a major element in chapter 0001 of my new book! Read it, guys, and be astounded at my prophetic powers.
