User Input Interfaces in the Digital Age

Put a pen or a stick in a little child's hand and you'll never doubt the essential naturalness of handwriting, drawing and gesturing.  Nevertheless, the dominance of personal computers has made keyboard users out of all of us.

Now this is changing. Tablets and tablet computers designed to capture writing and sketching on their displays are increasingly prevalent. Digital pens that write on paper and simultaneously capture digital ink are being pushed by technology innovators such as Wacom and LiveScribe.  These devices are doing well on international markets.

User Interfaces intended for data input and gaming are moving in the direction of capturing three-dimensional (3D) gestures.  Starting in 2006, the Nintendo Wii has convinced millions that performing such gestures is more intuitive than pushing buttons and twisting knobs.  The sales figures for Microsoft’s Kinect and Sony’s Move are confirming this realization.

NaviScribe’s Contribution

Since 2002 the researchers at ESPi have been preparing for a transition of User Interfaces to true 3D.  Our 3D input solutions, called NaviScribe, are based on accurate full motion capture.  NaviScribe recovers movements of user-manipulated devices in up to six degrees of freedom (x, y, z, pitch, yaw, roll).  These six degrees of freedom, collectively referred to as pose, are expressed in stable world coordinates.  Conveniently, the same coordinates also describe the user’s familiar environment.

ESPi’s approaches build on optical pose estimation originally developed in computer vision.   Advances in electro-optics and a wise choice of features for optical tracking support efficient, rapid and low-cost pose recovery in stable coordinates (absolute pose).  For example, IR LEDs enable absolute pose recovery at frame rates above 100 Hz.  Edges of display screens also present high-contrast optical features that support robust absolute pose recovery.

Some NaviScribe units combine optical pose recovery and relative motion measurements.  The latter are obtained from inertial devices, such as gyros and/or accelerometers.  Fusion of optical and inertial sensors is advantageous in low-cost and high-speed systems.  Such systems support high precision 3D wands for TV controllers and gaming wands, as well as styluses sufficiently accurate to record handwriting.