Controlling Computers With Hand Gestures

For awhile now gesture control systems have been used to control game consoles and TVs by gesturing has been available, and while there have been some great gadgets like the Natural Point TrackIR that enable head movements to control games, it's only recently that motion control devices have begun to make their way into desktop and laptop computer systems that allow you to control your computer with your hands by making gestures and without using a mouse or keyboard.

The 2002 Movie - Minority Report has a now famous scene showing Tom Cruise showing off a gesture controlled computer, shifting through images and data just using his hands, virtual reality fans have dreamt about different ways of controlling computers, very very soon there will be a wide range to choose from and the ability to control computer with gestures will be ubiquitous.

Initially these devices were invented to help people with spinal injuries or suffers of motor neurons disease that left them without the ability to use their hands, however the benefits that these devices can bring to everyone are quite astonishing, they are set to completely revolutionize how we interact with computers.

Now companies such as AMD are building the technology to allow facial and hand recognition into their chipsets, and Microsoft want to miniaturize and embed its Kinect technology into everything from your kitchen worktop and surface tablet, which means that 2013 will see and explosion of gesture control computer applications.

Just recently Samsung announced it was building "Samsung Eye Scroll" into its next generation of Smartphones, already the Galaxy S3 can tell if a user is looking at the screen and keep it illuminating if it is, however in the future users will be able to scroll through the screen just by looking at it!

Leap Motion Controller image Courtesy of Amazon.co.uk

There are a growing number of players in a market that is rapidly hotting up, at present there are three approaches being taken by the main developers that provide motion control for computer applications.

1. Software Only - products such as those from Point Grab rely on the 2D camera built into many laptops, smartphones, or an add on web camera, to provide computer hand control.
2. Wearable Sensors - there have been gloves and there are headsets such as the Oculus Rift that can sense movement and translate that into computer control, the latest is the MYO from Thalmic Labs. While the MYO is able to sense movement, it also can measure the electrical activity in your muscles, allowing for very complex hand gesture computer control.
3. 3D spatial cameras - The Kinect, Leap Motion and the Camboard Pico all work by using a camera or cameras to sense the surroundings, detect and recognise movement.
4. A hybrid is the Natural Point SmartNav and TrackIR, these rely on the user wearing a small reflective dot or a transmitter attached to a hat, heatset or other device to communicate with a receiver connected to your PC.

The Leap Motion Natural User Interface (NUI) or Gesture Recognition technology was announced in May 2012 and is expected to ship May 13 2013 and gives both Mac and Windows Gesture Control.

Pre-orders are being taken now, go to the Leap Motion Website for more.

If you are in the UK you can also pre-order from Amazon.co.uk

Leap Motion Controller

Leap gesture control is provided by a small, silver box that is connected to, and powered via a standard USB connection, although plans are in the pipeline for a wireless version. It is able to sense individual fingers in both hands from about 1 metre away. Leap claim that the Motion has an accuracy or precision down to 0.01mm, making it easily able to detect the tip of a pen or stylus.

What gives the Leap Gesture Controller a significant advantage is the relatively low price for such a cutting-edge gadget and the >$12m investment it has received that has allowed it to distribute "thousands" to developers to ensure that as much software as possible is available on the consumer launch.

Also launching in May is the Leap Motion App Store - Airspace, which will allow you to quickly find applications that make use of this incredible technology.

"Unleash your inner Jedi" - Thalmic Labs

At first the idea of wearing an armband to control your computer seems a bit clunky, however, because the device picks up minute electrical signals in your muscles, down to the movement of an individual finger, and because the electrical impulses begin even before your arm moves, the response time promises to be pretty quick. Also there is no need to map and analyze a visual image like some other devices, which again could give the MYO a distinct advantage over the competition.

The MYO communicates with devices using Bluetooth 4.0 Low Energy, power comes from rechargeable Lithium Ion batteries and computing power is provided by an ARM processor. Thalmic have developed their own muscle electrical activity sensing hardware which is partnered with 6-axis motion measurement.

Pre-orders are now being taken on the Thalmic Labs web-site.

The Camboard Pico is even more accurate than the Leap Motion, at least that is the claims. German company PMD Technologies has been making reference designs for some time now, and provides them to other manufacturers to integrate into their products.

You would expect something that has been in development for nearly 10 years to be pretty special. It follows on from their Camboard Nano, which was made available at a price of $690, although it is fair to say this was created as a pilot development rig and was never really intended to be in the hands of consumers, gamers and designers.

It remains to be seen if the Pico will ever be sold as a consumer item, but it's clear the technology will be influential and is very likely to appear in laptops and tablets soon.

The PointGrab solution is one of the few to rely solely on existing hardware, that is the 2D camera built into you laptop (it also works with standalone webcams). It also claims to work on tablets and smartphones that have a camera built in. However, as yet there is not date for it's release or price.

Ok so this, the original PC motion controller, isn't yet for you if you don't want to get your hands dirty coding and developing, however, it is the first commercially available gesture recognition device and there are plenty of great examples out on Youtube and the web to give you a good idea as to what is possible with this device.

Microsoft have, as of March 2013, released 22 examples of the underlying code as open source, presumably in an effort to increase usage of this device now that competitors such as the MYO and Leap Motion are getting closer to being released.

Books on developing natural user interfaces for gesture recognition projects.

AMD Face Login will allow you to use your face as a password, but the Chipset also has the capability to handle gesture control as well, with Microsoft also planning on making the Kinect hardware smaller and more easily incorporated into devices there seems little doubt that the technology will be embedded in everything from smartphones to car dashboards and performance art.

The MYO from Thalmic Labs - Jedi like control of your devices
The MYO has been designed to work with you Mac and PC. Anything you can control with a mouse you can control with the MYO, be that games, web browsing, music...

Virtual Reality Gaming Headsets
Ever since I made a special trip to London to play on the Virtuality Video Game in Covent Garden I've been fascinated by Virtual Reality, this was compounded...

Controlling Computers With Hand Gestures
For awhile now gesture control systems have been used to control game consoles and TVs by gesturing has been available, and while there have been some great ...