The TV show Star Trek: The Next Generation introduced millions of people to the idea of a holodeck: an immersive, realistic 3D holographic projection of a complete environment that you could interact with and even touch.
In the 21st century, holograms are already being used in a variety of ways, such as medical systems, education, art, security and defense. Scientists are still developing ways to use lasers, modern digital processors, and motion-sensing technologies to create several different types of holograms that could change the way we interact.
My colleagues and I working in the University of Glasgow’s bendable electronics and sensing technologies research group have now developed a system of holograms of people using “aerohaptics,” creating feelings of touch with jets of air. Those jets of air deliver a sensation of touch on peoples’ fingers, hands, and wrists.
In time, this could be developed to allow you to meet a virtual avatar of a colleague on the other side of the world and really feel their handshake. It could even be the first step towards building something like a holodeck.
To create this feeling of touch we use affordable, commercially available parts to pair computer-generated graphics with carefully-directed and controlled jets of air.
In some ways, it’s a step beyond the current generation of virtual reality, which usually requires a headset to deliver 3D graphics and smart gloves or handheld controllers to provide haptic feedback, a stimulation that feels like touch. Most of the wearable gadgets-based approaches are limited to controlling the virtual object that is being displayed.
Controlling a virtual object doesn’t give the feeling that you would experience when two people touch. The addition of an artificial touch sensation can deliver the additional dimension without having to wear gloves to feel objects, and so feels much more natural.
Using Glass and Mirrors
Our research uses graphics that provide the illusion of a 3D virtual image. It’s a modern variation on a 19th-century illusion technique known as Pepper’s Ghost, which thrilled Victorian theatergoers with visions of the supernatural onstage.
The systems uses glass and mirrors to make a two-dimensional image appear to hover in space without the need for any additional equipment. And our haptic feedback is created with nothing but air.
The mirrors making up our system are arranged in a pyramid shape with one open side. Users put their hands through the open side and interact with computer-generated objects which appear to be floating in free space inside the pyramid. The objects are graphics created and controlled by a software program called Unity Game Engine, which is often used to create 3D objects and worlds in videogames.
Located just below the pyramid is a sensor that tracks the movements of users’ hands and fingers, and a single air nozzle, which directs jets of air towards them to create complex sensations of touch. The overall system is directed by electronic hardware programmed to control nozzle movements. We developed an algorithm which allowed the air nozzle to respond to the movements of users’ hands with appropriate combinations of direction and force.
One of the ways we’ve demonstrated the capabilities of the “aerohaptic” system is with an interactive projection of a basketball, which can be convincingly touched, rolled, and bounced. The touch feedback from air jets from the system is also modulated based on the virtual surface of the basketball, allowing users to feel the rounded shape of the ball as it rolls from their fingertips when they bounce it and the slap in their palm when it returns.
Users can even push the virtual ball with varying force and sense the resulting difference in how a hard bounce or a soft bounce feels in their palm. Even something as apparently simple as bouncing a basketball required us to work hard to model the physics of the action and how we could replicate that familiar sensation with jets of air.
S...