Feeling objects that aren’t there

Imagine this. You wake up in the morning to the irritating buzz of your alarm. Instead of fumbling for a snooze button, you wave your hand in the air in the general direction of the clock. There, in mid-air, you find it: an invisible button. It’s an illusion you can feel, like a hologram for your fingers. One swipe at the button, and the alarm shuts off. You’re free to sleep for a few more minutes — even though you never touched the clock.

The science of touch is called haptics. Sriram Subramanian describes the floating alarm clock button as one example of how a new technology called “ultrahaptics” might be used. “It does seem a bit far-fetched,” admits this computer scientist at the University of Sussex in England. But, he quickly adds, such a device is possible. Researchers in his lab now create virtual, three-dimensional objects that people can feel.

The secret to their success — sound waves. Actually, it’s no secret. A growing number of researchers around the world are investigating how sound waves can be used to simulate touch. These sound waves are ultrasonic. That means they’re so high-pitched people can’t hear them. At the same time, they’re strong enough to put pressure on human skin and trigger the sensation of touch. Scientists can change the location and shape of a tactile (touch) illusion by adjusting the sound waves, focusing them on a particular spot.

Invisible technology

An alarm clock with a levitating snooze button is just one example. Tom Carter, an engineer, joined with Subramanian to launch a company called Ultrahaptics. Carter imagines a future in which people use electronic devices with a wave of the hand. He and other researchers say the touchscreens and keyboards on current devices are limiting. They wonder: Why can’t we use the air around our devices as another way to interact?

Their research points to a whole new way to use electronics. Drivers might control phones or radios by twiddling their fingers in the air — while keeping their eyes on the road. Video gamers could feel the imaginary worlds that they already see and hear in their games.

Hiroyuki Shinoda, an engineer at the University of Tokyo in Japan, has been studying haptics for decades. In 2008, he became one of the first people to use ultrasonic waves to float virtual objects in midair. Since then, he has looked for ways for real and virtual objects to interact. He thinks that ultimately, the approach could help people connect with each other. For example, the technology might simulate the sensation of touching another person — like holding hands.

Subramanian says the idea of floating, three-dimensional illusions can inspire the imagination. Even though he developed the technology, he’s confident people will find other creative ways to use it. Fellow scientists, entrepreneurs (AHN-trah-preh-NOORS) and politicians flock to his lab. And immediately they become inspired.

“Everybody comes up with their own uses,” says Subramanian. “It’s amazing.”

Sounds and solids

Sound travels through the air as waves. But these waves are not like the ones that move up and down through water. A sound wave is an example of a longitudinal wave. It is made up of a series of compressions — places where the air is pressed together. To understand how a longitudinal wave travels, stretch out a spring. Give one end a quick push and pull, first toward and then away from the other end. A compressed group of coils will move down the spiral. In a sound wave, air particles bunch together like those coils.

Anyone who has been to a loud concert knows about the connection between sound waves and the feeling of touch. A low bass note doesn’t only reach the concert goers’ ears — it also vibrates their bodies. Subramanian says the experience of feeling such low notes inspired him to investigate sound waves. 

The human body detects sound and touch in similar ways. Cells in the skin have nerve endings, called mechanoreceptors (Meh-KAN-oh-ree-SEP-terz). They detect pressure, which triggers the release of signals to the brain. The inner ear also has mechanoreceptors. Called hair cells, they convert sound into electrical signals that travel along nerves to the brain.

Whether a sound is high or low depends on how many waves pass a single point during a given time. This measurement is called frequency. The higher the rate, the higher the frequency. Sound waves that make high notes have a higher frequency than do those that make low notes. An average person can hear sounds up to about 20,000 hertz, meaning 20,000 vibrations per second. (As people age, that upper limit drops. So children and teens generally can hear higher pitches than can older folks.) Ultrasonic waves are frequencies higher than those that the human ear can hear.

Many devices use ultrasonic frequencies. Some cars have parking sensors that send out ultrasonic waves and detect those that bounce back to identify obstacles. Medical ultrasound devices emit high-pitched sound waves to peer inside the body and “see” things, such as a growing fetus.

Feeling without touching

Physicists have been exploring the physical feeling of sound waves for more than 100 years. When sound waves hit the skin, their pressure triggers the mechanoreceptors. But scientists have only recently looked for ways to use that knowledge in electronic devices.

Subramanian began thinking about using sound waves to control devices a few years ago. He had been working with touchscreens, which always feel hard under the fingertips. He and his colleagues wondered if instead, the screens could communicate with users before someone even touched the device. For example, people might be able to start a program by waving their hands in front of the screen — not touching it. That led him to think about using ultrasonic waves to float objects in the air around the screen.

He started telling other people. “They laughed,” he recalls, saying “This is crazy. It’s not going to work.” But Subramanian’s team did not give up. “Other people never believed in our ambitions,” he says. “But they could not give us a good reason why it should fail.”

About five years ago, while he was at the University of Bristol in England, Subramanian began working with Carter. At the time, Carter was a college student looking for an interesting project.

Subramanian, Carter says, “had this crazy idea that you could feel things without touching them.” He asked Carter to build a grid of ultrasonic transducers (Trans-DU-serz). These are devices that send out high-frequency sound waves. His goal was to use those sound waves to push small objects.

After years of work, the researchers found a way to focus the ultrasound waves. Their device used 320 transducers connected to a computer. That setup allowed them to tune those waves precisely and create the illusion of an object floating in space. They debuted their first ultrahaptic device at a scientific meeting in 2013.

Since then, Subramanian has continued to push the science forward. Last October, he and his team showed how ultrasonic waves could be used to levitate, move and guide small objects. They called their invention a “tractor beam” — an idea made famous by science fiction. Those beams were supposed to use energy to capture objects, such as enemy space ships. The new acoustic tractor beam instead acts more like invisible tweezers.

Carter left graduate school to run the Ultrahaptics company. Next he wants to use the technology to simulate the feeling of touching different textures. “We can tailor the sound waves to any type of vibrations,” he says. At one frequency, the sound waves may feel like dry raindrops falling on your hand. At a higher frequency, they may feel like foam.

“How do you feel anything? You feel it by sliding your hand across the texture,” he explains. “Your skin is vibrating in a pattern as you drag it across.” The idea, he says, is that “if we can work out those vibrations, we can start to recreate complicated textures like rough or smooth wood, or metal.”

A personal touch

In Tokyo, Shinoda and his team recently unveiled a system called the HaptoClone. It uses a similar technology for communication. The system looks like two bulky boxes, each big enough to hold a basketball. One box contains a real object. The other displays the object’s reflection. Thanks to a series of mirrors between the two, the copy looks and moves identically to the original.

Shinoda and his team also installed a set of ultrasonic transducers. These allow the real object and its copy to “communicate” by touch. For example, if a person pushes on the real object, it moves. And so does the copy. That’s obvious — and would happen for any reflection! But here’s the interesting part. If someone reaches into the box and pushes on the reflection, their hand will truly feel it, because of the sound waves. And when they touch it, the copy will move — as will the original. Any action done to one side happens immediately to the other.

For example, imagine that one side contains a real ball. Someone can push on the reflected image — and thereby also shove the original ball out of its box. If two people each stuck their fingers into the box, they would get the sense they had actually touched each other — even though it had been sound waves creating that illusion.

“In the HaptoClone, real interactions between real objects can be realized,” says Shinoda. He thinks such a system might be most useful for people who want to connect with each other. “Physical contact between people is very important,” he notes. “Whether it’s simply shaking hands or stroking a person’s skin.”

Touching is a kind of nonverbal communication. He says it sends messages unlike anything people can say with images or words. He imagines a device like the HaptoClone may, for example, help children feel closer to a parent who is far away.

“My mission is to help people who have lost something,” he says.

He’s still fine-tuning the HaptoClone. Right now, the device is far too bulky to sell to people to keep in their houses. He’s working to make it smaller and easier to use.

Physicists may have first connected sound waves to feeling a century ago, but these new devices are truly cutting-edge. They’re also the result of hard work — often requiring years of research and testing.

Carter says his company, Ultrahaptics, began with an uphill battle. “We spent 18 months with our device not working, in various forms,” he says. But the struggle was worth it. In fact, he thinks the technology is only possible because of the hiccups he and his collaborators encountered along the way.

“You learn best by failing,” he says. “The fastest way to learn is to try to learn, and fail, and learn how to fail fast. If you don’t try to do something, you won’t fail, and you’ll never succeed.”

Power Words

(for more about Power Words, click here)

acoustics    The science related to sounds and hearing.

clone     An exact copy (or what seems to be an exact copy) of some physical object. (in biology) An organism that has exactly the same genes as another, like identical twins.

compression    Pressing on one or more sides of something in order to reduce its volume.

engineer   A person who uses science to solve problems. As a verb, to engineer means to design a device, material or process that will solve some problem or unmet need.

entrepreneur    Someone who creates and/or manages a major project, especially a new company.

fetus   (adj. fetal)  The term for a mammal during its later-stages of development in the womb. For humans, this term is usually applied after the eighth week of development.

frequency    The number of times a specified periodic phenomenon occurs within a specified time interval. (In physics) The number of wavelengths that occurs over a particular interval of time.

graduate school    Programs at a university that offer advanced degrees, such as a Master’s or PhD degree. It’s called graduate school because it is started only after someone has already graduated from college (usually with a four-year degree).

hair cells    The sensory receptors inside the ears of vertebrates that allows them to hear. These actually resemble stubby hairs.

haptic    Of or relating to the sense of touch.

hertz    The frequency with which something (such as a wavelength) occurs, measured in the number of times the cycle repeats during each second of time.

hologram     An image made of light and projected onto a surface, depicting the contents of a space.

illusion    A thing that is or is likely to be wrongly perceived or interpreted by the senses.

levitation     The act of suspending or causing to float in air a person or object — seemingly in violation of gravity.

mechanoreceptor    Specialized cells that respond to touch.

nonverbal      Without words.

particle    A minute amount of something.

receptor   (in biology) A molecule in cells that serves as a docking station for another molecule. That second molecule can turn on some special activity by the cell.

sensor   A device that picks up information on physical or chemical conditions — such as temperature, barometric pressure, salinity, humidity, pH, light intensity or radiation — and stores or broadcasts that information. Scientists and engineers often rely on sensors to inform them of conditions that may change over time or that exist far from where a researcher can measure them directly. (in biology) The structure that an organism uses to sense attributes of its environment, such as heat, winds, chemicals, moisture, trauma or an attack by predators.

simulate    To deceive in some way by imitating the form or function of something. A simulated dietary fat, for instance, may deceive the mouth that it has tasted a real fat because it has the same feel on the tongue — without having any calories. A simulated sense of touch may fool the brain into thinking a finger has touched something even though a hand may no longer exists and has been replaced by a synthetic limb. (in computing) To try and imitate the conditions, functions or appearance of something. Computer programs that do this are referred to as simulations.

sound wave    A wave that transmits sound. Sound waves have alternating swaths of high and low pressure.

tactile    An adjective that describes something that is or can be sensed by touching.

technology   The application of scientific knowledge for practical purposes, especially in industry — or the devices, processes and systems that result from those efforts.

tractor beam   A device in science fiction that uses a beam of energy to move an object.

transducer   A device that converts a variation in a physical quantity, such as sound, into an electrical signal. It also can convert an electrical signal into a physical quantity.

ultrahaptics    A technology that creates virtual, three-dimensional objects that can be felt without being touched.

ultrasound    (adj. ultrasonic) Sounds at frequencies above the range that can be detected by the human ear. Also the name given to a medical procedure that uses ultrasound to “see” within the body.

vibrate    To rhythmically shake or to move continuously and rapidly back and forth.

wave  A disturbance or variation that travels through space and matter in a regular, oscillating fashion.

Word Find (click here to enlarge for printing)