RCA students design phones that turn speech into physical sensations

Royal College of Art students Sandeep Hoonjan and Xianzhi Zhang have designed a pair of devices that allow people to physically experience conversations while apart during the coronavirus lockdown.

The project, called Feel the Conversation, comprises two handheld machines that connect to a phone and translate the user's speech into a series of simple haptic patterns based on their intonation and volume.

Innovation design engineering students Hoonjan and Zhang created the devices to connect people who have been physically separated as a result of situations such as the coronavirus lockdown in a "more intimate" way.

Two attachments, a brush-style blue silicone feeler and a piece of silicone shaped like a tongue, produce different sensations.

"A friend told me, 'people are missing the feeling of touch, I can't lean into the zoom window, or my phone to actually feel like I'm close to them'," Hoonjan told Dezeen.

"In conversation with colleagues, I found there is an increase in 'skin hunger', which is a real issue for mental health, as we are social creatures, and touch is a big part of that," the designer continued.

"This project allows people to feel close to others in a way that is real, by using their speech as a reference point, but adding the feeling of touch."

The up and down movement made by the silicone attachments is a more exaggerated version of a smartphone's vibration function, explained the designers. The feelers create a massaging effect that's "both pleasant yet exciting" when held to the user's ear.

Hoonjan and Zhang designed the additional silicone tongue to offer something more recognisably human.

"The up-and-down movement of the tongue is a caricature of real movements of speech, something that people came to love, as it represented a relatable embodiment of their conversational partner, even though they may be far away," they explained.

Connected to a phone via a 3.5 millimetre port, the device records the user's speech with an embedded microphone. The peaks are then detected in real time to determine the intonation of what is being said and the volume at which it is being spoken.

This is then mapped as either an upward or downward movement of the silicone feelers of the partner's phone – with loud speech generating larger movements  and quieter speech or whispers making smaller ones.

Speech with more rapid changes in intonation cause more rapid up and down movements, while slower speech makes slower movements.

The phones could also lower language barriers, said the designers. An identical but flipped pattern is produced from "hello"and "ni hao", which means "hello" in Mandarin Chinese.

While Feel the Conversation was designed before the coronavirus outbreak, Hoonjan and Zhang have received multiple requests to produce more of the phones due to the distancing measures that have been put in place as a result of the pandemic.

"We both work in interdisciplinary and multi-cultural spaces, where it can be difficult to convey the feeling of what you're trying to say to each other, especially while distancing," said Hoonjan.

"Sending audio through the telephone doesn't really cut it," he continued. "So we searched for another way to playfully express the feeling of a conversation between people, that maybe don't even speak the same language, share cultures, or just haven't had physical contact for a while."

"It came back to touch. Many haptic devices on the market were unsatisfying to us, mostly relying on vibration. But we wanted something more intimate and with a greater bandwidth of sensation."

"Users of the Feel the Conversation phones have reported feelings of intense intimacy with their partners despite the lack of real contact or similarities in language or culture," Hoonjan added.

The duo's current models can operate over cables up to 100 metres long without needing boosting. However, as the signal is digital, it can be adapted to suit any modern telecommunication method.

It completes all the audio processing before sending any data to the partner phone, meaning that only a small amount of data is required to send a haptic message.

Messages therefore can be sent across far distances without losing their clarity. In addition to this, because the data is in such a simplified form, it could be send via USB, WiFi, Bluetooth, or any other common method for digital data transfer.

As the designers explain, they are currently developing a wireless model that would work over the internet using existing telephone networks – like a mobile phone – to reach two people living in different countries.

They are also working on a second method that would connect the devices to the user's laptop or smartphone via the 3.5 millimetre headphone or microphone port, sending the data back and forth over a web connection.

In a similar design, Studio Tada has created a fingernail-mounted haptic device called Fulu that introduces the sense of touch to audio-visual interactions such as video calls and online gaming.

Presented as part of the VDF collaboration with Ventura Projects, the Bluetooth-connected, wearable device recreates sensations including stroking an animal's fur and touching someone's hand.