The Stack Archive

Augmenting haptic feedback in virtual reality with robotic proxies

Wed 1 Feb 2017

Pass the Mug virtual reality

A new approach to haptic interaction in virtual reality environments has been proposed by a group of researchers at New York University’s (NYU) Future Reality Lab.

The system, outlined in the paper Robotic Haptic Proxies for Collaborative Virtual Reality, seeks to improve the integration between the physical and virtual worlds by boosting haptic feedback and collaborative experiences, both remote and local.

By intelligently controlling mobile robotic proxies, the research team, led by Professor Ken Perlin, were able to provide haptic feedback to multiple users in virtual reality. Acting as proxies for real, physical objects/forces, the robots could provide a deeper level of immersion in virtual scenarios.

The study employed the Holojam Software Development Kit and OptiTrack tracking technology to detect the position and rotation of real objects in the environment, such as the user’s wrists or a table. With this information, the team could send control messages to the m3pi robots in real-time.

Screen Shot 2017-02-01 at 11.29.27

‘Clinking Drinks’ Scenario

In a series of tests, the researchers trialled the robotic technology across a number of applications. The first scenario, ‘Pass the Mug’, saw users push a ‘proximate’ mug across the table via a pushing gesture and pull a distant mug towards them with a pulling gesture. They could also motion in an alternate direction to slide the mug towards another part of the table.

“Users must first select a target by aiming their palm at an object – the object will shake to indicate it is being targeted – afterwards the object will glow to indicate it is under gesture mode. Users can…pick up objects and interact with them normally when they are close enough to reach,” the paper explains.

A further prototype used one-to-one mapping to simulate the effect of two users clinking their drinks together while in separate physical locations. In each local environment, the position of the user’s cup is governed by a non-robotic, tracked, physical object, and the force of the strike is simulated by a synchronized mobile robotic proxy.

Having returned positive results from the experiments, the team concluded that: “Using our system, several virtual objects can be represented physically by one or more robotic proxies, and multiple users can share touch on the same virtual object, or use gestures to command objects without touching them.”

The team suggested that while several methods have been put forward for stimulating the sense of touch in VR, it claims that these are not as mobile nor as lightweight as the robotic proxy system. In the future, the researchers plan to focus on deeper application interactions and expand hardware capabilities.


haptics news research Virtual Reality
Send us a correction about this article Send us a news tip