Mobile virtual reality (VR) platforms, such as the popular Google Cardboard, are a low-cost way for anyone with a smartphone to experience the rapidly growing world of virtual reality.
But mobile VR has traditionally come with limited opportunities for interaction, and the most natural way to interact with a virtual world - using your hands - is difficult to implement on a smartphone because hand tracking requires expensive 3D sensors that are not available on current phones.
Now, researchers from the University of ÁùºÏ±¦µä, Reno have introduced a low-cost way to add hand tracking to mobile virtual reality using nothing more than a pair of ear buds, in a submission that won the "best note" award at the 2016 ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHI PLAY).
PAWdio, developed by doctoral students Majed Al-Zayer and Sam Tregillus, along with Eelke Folmer, associate professor of computer science and engineering, uses acoustic sensing to track the movement of a hand through space.
PAWdio emits a high-frequency sound wave that is inaudible to humans and then, using Doppler shifts, calculates the distance between the ear bud and the smartphone to find the position of the hand.
Low cost-solutions make mobile VR accessible
"One of the implicit missions of our lab is to bridge the gap between high-end but expensive and cheap but low-end VR platforms," Al-Zayer said. "Realizing this mission would make high-end VR technology more accessible to the masses. I believe that solutions like PAWdio demonstrate that getting the best out of the two platforms is possible."
In addition to being affordable, PAWdio doesn't require the extensive computing power that other approaches to hand tracking demand.
"There are a few approaches that use computer vision to track the hand but these are all very computationally intensive, which drains the battery, but also lowers the frame rate and increases latency, both of which are known to induce cybersickness," Folmer said.
Now, Folmer and his team are working on developing a math game that has children solve simple math equations by selecting objects in 3D using PAWdio. To make the game exciting to play, it borrows popular mechanics from games like Fruit Ninja and Candy Crush.
"My daughter always complains that using math games on her iPad is extremely boring," Folmer said. "Because Google Cardboard is extremely low cost and PAWdio doesn't require any extra sensors, this could be a low-cost solution to make learning math more fun."
The researchers presented the paper at the October CHI-Play conference, which brings together researchers devoted to better understanding human-computer interaction in digital games and VR.
