[A clever developer has created what looks like an effective presence illusion in which a mixed reality user feels like they’re petting a (virtual) cat. The details are in the short story from Mixed below, including the potential implications for more realistic and compelling virtual pets; the short video described and embedded in the original story is available on X/Twitter, and a second one is available there as well. In a related story, a recent article in Nature’s Scientific Reports titled “Phantom touch illusion, an unexpected phenomenological effect of tactile gating in the absence of tactile stimulation” includes this summary:

“In this study we used immersive virtual reality to simulate scenes of self-touch with a tool without accompanying tactile stimulation. Our subjects reported a tingling sensation at the spots where the virtual stick touched their virtual hand. This happened despite the lack of any physical stimulation. Interestingly, the majority of the subjects reported feeling this phantom sensation when touching their invisible forearm too. We called this sensation ‘phantom touch illusion’ to relate to an apparently similar phenomenon reported in the VR community for social touch [9].”

The authors note that, “While our finding involves feeling touch that is not physical, it does not seem to relate to widely-described embodiment illusions, such as rubber hand [18]” and they go on to explain why. See the article and a press release from Ruhr University Bochum for more details. See also the Phantom Touch in VR Research website that showcases work by Sasha Alexdottir and others at Bournemouth University. –Matthew]

This is as close as you can get to petting a cat on Meta Quest 3

A VR developer shows in a video how passthrough, hand tracking, and simulated physics can create the illusion of petting a cat.

By Tomislav Bezmalinovic
October 23, 2023

Dennys Kuhnert posted a video of his augmented reality cat on Twitter last week. The virtual pet follows his hands with its eyes and meows and purrs as he strokes it.

The video shows the view through a Meta Quest 3. The color passthrough of the device shows the physical environment, making the experience more realistic.

Note how the virtual hands physically adapt to the cat’s body. Kuhnert is a hand tracking virtuoso and the creator of Hand Physics Lab, the first VR hit game for this new controller-less way of playing.

Anyone who has tried Hand Physics Lab before knows that a simulated physical resistance can trick the brain into thinking it is dealing with inert objects. The same could be true for the cat, which is really just air.

First mixed reality experiments

Attentive observers may wonder why Kuhnert is projecting skeletal hands onto his real hands.

The reason is that Meta Quest 3 does not yet support real-time dynamic occlusion, which means that digital objects like the virtual cat are always rendered in front of moving objects like your hands. This destroys the illusion of having a physical object in front of you. If you look closely, you can see that Kuhnert’s real hands disappear behind the cat. The digital skeletal hands from Physics Lab counteract this effect.

Kuhnert writes me that the virtual cat is not new and was implemented in Hand Physics Lab years ago (spoiler alert: the cat appears in the very last level).

The developer says the video shows one of his latest mixed-reality experiments. And he can’t say where it’s going yet. “I’m primarily experimenting and prototyping with AR since a few months,” Kuhnert writes to me. “Real-time hand occlusion for AR and depth sensing are the next thing I will experiment with.”

Mixed reality Tamagotchis are coming soon

Virtual pets could be an interesting new use case for mixed reality headsets. A group of Oculus veterans have even started a studio dedicated to creating digital companions.

“Modern game AI is capable of engagement and responsiveness far beyond what Tamagotchi offered, and modern game engines can deliver performances far more nuanced than Sony’s robot dog — but until now, a truly believable pet sim was missing something,” the studio writes. That missing element is virtual and mixed reality. The technology, the startup hopes, will make the virtual companions so realistic that users will develop an emotional attachment to them.

We’ll have to wait and see. After all, you have to wear a headset to see the pets. But they should be enough for a fun pastime, and not a hindrance if you are allergic to cat and dog dander.