[A new tool developed by researchers at the Gwangju Institute of Science and Technology (GIST) in South Korea provides a way to preserve sensory immersion and presence during a virtual reality experience while allowing the user to interact with objects in their physical environment, a potentially valuable alternative to image passthrough and removing the VR headset. Basic details are in this story from the Chosun Daily (the English edition of the Chosun Ilbo); more information is available in this published article:

Elsharkawy, A., Gim, B., Ataya, A., & Kim, S. (2026). SelfBlending: Artificial Intelligence-Driven Augmentation with Hand Interactions for Seamless Reality Blending in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 32(7), 6958-6971. https://doi.org/10.1109/TVCG.2026.3690947

The first author has also posted a 4:04 minute video on YouTube. –Matthew]

[Image: An illustration depicting the ‘Self-Blending’ technology that allows seeing real-world objects overlaid with the virtual world even while wearing a VR, virtual reality headset. Credit: Image generated by ChatGPT]

SelfBlending Keeps VR Immersion While Interacting With Real Objects

GIST’s SelfBlending Outperforms Passthrough, Headset Removal in Usability, Immersion Tests

By Song Hye-jin
June 25, 2026

A technology has been developed that allows users to easily pick up a coffee cup on a desk without removing a VR (virtual reality) headset resembling goggles. Until now, if someone wanted a drink while playing a VR game, they had to activate the “Passthrough” feature, which displays the real surroundings, or remove the VR headset entirely. This process could disrupt immersion in the virtual world.

The Gwangju Institute of Science and Technology (GIST) recently announced that Professor Kim Seung-jun’s team in the AI (artificial intelligence) Convergence Department developed an AI technology called “SelfBlending,” which displays only desired real-world objects in a virtual space.

SelfBlending is a technology that naturally overlays real-world objects onto a virtual environment. When users pre-select objects like a coffee cup, water bottle, or memo pad on a desk, the AI recognizes and tracks their positions. As the user wearing the VR headset reaches for an object, only that item appears in the virtual space.

Users can continue gaming or virtual tasks while checking only necessary objects like a coffee cup or water bottle, without displaying the entire surroundings on the screen. This allows them to maintain the virtual display while drinking coffee or taking notes, reducing immersion disruption.

The research team compared three methods—removing the VR headset, the existing Passthrough method showing the entire real environment, and SelfBlending—with 18 adults. The analysis showed SelfBlending received the highest scores in key evaluation categories, including usability and immersion. Participants rated SelfBlending’s usability as comparable to completely removing the VR headset. The research results were published in the international journal *IEEE Transactions on Visualization and Computer Graphics*.

Professor Kim Seung-jun stated, “SelfBlending technology presents a new way to interact naturally with real-world objects while maintaining VR immersion. It can be applied to various extended reality fields, including education, training, and remote collaboration.”

—
This article has been translated by Upstage Solar AI.