Thermal camera bracelet reads your wrist to track your fingers

[New Atlas reports on an innovative wearable design with several presence applications; the original story includes a 3:18 minute video (also available via YouTube) and more information is also available from Cornell University. The new paper includes this:

“Another interest[ing] application of FingerTrak in the future is to replace the glove or controller in [a] Virtual Reality setting, to free the hands in VR interaction. Moreover, FingerTrak can be used to control a robotic hand remotely and thus provide a novel means of human robot interaction where precise human hand manipulation is needed.” (p. 71:21)


[Image: A digital model overlaid on a real hand, showing just how well the FingerTrak system can predict the positions of a wearer’s fingers. At the bottom right, a robot hand copies the wearer. Credit: Cheng Zhang]

Thermal camera bracelet reads your wrist to track your fingers

By Michael Irving
July 22, 2020

Capturing the complexities of the human hand is a difficult task – just ask any artist or animator. Now, engineers at Cornell University and the University of Wisconsin-Madison have developed a new wearable system that uses thermal sensors to accurately predict hand positions, with potential applications in VR, robotics and translating sign language.

The device is dubbed FingerTrak, and it’s basically a bracelet adorned with four small thermal cameras, each about the size of a pea. From their respective positions, the cameras snap images of the contours of the wearer’s wrist. That’s enough for a specially-designed algorithm to accurately reconstruct the entire hand, including the positions of each finger.

“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are,” says Cheng Zhang, an author of the study. “It’s the first system to reconstruct your full hand posture based on the contours of the wrist.”

The FingerTrak system uses machine learning to predict the position of 20 finger joints, based on the wrist contours. These hand poses can then be recreated on a virtual model, or even a robot hand. In tests, the device was able to accurately reproduce actions like opening a book, writing with a pen, drinking, and using a phone.

There’s no shortage of devices designed to try to track your hand movements, using various technologies. Some watch your fingers using depth-sensing cameras or infrared sensors, some use motion-sensing gloves, and others use electromagnetic sensors on your fingertips. But almost all of them are a bit too bulky for practical use.

The FingerTrak is designed to be more lightweight, and while it’s still not the most comfortable-looking piece of tech, it does look to be on the right track (excuse the pun).

With a bit more refining, the team says that the FingerTrak system could have a range of uses. It could allow virtual reality players to track their own hand movements in-game, it could make for remote-controlled robots that directly mimic a human operator’s motions, it could help translate sign language into text or speech, or it could help monitor health issues that affect motor skills.

“How we move our hands and fingers often tells about our health condition,” says Yin Li, an author of the study. “A device like this might be used to better understand how the elderly use their hands in daily life, helping to detect early signs of diseases like Parkinson’s and Alzheimer’s.”

The research was published in Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies.

The FingerTrak can be seen in action in the video [in the original story].

This entry was posted in Presence in the News. Bookmark the permalink. Trackbacks are closed, but you can post a comment.

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


  • Find Researchers

    Use the links below to find researchers listed alphabetically by the first letter of their last name.

    A | B | C | D | E | F| G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z