On the threshold of the avatar era

[From The Wall Street Journal]

OCTOBER 23, 2010

On the Threshold of the Avatar Era


In a garage in Palo Alto, Calif., in the 1980s, some friends and I were the first humans to experience becoming avatars—that is, movable representations of ourselves in cyberspace. Amazingly, all these years later, almost no one else has been able to experience a hint of what will be one of the great cognitive adventures of this century.

It has been possible for some years for visitors to theme parks to try out virtual-reality “rides,” but these don’t capture the experience. Becoming an avatar in virtual reality, as a full-bodied human (or even nonhuman), has the potential to be vastly more interesting and important than one would expect from a technological amusement. What is really going on is the opening up of a new frontier of human potential, which can be called “somatic cognition”—somatic meaning “of the body.”

I first became aware of somatic cognition while learning to improvise music at a piano. After enough practice, a moment comes when you notice that your hands have solved complicated puzzles of voice and harmony faster than your conscious mind can keep up. Fine basketball players, surgeons and pilots report similar moments.

In such cases, the human body is extended by physical objects that map body motion into a theater of thought and strategy not usually available to us. Objects like a basketball, a jet or a piano play a role in somatic cognition that is similar to the role of words in conscious thought.

The first hint that avatars could become the ultimate somatic objects came to me in the form of a software bug in that Palo Alto garage. I was inhabiting a humanoid avatar, but my hand was gigantic by accident. I bent my fingers and was suddenly engulfed by my fingertip.

The surprise was that I could still operate a huge hand effectively. This led to a question: How strange could an avatar get before a person could no longer control it?

The answer turns out to be an even bigger surprise. People can inhabit awfully odd avatars. One of the early avatars was a lobster—a creature with more limbs than a human. By mapping values from body poses, it turns out people can learn to inhabit other bodies not just with oddly shaped limbs, or limbs attached in unfamiliar places, but even bodies with different numbers of limbs.

This phenomenon is called “homuncular flexibility.” The homunculus is the mapping of the body into the motor cortex, which is a portion of the brain located approximately under the portion of the scalp that would be occupied by a Mohawk hairdo.

That the mapping of the homunculus could be so flexible as to adapt to non-human bodies was initially a shock, but a delightful one. The sensation of inhabiting a nonhuman avatar is a new kind of pleasure. Think about what it would be like to wear wonderful clothing, combined with driving a superb vehicle, combined with mastering an extraordinary physical skill. It is like all those things together, but more expressive.

When we can successfully inhabit a nonhuman avatar, we are exploring not only the brain’s deep history, but also the potential far future of all the creatures for which it is preadapted—what might happen in hundreds of millions of years. Becoming an avatar is a form of extreme time travel for the brain.

One of the principal impediments to becoming an avatar in full has been the physical instrumentation. For many years, you’ve had to put on a special body suit.

But at long last, it has recently become possible to measure what the human body is doing with a specialized camera, so that the body can instantly be mapped to an avatar without donning instrumented tights. The first example will be available to consumers this year: the Kinect camera for Microsoft’s Xbox. (Full disclosure: I do my work at Microsoft Research, which produced Kinect.)

As has always been the case for virtual reality, I expect the revelation of its true potential to take place gradually. The early years of popular access to avatar tracking will almost certainly not spark radical exploration of somatic cognition. Instead, people will turn into merely human avatars for playing sports, hunting dragons, settling into yoga poses and so on.

But then what? My favorite experiment so far involved turning elementary-school kids into the things they were studying.

Some were turned into molecules, dancing and squirming to dock with other molecules. In this case the molecule serves the role of the piano, and instead of harmony puzzles, you are learning chemistry. Somatic cognition offers an overwhelming emotional appeal for education, because it leverages vanity. You become the thing you are studying. Your sensory motor loop is modified to incorporate the logic of a science, and you develop body intuition about that logic.

No one knows how big a deal avatar-directed cognition will be. Will students routinely dance to learn chemistry in the future? Quite possibly. A student might also become a triangle to learn trigonometry, or a strand of DNA to learn about biology. Will professional nanotechnology engineers become molecular structures in order to refine them? Once again, it seems quite possible.

A natural question to ask is whether somatic cognition could do for the Internet what visual computing did for individual devices. For instance, could you map your body into your Facebook preferences, so that you could dance in order to set them (finally making them intuitive)?

There are vast problems to solve before we can see how far the somatic frontier of human cognition can go. It will be painfully slow work at times, but it is possible that we are waking up a huge part of ourselves that will expand what it means to think. What could be a lovelier adventure?

—Jaron Lanier is a partner architect at Microsoft Research, innovator in residence at the Annenberg School at USC and author, most recently, of “You Are Not a Gadget.”

This entry was posted in Presence in the News. Bookmark the permalink. Both comments and trackbacks are currently closed.
  • Find Researchers

    Use the links below to find researchers listed alphabetically by the first letter of their last name.

    A | B | C | D | E | F| G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z