[A new “surreal” demonstration of the potential of artificial intelligence may portend future personalized presence experiences generated in real-time in games, on other entertainment platforms, and beyond. The IFL Science report below provides the details (see the original version for two videos), and it’s followed by excerpts from coverage in Wired and MIT Technology Review. –Matthew]

[Image: The game has been described as “kinda creepy”. Credit: Alainara/Shutterstock]

The World’s First AI-Generated Game Is Playable By Anyone Online, And It Is Surreal

The AI was forced to watch millions of hours of Minecraft footage, like a parent of any 8-year-old.

By James Felton, Senior Staff Writer
Edited by Holly Large
November 15, 2024

In news your 8-year-old kid probably knew about weeks ago somehow, a new Minecraft rip-off is available to play online. In a surprising twist on the genre, every frame in this one is entirely generated by artificial intelligence (AI).

The game, Oasis, lets players explore a 3D world filled with square blocks, mine resources, and craft items, just like the ridiculously popular Minecraft. It’s a little surreal to play, with distant landscapes morphing into other shapes and sizes as you approach.

But underneath the hood, the game is quite different from any other you have played. It lets you choose from a host of starting environments, as well as the option to upload your own image to be used as a starting scene. However, the player’s actions and movements soon change the environment around you.

“You’re about to enter a first-of-its-kind video model, a game engine trained by millions of gameplay hours,” the game’s creators, Decart and Etched, explain as you enter. “Every step you take will shape the environment around you in real time.”

“The model learned to allow users to move around, jump, pick up items, break blocks, and more, all by watching gameplay directly,” the team explained in a statement. “We view Oasis as the first step in our research towards foundational models that simulate more complex interactive worlds, thereby replacing the classic game engine for a future driven by AI.”

There are all sorts of weird quirks to the game, a result of everything being generated frame-by-frame, and being trained on real gameplay. Items will randomly pop up in your inventory, as the game tries to match what Minecraft players would have in their inventory in similar environments. The landscape warps and changes as you look away from it, with the entire world feeling like it is simulating what it’s like not to have object permanence yet.

For a game concept that revolves around building permanent structures, this isn’t ideal, and we doubt anyone would play it for fun rather than out of curiosity. AI is not replacing video game developers any time soon.

However, as a curiosity, it is pretty interesting.

“In essence, diffusion models learn to reverse the iterative process which adds Gaussian noise to the input and thereby enable generating new samples given noise,” the team explained. “This approach can be extended towards video generation by adding additional temporal layers to the model architecture that receive a context consisting of the previous frames that were generated (e.g., in an autoregressive fashion).”

The team has high hopes that AI generation could “revolutionize” games, enabling more interaction between gamer and the game they are playing.

“Simply imagine a world where this integration is so tight that foundation models may even augment modern entertainment platforms by generating content on the fly according to the user preferences,” the team says, “or perhaps a gaming experience that provides new possibilities for the user interaction such as textual and audio prompts guiding the gameplay (e.g., ‘imagine that there is a pink elephant chasing me down’).”

Whether gamers want this or well-made, human-designed video games, remains to be seen. You can play the game for yourself here, though be warned, you may have to join the queue.

—

[An excerpt from coverage in Wired]

The First Entirely AI-Generated Video Game Is Insanely Weird and Fun

Generative AI may transform video game design, but the first playable title is just bizarre and fascinating.

By Will Knight
November 13, 2024

Oasis is similar to a video-generating model like Sora except a user can control its output.

You can play Oasis online for free, and it is both fascinating and surreal to explore. Besides harboring bizarre artifacts, like misshapen livestock and stairs that go nowhere, the game has an amazing, Inception-like quality. Because each frame is generated based on what the AI model imagines should come after the frame it currently sees, the in-game world is never entirely stable, and will gladly shift and morph with a little nudging. If you stare too closely at a texture, for example, when you look up again, the block world in front of you may be completely different from the one you last saw.

It’s also possible to upload your own image for Oasis to work with. I tried adding a photo of my cat, Leona, and the game turned her into a beautiful blockish landscape (sadly not a feline character in the game, but hey …).

Oasis has become a viral hit with people exploring ways to get its AI engine to hallucinate new environments. Sometimes, it can even be tricked into teleporting you to a dark moonscape resembling The End of Minecraft. It’s telling that this generative AI project is not entirely original, but rather seems to be a bizarro knock-off of the world’s most popular game (it was trained on an open source Minecraft dataset from OpenAI).

“People are trying to teleport into different worlds and speed run,” says Robert Wachen, chief operating officer at Etched. “It’s one of the main reasons it went viral.”

—

[An excerpt from coverage in MIT Technology Review]

This AI-generated version of Minecraft may represent the future of real-time video generation

The game was created from clips and keyboard inputs alone, as a demo for real-time interactive video generation.

By Scott J Mulligan
October 31, 2024

For Decart and Etched, this demo is a proof of concept. They imagine that the technology could be used for real-time generation of videos or video games more generally. “Your screen can turn into a portal—into some imaginary world that doesn’t need to be coded, that can be changed on the fly. And that’s really what we’re trying to target here,” says Dean Leitersdorf, cofounder and CEO of Decart, which came out of stealth this week. …

The companies acknowledge that their version of Minecraft is a little wonky. The resolution is quite low, you can only play for minutes at a time, and it’s prone to hallucinations like the one described above. But they believe that with innovations in chip design and further improvements, there’s no reason they can’t develop a high-fidelity version of Minecraft, or really any game.

“What if you could say ‘Hey, add a flying unicorn here’? Literally, talk to the model. Or ‘Turn everything here into medieval ages,’ and then, boom, it’s all medieval ages. Or ‘Turn this into Star Wars,’ and it’s all Star Wars,” says Leitersdorf.