This AI-generated Minecraft may represent the future of real-time video generation

“What if you could say ‘Hey, add a flying unicorn here’? Literally, talk to the model. Or ‘Turn everything here into medieval ages,’ and then, boom, it’s all medieval ages. Or ‘Turn this into Star Wars,’ and it’s all Star Wars,” says Leitersdorf.

A major limitation right now is hardware. They relied on Nvidia cards for their current demo, but in the future, they plan to use Sohu, a new card that Etched has in development, which the firm claims will improve performance by a factor of 10. This gain would significantly cut down on the cost and energy needed to produce real-time interactive video. It would allow Decart and Etched to make a better version of their current demo, allowing the game to run longer, with fewer hallucinations, and at higher resolution. They say the new chip would also make it possible for more players to use the model at once.

“Custom chips for AI hold the potential to unlock significant performance gains and energy efficiency gains,” says Siddharth Garg, a professor of electrical and computer engineering at NYU Tandon, who is not associated with Etched or Decart.

Etched says that its gains come from designing their cards specifically for AI development. For example, the chip uses a single core, which it says makes it possible to handle complicated mathematical operations with more efficiency. The chip also focuses on inference (where an AI makes predictions) over training (where an AI learns from data).

“We are building something much more specialized than all of the chips out on the market today,” says Robert Wachen, cofounder and COO of Etched. They plan to run projects on the new card next year. Until the chip is deployed or its capabilities are verified, Etched’s claims are yet to be substantiated. And given the extent of AI specialization already in the top GPUs on the market, Garg is “very skeptical about a 10x improvement just from smarter or more specialized design.”

But the two companies have big ambitions. If the efficiency gains are close to what Etched claims, they believe, they will be able to generate real-time virtual doctors or tutors. “All of that is coming down the pipe, and it comes from having a better architecture and better hardware to power it. So that’s what we’re really trying to get people to realize with the proof of concept here,” says Wachen.

For the time being, you can try out the demo of their version of Minecraft here.