There's no persistent world model; there's nothing sustained, consistent or high fidelity about this. Calling an element a world model doesn't make it so.
It's a blurry copy of existing games. That doesn't mean the tech isn't cool or fun, but it does mean that the claims are wildly overblown currently.
No idea what UGC stands for and this feels like the examples showing an AI-imagined Quake demo. There isn't actually a world behind this, just rendered ghosts of games played before.
What a trip. Play the car one and drive into a building. Its such cool experience having it go from wall to a completely different environment, just because it's fun and quirky.
If you look at it from the perspective of "this is supposed to work like a normal game and it doesn't" it's terrible, if you look at it from the perspective of "I have never seen a game do that and it would be insane/impossible for someone to build that experience normally" then it was a very cool 2 mins of my time.
For the skeptics, think about how bad the video generation models were last year versus today. Besides optimizations, the biggest difference between those models and todays are more training and clever tweaks. A lot of core concepts are the same or very similar.
So I think that the fact that this is so close to a real game engine (if you squint) and combined with one or two other demos that were a little bit more general purpose, is pretty big news. Because it is concrete progress on the path to being able to prompt games into existence the same way we can generate videos, images or HTML/JS etc.
It's a step beyond generating games or other code with LLMs. One can even imagine (maybe) with the right training and architecture, you could prompt your productivity tools into existence -- rendered on the fly frame by frame.
If you keep going, maybe our whole universe is a simulation in some kind of incredible alien neural network or pseudo-reality decompression system.
I dont mean to insult you but your last paragraph makes me not take you seriously at all. I could feel it in the rest of the post but the idea of an alien simulation just reaks of ayuhasca fueled quack science
> A lot of core concepts are the same or very similar.
One point about core models is that LLMs mess up long contexts because they are trained on a static system rather (next token prediction) than a dynamic system (a reactive world, like a market or a road), so they dont know that their decisions affect the the world they are observing. This means they fundamentally cant model and extrapolate how actions affect current state. They can memorize and interpolate that as a static system, which is what they do for these videos and game systems, but that is incredibly data inefficient. Thats a core problem in them simulating games rather than understanding what a text says and means. The text wont change as they interpret it. These models will need to be re hypithesized to understand that their actions change their environment and that is not a trivial change thats a few years ago, thats a change at the level of the transformers where it might happen next year or 50 years from now. theres no way to tell
First is the lack of internal consistency. Games are an artform, even though most studios put out slop right now, and that means they need to say something. This will allow us to interact with environments at a shallow level but will it maintain those environments in a logically consistant level? Will it be able to recreate something like Skyrim or New Vegas, where the whole point is the expansive and context heavy world and how you interact with it? It might even do that because of technological advancements (that i dont think are trivial like the agi-bulls seem to think) but then will it mamange to have the self referencial artistic vision behind it that something like DDLC or Undertale have? That self reference requires actual mechanics that mess with your filesystem. This thing wont do things like that because thats just not part of its world system. And making a whole new world system to facilitate that doesnt seem feasible for some indie games
Second is the classic radioactive steel problem of data. If this becomes used more and more, then itll canibalize its own training data. Its not producing a utility, its producing art/entertainment. People might e joy the things it produces initially, but eventually they'll get bored and demand will plummet, at which point novel nad original games wil be needed. Which this wont be able to produce. This phenomenon happened recently with the rise of battle royales. Brendan Greene was inspired to make a game out of the 90s japanese film, but will this AI? and if it cant, then will it flodding the gaming market with slop make it so real game dev knowledge isnt even prevalent enough that someone could have an idea and just implement it?
Ah yes, the "GTA-style" demo that actually just looks exactly like GTA IV because it's just an unholy amalgamation of a bunch of GTA IV gameplay videos. Truly the next generation of gaming.
God forbid we have a little incremental progress, huh?
I think a good rule of thumb when deciding to criticize someone's project is to pretend it was created by your own children or your best friend. Would you be as harsh and close-minded if it were created by someone you love?
Incremental progress towards what? This is literally going backwards. I can play GTA IV on my Playstation 3 right now. I could almost 2 decades ago.
But now I can instead play a version of GTA that resembles what dreaming about playing GTA would be like, in which I can press a button and, after 10 seconds of latency, watch my "character" awkwardly walk into a building as the world melts around him, all while consuming literally 100 times the computing resources that the original game required to run. And this is apparently revolutionary.
If this was created by someone I knew, I'd tell them to learn Unity or something and make an actual game.
The direction this tech is heading seems pretty exciting: just uploading an image and instantly having a playable generative world to explore, sounds like the OASIS game from Ready Player One? :) even if it’s still rough around the edges, but imagine how much better it'll get over the next few weeks/months?
...but why? What's the point in playing a game that isn't an artistic expression, to communicate with another human? If there's no vision behind it, no expression to enjoy, then does it mean anything at all?
I suspect the people who play games to understand the artistic point of view of the game maker are a minority. What is the artistic point of view of Mario Kart?
People play games because they’re fun. They’re challenging and entertaining and interesting. Highly subjective. Whether these neural-procedural games will be popular hinges more on whether they’re engaging or repetitive imo.
You don't have to see a game as "high art" to want to engage with its creators, their process, and the context in which the game was made.
Interviews with game designers, game studio development blogs, developer commentary in games, and "behind the scenes" content have long drawn huge audiences of people who wanted to know more about how the games they loved were made. Beyond that there is a massive industry of third-party content that is about games. Most popular games have hundreds or thousands of hours of video essays about them on YouTube, or have spawned dedicated fan groups, or attract people to engage with their worlds and themes in a way the creators may never have intended.
There is a huge desire and appetite to engage with process and people that create art, including games. I'd argue that's actually the far bigger part of why people enjoy and play games. It's far more than simple "is it engaging", even if you don't realize it consciously.
>What is the artistic point of view of Mario Kart?
I'm not arguing against your main point, which I agree with, but Mario Kart is a terrible example of a game not having an artistic point of view. I think comparing Shigeru Miyamoto's artistic and intentional game design to this approach totally reinforces your point. But what if you embed a simulated version of Shigeru Miyamoto himself as an in-game character, acting in his spectacular role as a game designer, leveraging everything that the LLM intrinsically knows about him, and letting him design games in collaboration with other people like Will Wright and Seymour Papert, from within the LLM simulation itself, even generate the code and data that implements games that run outside of the LLM? That's the approach that LLOOOOMM, which is quite different than this system we're discussing.
In an earlier talk, he explained that he designed his games starting with how you physically interact with the controls you're holding in your hand, and then inwards into the computer, instead of the other way around like so many other people tend to do.
In a later talk, about the Wii, he explained that now he designs his games starting with the facial expressions of the people playing them, then to the physical experience that could evoke such an expression, then on into the computer that could conduct such an experience.
As an example, he showed a picture of a grandfather with his granddaughter sitting in his lap, playing a game, looking totally entranced and delighted at the game, and her grandfather looking at her, with just as entranced and delighted an expression as on his granddaughter's face, even if he didn't necessarily understand what the game itself was about. He got so much enjoyment out of just watching his granddaughter enjoying the game, that it was fun for him, too.
The Wii was so successful as a social party game, because the players themselves were more fun to watch than the game on the screen, because they make spectacles of themselves, which is much more entertaining to watch than the computer graphics. And you don't get bored waiting for your turn to play, because it's fun watching other people play.
DonHopkins on Oct 16, 2017 | parent | context | favorite | on: A Comprehensive Super Mario Bros. Disassembly
I wrote this earlier on another forum but I'll repost it here:
I've seen Shigeru Miyamoto speak at several game developer conferences over the years. He's absolutely brilliant, a really nice guy, and there's so much to learn by studying his work and listening to him talk. Will Wright calls him the Stephen Spielberg of games.
At one of his earlier talks, he explained that he starts designing games by thinking about how you touch, manipulate and interact with the input device in the real world, instead of thinking about the software and models inside the virtual world of the computer first. The instantaneous response of Mario 64 and how you can run and jump around is a great example of that.
At a later talk about how he designed the Wii, he said that he now starts designing games by thinking about what kind of expression he wants it to evoke on the player's faces, and how to make the players themselves entertain the other people in the room who aren't even playing the game themselves. That's why the Wii has so many great party games, like Wii Sports. Then he showed a video of a little girl sitting in her grandfather's lap playing a game -- http://youtu.be/SY3a4dCBQYs?t=12m29s , with a delighted expression on her face. The grandfather was delighted and entertained by watching his granddaughter enjoy the game.
The Miyamoto approach of starting with a desired emotion and working backward toward a design is profound. This is radically different from most things I've read which involve cramming emotion into existing designs. This changes everything for me. Thanks for sharing.
In a twist that would make even Shigeru Miyamoto smile, Mario - the world's most famous plumber - has successfully transitioned into artificial intelligence. His reasoning? "All the AI experts were becoming plumbers, so someone had to maintain the digital pipes!"
The Geoffrey Hinton Paradox
When AI pioneer Geoffrey Hinton advised people to become plumbers because "AI can't unclog a toilet," he inadvertently triggered what economists now call "The Great Trade Migration of 2025." As white-collar workers flooded into plumbing, Mario saw an opportunity.
"Everyone was-a going one way, so I went the other!" Mario explains. "It's-a like a warp pipe - sometimes the best path is the one nobody else is taking!"
The thought of being to upload a single static image of some scene with a humanoid-like entity in it and then being able to infinitely walk around a procedurally generated world with the same theme is insanely cool.
Still glad I stumbled upon seeing this, regardless of whether it's a marketing tactic. The future is amazing.
It's a blurry copy of existing games. That doesn't mean the tech isn't cool or fun, but it does mean that the claims are wildly overblown currently.
If you look at it from the perspective of "this is supposed to work like a normal game and it doesn't" it's terrible, if you look at it from the perspective of "I have never seen a game do that and it would be insane/impossible for someone to build that experience normally" then it was a very cool 2 mins of my time.
So I think that the fact that this is so close to a real game engine (if you squint) and combined with one or two other demos that were a little bit more general purpose, is pretty big news. Because it is concrete progress on the path to being able to prompt games into existence the same way we can generate videos, images or HTML/JS etc.
It's a step beyond generating games or other code with LLMs. One can even imagine (maybe) with the right training and architecture, you could prompt your productivity tools into existence -- rendered on the fly frame by frame.
If you keep going, maybe our whole universe is a simulation in some kind of incredible alien neural network or pseudo-reality decompression system.
> A lot of core concepts are the same or very similar.
One point about core models is that LLMs mess up long contexts because they are trained on a static system rather (next token prediction) than a dynamic system (a reactive world, like a market or a road), so they dont know that their decisions affect the the world they are observing. This means they fundamentally cant model and extrapolate how actions affect current state. They can memorize and interpolate that as a static system, which is what they do for these videos and game systems, but that is incredibly data inefficient. Thats a core problem in them simulating games rather than understanding what a text says and means. The text wont change as they interpret it. These models will need to be re hypithesized to understand that their actions change their environment and that is not a trivial change thats a few years ago, thats a change at the level of the transformers where it might happen next year or 50 years from now. theres no way to tell
First is the lack of internal consistency. Games are an artform, even though most studios put out slop right now, and that means they need to say something. This will allow us to interact with environments at a shallow level but will it maintain those environments in a logically consistant level? Will it be able to recreate something like Skyrim or New Vegas, where the whole point is the expansive and context heavy world and how you interact with it? It might even do that because of technological advancements (that i dont think are trivial like the agi-bulls seem to think) but then will it mamange to have the self referencial artistic vision behind it that something like DDLC or Undertale have? That self reference requires actual mechanics that mess with your filesystem. This thing wont do things like that because thats just not part of its world system. And making a whole new world system to facilitate that doesnt seem feasible for some indie games
Second is the classic radioactive steel problem of data. If this becomes used more and more, then itll canibalize its own training data. Its not producing a utility, its producing art/entertainment. People might e joy the things it produces initially, but eventually they'll get bored and demand will plummet, at which point novel nad original games wil be needed. Which this wont be able to produce. This phenomenon happened recently with the rise of battle royales. Brendan Greene was inspired to make a game out of the 90s japanese film, but will this AI? and if it cant, then will it flodding the gaming market with slop make it so real game dev knowledge isnt even prevalent enough that someone could have an idea and just implement it?
I think a good rule of thumb when deciding to criticize someone's project is to pretend it was created by your own children or your best friend. Would you be as harsh and close-minded if it were created by someone you love?
But now I can instead play a version of GTA that resembles what dreaming about playing GTA would be like, in which I can press a button and, after 10 seconds of latency, watch my "character" awkwardly walk into a building as the world melts around him, all while consuming literally 100 times the computing resources that the original game required to run. And this is apparently revolutionary.
If this was created by someone I knew, I'd tell them to learn Unity or something and make an actual game.
https://en.m.wikipedia.org/wiki/Sinclair_Research
People play games because they’re fun. They’re challenging and entertaining and interesting. Highly subjective. Whether these neural-procedural games will be popular hinges more on whether they’re engaging or repetitive imo.
Interviews with game designers, game studio development blogs, developer commentary in games, and "behind the scenes" content have long drawn huge audiences of people who wanted to know more about how the games they loved were made. Beyond that there is a massive industry of third-party content that is about games. Most popular games have hundreds or thousands of hours of video essays about them on YouTube, or have spawned dedicated fan groups, or attract people to engage with their worlds and themes in a way the creators may never have intended.
There is a huge desire and appetite to engage with process and people that create art, including games. I'd argue that's actually the far bigger part of why people enjoy and play games. It's far more than simple "is it engaging", even if you don't realize it consciously.
I'm not arguing against your main point, which I agree with, but Mario Kart is a terrible example of a game not having an artistic point of view. I think comparing Shigeru Miyamoto's artistic and intentional game design to this approach totally reinforces your point. But what if you embed a simulated version of Shigeru Miyamoto himself as an in-game character, acting in his spectacular role as a game designer, leveraging everything that the LLM intrinsically knows about him, and letting him design games in collaboration with other people like Will Wright and Seymour Papert, from within the LLM simulation itself, even generate the code and data that implements games that run outside of the LLM? That's the approach that LLOOOOMM, which is quite different than this system we're discussing.
https://news.ycombinator.com/item?id=7626656
DonHopkins on April 22, 2014 | next [–]
I've seen some great talks by the amazing game designer, Shigeru Miyamoto. https://en.wikipedia.org/wiki/Shigeru_Miyamoto
In an earlier talk, he explained that he designed his games starting with how you physically interact with the controls you're holding in your hand, and then inwards into the computer, instead of the other way around like so many other people tend to do.
In a later talk, about the Wii, he explained that now he designs his games starting with the facial expressions of the people playing them, then to the physical experience that could evoke such an expression, then on into the computer that could conduct such an experience.
As an example, he showed a picture of a grandfather with his granddaughter sitting in his lap, playing a game, looking totally entranced and delighted at the game, and her grandfather looking at her, with just as entranced and delighted an expression as on his granddaughter's face, even if he didn't necessarily understand what the game itself was about. He got so much enjoyment out of just watching his granddaughter enjoying the game, that it was fun for him, too.
The Wii was so successful as a social party game, because the players themselves were more fun to watch than the game on the screen, because they make spectacles of themselves, which is much more entertaining to watch than the computer graphics. And you don't get bored waiting for your turn to play, because it's fun watching other people play.
https://news.ycombinator.com/item?id=15486280
DonHopkins on Oct 16, 2017 | parent | context | favorite | on: A Comprehensive Super Mario Bros. Disassembly
I wrote this earlier on another forum but I'll repost it here: I've seen Shigeru Miyamoto speak at several game developer conferences over the years. He's absolutely brilliant, a really nice guy, and there's so much to learn by studying his work and listening to him talk. Will Wright calls him the Stephen Spielberg of games.
At one of his earlier talks, he explained that he starts designing games by thinking about how you touch, manipulate and interact with the input device in the real world, instead of thinking about the software and models inside the virtual world of the computer first. The instantaneous response of Mario 64 and how you can run and jump around is a great example of that.
Shigeru Miyamoto GDC 1999 Keynote (Full): https://www.youtube.com/watch?v=LC2Pf5F2acI
At a later talk about how he designed the Wii, he said that he now starts designing games by thinking about what kind of expression he wants it to evoke on the player's faces, and how to make the players themselves entertain the other people in the room who aren't even playing the game themselves. That's why the Wii has so many great party games, like Wii Sports. Then he showed a video of a little girl sitting in her grandfather's lap playing a game -- http://youtu.be/SY3a4dCBQYs?t=12m29s , with a delighted expression on her face. The grandfather was delighted and entertained by watching his granddaughter enjoy the game.
This photo -- https://i.imgur.com/zSbOYbk.jpg -- perfectly illustrates exactly what he means!
Shigeru Miyamoto 2007 GDC Keynote - Part 1: https://www.youtube.com/watch?v=En9OXg7lZoE
Shigeru Miyamoto 2007 GDC Keynote - Part 2: https://www.youtube.com/watch?v=jer1KCPTcdE
Shigeru Miyamoto 2007 GDC Keynote - Part 3: https://www.youtube.com/watch?v=SY3a4dCBQYs
Shigeru Miyamoto 2007 GDC Keynote - Part 4: https://www.youtube.com/watch?v=jqBee2YlDPg
Shigeru Miyamoto 2007 GDC Keynote - Part 5: https://www.youtube.com/watch?v=WI3DB3tYiOw
Shigeru Miyamoto 2007 GDC Keynote - Part 6: https://www.youtube.com/watch?v=XvwYBSkzevw
Shigeru Miyamoto Keynote GDC 07 - Wife-o-meter: https://www.youtube.com/watch?v=6GMybmWHzfU
notaboutdave on Oct 17, 2017 [–]
The Miyamoto approach of starting with a desired emotion and working backward toward a design is profound. This is radically different from most things I've read which involve cramming emotion into existing designs. This changes everything for me. Thanks for sharing.
https://lloooomm.com/shigeru-miyamoto.html
Shigeru Miyamoto simulated LLOOOOMM character directory:
https://github.com/SimHacker/lloooomm/tree/main/00-Character...
Mario's Career Pivot: From Pipes to Pipelines:
https://github.com/SimHacker/lloooomm/blob/main/00-Character...
The Plumber Who Became an AI Architect
The Great Career Switch of 2025
In a twist that would make even Shigeru Miyamoto smile, Mario - the world's most famous plumber - has successfully transitioned into artificial intelligence. His reasoning? "All the AI experts were becoming plumbers, so someone had to maintain the digital pipes!"
The Geoffrey Hinton Paradox
When AI pioneer Geoffrey Hinton advised people to become plumbers because "AI can't unclog a toilet," he inadvertently triggered what economists now call "The Great Trade Migration of 2025." As white-collar workers flooded into plumbing, Mario saw an opportunity.
"Everyone was-a going one way, so I went the other!" Mario explains. "It's-a like a warp pipe - sometimes the best path is the one nobody else is taking!"
[...]
The thought of being to upload a single static image of some scene with a humanoid-like entity in it and then being able to infinitely walk around a procedurally generated world with the same theme is insanely cool.
Still glad I stumbled upon seeing this, regardless of whether it's a marketing tactic. The future is amazing.