ChatGPT as Forever DM

[Paragon]

I have heard lots of things on the ChatGPT fever that has been going around (at least in France), some good, some bad so I took it upon myself to see what was behind the hood. In the light of my meager understanding of "AI", I discovered that the program was a simple "fill the sentence with a next word that is appropriate" one.

OTOH, I've seen people on this forum try to use it to settle unending RAW debates over rules (Dragonwrought Kobolds etc.)

Now if someone with enough know-how were to try their hands at customizing ChatGPT so that it would be fed the rules of 3.5 as well as all the stories, monsters, traps, dungeons, ... we could find, could it be possible to make it the ultimate forever DM ?

Maybe the first course would be to give people that want to play-by-post a "room" with the bot and see how that goes ?

If this idea is impossible and you know why it is, could you take the time to explain it to me so I can strike it from my mind please ? :)

[Inevitability]

You might be interested in AI Dungeon (https://en.wikipedia.org/wiki/AI_Dungeon): not quite ChatGPT based (which is GPT 3.5 or GPT 4), but with similar underlying technology and basically what you're describing. It does lean more towards the text adventure games of the 80s than actual D&D, mostly because it's hard to make a large language model keep track of all your stats and rolls compared to freeform play.

Don't have much time right now but I might go a bit more in-depth about the feasibility and difficulties of the concept later.

[Crake]

I discovered that the program was a simple "fill the sentence with a next word that is appropriate" one.

That's a very oversimplified statement that can pretty much be used to acccurately describe how the brain determines how to construct a sentence....

[Biggus]

That's a very oversimplified statement that can pretty much be used to acccurately describe how the brain determines how to construct a sentence....

Can it? Do you have a link about that or some more details?

[Crake]

Can it? Do you have a link about that or some more details?

Think you misunderstand, I'm not making a comment on the capabilities of chatGPT, but rather the fact that "fill the sentence with a next word that is appropriate" is sufficiently vague that it can basically describe literally anyone or anything building a sentence, regardless of how or by what criteria they determine to be "the next word that is appropriate".

[Paragon]

The vague thing to me is what you are trying to say through this random comment (if anything)

[137beth]

You might be interested in AI Dungeon (https://en.wikipedia.org/wiki/AI_Dungeon): not quite ChatGPT based (which is GPT 3.5 or GPT 4), but with similar underlying technology and basically what you're describing. It does lean more towards the text adventure games of the 80s than actual D&D, mostly because it's hard to make a large language model keep track of all your stats and rolls compared to freeform play.

Don't have much time right now but I might go a bit more in-depth about the feasibility and difficulties of the concept later.

Side note: AI Dungeon (which I am a fan of) used to use GPT3, but OpenAI (the company that owns the GPT series) told them they weren't allowed to use it to depict fictional violence, ruling out using it for a D&D-like game. Hence, AI Dungeon switched to using a competitor.

[Chronos]

Current versions of chat AIs can superficially look like they're doing a competent job of DMing, but if you look closer, it becomes clear that they're just ignoring the rules and saying whatever sounds about right. And I'm not sure if this is a problem that's solvable with the current methodology, no matter how much they scale it up: My suspicion is that it would require interactive training, rather than static training like AIs are currently receiving (i.e., you'd need many rounds of an already-intelligent teacher telling the bot to do things, and then telling it what it got right and wrong and having it repeat the process).

To be a good DM, capable of handling unexpected situations, it would also need a strong and accurate world-model, which would almost certainly require more forms of data than they're currently using, and likely also require interactive training (both interacting with a teacher, and with the world).

[Paragon]

So you're saying, the first players that might use this DM/AI will have to feed it their interpretation of how each encounter should be handle so that it can repeat it.
I'm down for this, it might even help us getting refreshing ways of playing this game since we tend to surround ourselves with people playing like we do in a confined space within this game's possibilities I found out.