Has anyone tried to use ChatGPT as a GM

[Easy e]

Bear with me as this is just at a thought experiment level, but has anyone tried to use Generative AI as a form of GM in a game. The idea would be that the player would input some criteria, the Generative AI would start the scene, and the players would input their reactions, and then the Generative AI would determine outcomes.

Obviously, there are some guard rails that would need to be in place to make this more effective. What those are is less clear to me. Here are some random thoughts:

1. It helps if you use an established setting in Media that the Generative AI can scrape. For example, In the world of Dragonlance we would like to start a scenario in a tavern. What does the scene look like?

2. The players would need to be able to "lead" the Generative AI a bit. Obviously, the input needs hooks that the Generative AI can respond to. However, I am not sure how that would taint the game. For example, a half-elf paladin tries to strike the Draconian in the scenario above with my sword. What happens?

3. The Players would need to keep put in the right inputs to get the story started. For example, Start a scenario in the world of Dragonlance that features a dwarf wizard named Tinder Forgefire, a Kender named Bimbo Bagg'em, and a Half-elf ranger named Dribble in a tavern.

4. The players would need to keep adding questions to keep the AI Generating the next scene or content. For example, we fought the Draconians and they ran off. We want to chase them. Where do they go?

There are obvious downsides to this, but if you try it can you let us know how it worked and what you learned from the experience?

[KorvinStarmast]

Didn't we have this conversation recently, or did that thread get closed?

Media that the Generative AI can scrape. And there is copyright stuff to consider. :smallwink:

[KaussH]

Overall using AI as an assist at the table, that seems fine. A GMs little helper and all that but.... I like to GM, I like to spin the tale and make the description and flow. Someone showed me a AI for Cthulhu recently, and tried to sell me on it showing how well it could describe a chase. It looked ok but... thats the fun part for me, spinning the words and the tale and the flow.. So I have mostly avoided the AI GM idea so far.

[NichG]

When I've messed with this, the biggest problems were to do with momentum and pacing - it's very reactive, doesn't tend to interrupt with external pressures or push things forward or create things that demand responses.

So if you specifically avoid relying on it for that stuff, it'd be better.

The chatbot ones also tend to be less wildly creative, so I'd use GPT3 over ChatGPT to e.g. come up with magical locations or artifacts or details about a local culture or whatever.

I could see it as like a 'I have a hundred NPCs and factions, let's automate something that just says how they respond to something the party did, and which ones would be likely to intervene' though. So you prepare a prompt like: Faction X is Y distance away, concerned with Z, ethos W, holdings P ... News arrives that Q occurred. What does X think? Does X spend any resources to respond, and how much?

[Easy e]

And there is copyright stuff to consider. :smallwink:

I'm not sure it is a big issue in the sense of a group of friends playing a one-short game with each other and no intention of monetizing any of it.

[OldTrees1]

There are obvious downsides to this, but if you try it can you let us know how it worked and what you learned from the experience?

Some of this is covered in the previous threads:
https://forums.giantitp.com/showthread.php?654197-ChatGPT-is-an-amazing-tool-for-DMs
https://forums.giantitp.com/showthread.php?655475-Chat-GPT-as-DM-Link-and-Discussion

Generative AI 101:
1) It does not care about the accuracy of what it is saying. It is optimized for giving a reply that sounds like a reply. It cannot and will not attempt to maintain a game state or any other type of veracity. That includes the choices or actions your characters make.
2) The further you stray from the common training topics, the less "creative" and more repetitive the output becomes.

As a result ChatGPT does not attempt to GM, so only use it as a GM if playing a GM-less game. On the other hand it can function as a GM assistant, but performs better the more cliché your campaign is.

I have tested it in brainstorming about dungeons. After setting some parameters it initially gave bad answers much faster than I could. However it started repeating itself and I outpaced it in quantity and quality. (Outpacing it in quantity really disappointed me. I knew I would outpace it in quality).


Didn't we have this conversation recently, or did that thread get closed?
And there is copyright stuff to consider. :smallwink:
Yes.

[Eldan]

Though, I must say, feeding all the materials for a setting into an AI is one of the better ideas I've heard on how to use it. We've previously tried making NPCs and adventure seeds and it wasn't especially good about that. On the other hand, just having a little AI assistant on the desk I can ask "Hey Mimir, what happened in 201 on the Moonshae isles?" could be quite useful in running some of the weightier settings.

[Mastikator]

I've found if I have writers block ChatGPT can sometimes give me ideas, never anything good, but good enough for me to build on. But the thing to consider about quality is that the players don't care, they care about their meaningful choices, not your originality.

However using it during the game is totally untenable.

[False God]

I would be hesitant to use AI for "at home use only resources", or AI at all for any kind of scraping of existing artwork to create something for supposed at-home-use.

I'm certain there are artists out there who sell generic token packs or character concepts for affordable prices. I'd very much favor supporting them, rather than resorting to AI.

[Easy e]

To be clear, this is for running a narrative for a game, not artwork or props and the like.


Sorry if I am duplicating a previous thread. This is a whole new world.

[OldTrees1]

To be clear, this is for running a narrative for a game, not artwork or props and the like.


Sorry if I am duplicating a previous thread. This is a whole new world.

For running a narrative for a game, ChatGPT will not attempt to track the game state. When a player makes a choice or a character takes an action, ChatGPT will not include that in the game state, because ChatGPT will not attempt to track the game state. When you ask ChatGPT for content, it will not attempt to have its generation be compatible with the game state, since ChatGPT will not attempt to track the game state.


So, who is tracking the game state and sanitizing ChatGPT's output when it is not compatible with the game state? Call that person the GM. They are the one adding ChatGPT's suggestions to the game state and allowing player agency by acknowledging player choices via including them in the game state*.
*As long as someone is tracking the game state, and sanitizing ChatGPT's output, then ChatGPT can assist that GM in creating meaningful choices. The gap is mostly that ChatGPT, despite appearances, will not attempt to GM. Just like ChatGPT will not attempt to be a search engine for Legal Cases (if you know that reference).

[Enceladus]

In my own experiences, I have tried it as both a GM tool and to be the actual GM. This done with 5e D&D and VTM 20 to see what would happen for more simplistic and complex systems. This was also for both solo and group play. This for solo play just because to see if it could work. Group play for the purposes of introducing a few of the friends in my circle who enjoy board games but do not like the TTRPG set up. So to see if an AI could make it more like a board game fashion as similar to those that use an app like Mansions of Madness.

In short, it does not make a good DM. As many have mentioned, it does not keep consistency and there were times it ignored rules and mechanics. Prompts could be entered to remind it, but at the same time, said prompts were also needed not to play as my character. Further, the more complex the system like VTM I noticed the greater the epic failure. Playing around with it and setting parameters to a rules light game like Fate did produce better results but still far from perfect.

I also began to experiment with other AI based chat tools like AI Dungeon and Google's Bard. Bard was even worse than ChatGPT while AI Dungeon amounts to being something like a choose your own adventure book with little to no regard for mechanics. Though that was using their free demo version, which even considering the features unlocked with a paid subscription, I still wasn't all that impressed.

The use of AI based programs is better served as a DM assistant. ChatGPT is good to use for quick ideas on characters, locations, or props of some type when combined with others. But acting and controlling things for a DM perspective, it is a straight up failure. Perhaps as the technology increases, maybe. But as to right now, no.

[warty goblin]

I have messed around a bit using either chatGPT or Google Bard (I forget which) as, not a GM of a formal game, but as the author of a self-generating open choose your own adventure story. Basically you give it some prompt, and tell it to give you some options for what to do next at the end. Pick an option and repeat. There's no formal game state to track

Technically this works pretty much just fine. As an activity it's sort of entertaining due to the novelty, but once that wears off it's kinda boring It's very weakly interactive, and it runs smack into chatbot's flat affect and sanitized content. Like, the fun of an actual CYoA book is 1) cheating by sticking your finger in the pages in order to deal with 2) the author's bizarre and unhinged ways of killing you off without warning. With a chatbot, you're just picking between various flavors of bland success.

It's possible that with better prompting you could get more engaging output, but frankly figuring out how to do that sounds more boring than pretty much any other use for an active internet connection. And even then you're going to run into the bot's fairly short memory, so you aren't going to get long running sensible plots, or persistent items (unless you repeatedly tell it what you are carrying,) and just a general lack of more than surface level coherence.

[DammitVictor]

I don't use it when I am running a game, but I use it when am planning or designing one. It remembers what I tell it, it can compare and contrast different theoretical objects, and it can dredge up deep folklore on subjects previous game designers have only touched upon lightly.

[hbtenerji]

It's a really impressive and helpful program. I use it in all areas.

[Atranen]

I've had some success using it to generate random encounter tables. In this case the ideas can be low quality and I can iterate on and improve them, either before or during the game. It's good for generating a lot of low quality ideas quickly.

[Connington]

As has been said, don't try to use ChatGPT to DM a game. It won't work.

If you're using an LLM for idea generation, you'll get better ideas if you give it something that pushes it away from generic ideas. Bing has internet access, so point it at your favorite article about creating better [x] and tell it to apply that. Or give it some type of classification scheme (e.g. these NPCs should all correspond to zodiac horoscopes) and have it use that.

Another route is to use it as a sounding board. Show it something you've written up, and ask for feedback or questions from whatever perspective you want to improve. or just anticipate.

Claude also has the ability to take in PDFs up to 10MB, so you could use it to understand a rule, adventure, or setting book better.

[Cikomyr2]

I did try. It plain doesnt work. ChatGPT doesnt have implied memory of what it tells you except if you explicitly ask it to in your questions.

It will make up stuff as it goes based on a BSing behavior.

Its way better to help you think or do first drafts

[Jakinbandw]

Claude 2 being able to remember about 75k words also sounds like it would work better overall.

[Cikomyr2]

Claude 2 being able to remember about 75k words also sounds like it would work better overall.

Its more than words. Its plot. Its moving parts of an existing setting. The AI wont make the connection that the guards you killed outside of town may alert the Sheriff to the PC presence.

You'd need an entirely different set of artifical intelligence skills.

[Zuras]

Everything I’ve heard from people who’ve tried it is that LLMs are great for quickly generating content that’s tangential to your actual plot, so detailing all the inhabitants of a village, or quickly generating details for a tavern.

Has anyone built a combination oracle/ChatGPT system that uses random tables to generate a prompt for the AI?