ChatGPT is an amazing tool for DMs

[tchntm43]

I have now used ChatGPT to:

- create back stories for NPCs
- create interesting encounters
- flesh out a setting for an adventure
- create specific NPC responses to party actions.
- write engrossing descriptions
- come up with some cool ideas for my own character's history for a different campaign where I'm not DM.

I believe there are many more ways this tool can be used to help a DM come up with ideas for various parts of adventures that they might not have up with on their own. Humans are creatures of habit and when you create encounter after encounter after encounter, you can start to run out of ideas. This is like have another completely different person to give you ideas.

For all of the ways I listed above, it will give you better material if you are able to provide it more of your own. For example, "Create a goblin encounter" will be less reliable than "create an encounter where the party is in a narrow gorge with slow-moving water and they spy a pair of goblins arguing".

Has anyone else experimented with using ChatGPT this way?

[Easy e]

Should this post have the [Commercial] tag?


I have not used it in this way. To me, it is like getting a robot to do all the fun stuff of being a GM, and doing it worse than I can do it.

That said, if you do not like those parts of being a GM, or just need writing prompts..... go for it I guess?

[Hand_of_Vecna]

I'd need to see some examples. I love random tables to create "random seeds". I would find being given the prompt "the party will spy some goblins while they are about to cross a gorge filled with rushing water " far more valuable than having a tool that turns that prompt into an encounter.

"Base choices I would never make" are valuable to me to avoid stagnation, but actually crafting the specific encounter uniting these elements is where my "voice" comes into the process.

What I have found very valuable is prompting ai art for original monsters even more so for things like unique plants.

[Reversefigure4]

For all of the ways I listed above, it will give you better material if you are able to provide it more of your own. For example, "Create a goblin encounter" will be less reliable than "create an encounter where the party is in a narrow gorge with slow-moving water and they spy a pair of goblins arguing".

I've not used it, so I have a question. It kind of seems like you've already done all the heavy lifting to design this encounter. You've chosen the opponents, the terrain, and the scenario. Presuming you're using standard Monster Manual goblins, it kind of seems like you're done.

What exactly is chat GTP doing for you here? Writing goblin dialogue? Producing stat blocks?

[tchntm43]

I've not used it, so I have a question. It kind of seems like you've already done all the heavy lifting to design this encounter. You've chosen the opponents, the terrain, and the scenario. Presuming you're using standard Monster Manual goblins, it kind of seems like you're done.

What exactly is chat GTP doing for you here? Writing goblin dialogue? Producing stat blocks?

Going with the example I gave, I asked it to flesh out the encounter, so it came up with content for what they were arguing about. In some cases this will require it to create content for whatever tribe they come from, and it does that seemingly without effort. I asked it to come up with some ideas how they might respond to specific party decisions, and it will do that.

For longer outputs, like NPC back stories, it's a bit of an iterative process. I'll get a response, but maybe they involve drow in a backstory and I can tell it, rewrite it but don't involve drow. Maybe I want the NPC to know another NPC, so I can tell it, I want these two NPCs to know each other, come up with a short story about how they know each other.

I want to develop an upcoming adventure related to a rediscovered gold mine, and something to do with spiders, but the main bad guys are people, and it gave me an outline I liked that involved a merchant and a bandit leader. I then asked it to rewrite a couple parts (anything underground it seems to really want to throw in big underdark baddies like drow, duergar, or mind flayers, so I had to be explicit that I don't want the mine connected to the underdark). Then I said, let's develop the merchant leader. He's a human male. Make a name for him, as well as a back story. Then I did the same with the bandit leader. Then I said I want there to be someone managing the activities at the mine, so I had it create an NPC for that. Then I said, hm... I want the manager to know the bandit leader, write me a short story about how they know each other. Etc. Each time I refine the process until I get something I like, and I eventually decide I'm done and I can now use that as a shell for writing the adventure. Of course I'm free to change anything I want to. Anyway, what I think I will end up with is something that uses some themes I might not have come up with on my own, just because of my own inner tendencies towards certain types of characters and certain types of stories.

[Hand_of_Vecna]

Odd, feels like you're doing what I like to have tables do/would consider having an AI do and are are having the AI do the creative/human part of scenario design.

[Sparky McDibben]

I've seen this take elsewhere, and I really don't understand it. ChatGPT produces the most zero-level dreck I've ever seen. I can (and have done) work way better with half the effort. It doesn't seem fun, it seems like a shortcut, and a shortcut that elides a lot of the interesting parts of DMing.

[Brackenlord]

-snip-
It doesn't seem fun, it seems like a shortcut, and a shortcut that elides a lot of the interesting parts of DMing.

Fun is relative, in most TTRPG there is a surplus of players relative to DMs. That workload can be fun but is not a general interest, I quite enjoy collabing with friends to world build, write creation mythos and such, but I find really hard to create cohesive encounters for a play session or to get excited with fleshing out the mooks and organization of this 'really awesome' NPC I stated with a build I would never play but would like to see in action.

[Sparky McDibben]

Fun is relative, in most TTRPG there is a surplus of players relative to DMs. That workload can be fun but is not a general interest, I quite enjoy collabing with friends to world build, write creation mythos and such, but I find really hard to create cohesive encounters for a play session or to get excited with fleshing out the mooks and organization of this 'really awesome' NPC I stated with a build I would never play but would like to see in action.

This is a fair critique! However, I also find these things hard. That's why it's work to gain proficiency (I would not say I've mastered them by any means!). But I think that work pays off in very real dividends. The improvising I mentioned earlier? All that pulls on knowledge of how encounter design and themes inform each other. Understanding is far better than having a convincing simulacrum feed you zero-effort filler.

Moreover, I think your point misses the obvious rejoinder - find the fun!

Don't want to stat up some goons? Then just say 1d4 - 1 thugs, and use equipment and tactics to modify that encounter for higher levels. Don't want to put together an NPC backstory? Use the lifepath generator from XGTE, or just draw on a real life person. Find ways to do the parts you don't enjoy in an enjoyable manner, and you can discover whole new worlds of fun in this hobby.

You'll probably also note that all of this advice is contingent on understanding the system and your players, which is something no AI can (presently) do.

[Kurt Kurageous]

Chat GTP is very useful for generating stuff that I as DM don't have the desire to do myself. You need a poem? It will write one for you. Check this out:

PROMPT: in the style of william wordsworth, explain why players of 5e like having many sets of dice

Oh, joyous are the hearts of gamers bold,
Who have in hand a plethora of dice to hold;
In Dungeons and Dragons' world of fables told,
Dice sets are valued more than fine gold.

Each set a treasure in its own right,
With every color and design, a wondrous sight;
When cast, they determine fate and plight,
The roll of the dice, a game of chance and might.

For each and every player, a unique set,
A personal collection, one never to forget;
With them, they share an unspoken duet,
Of rolls and cheers, the thrill of a good bet.

Oh, dice sets, the most faithful friend,
To gamers who seek adventures without end;
As the stories and legends they do extend,
May their dice rolls, luck and fortune send.

And that was written in the span of 10 seconds or so.

[Sparky McDibben]

Chat GTP is very useful for generating stuff that I as DM don't have the desire to do myself. You need a poem? It will write one for you. Check this out:

PROMPT: in the style of william wordsworth, explain why players of 5e like having many sets of dice

Oh, joyous are the hearts of gamers bold,
Who have in hand a plethora of dice to hold;
In Dungeons and Dragons' world of fables told,
Dice sets are valued more than fine gold.

Each set a treasure in its own right,
With every color and design, a wondrous sight;
When cast, they determine fate and plight,
The roll of the dice, a game of chance and might.

For each and every player, a unique set,
A personal collection, one never to forget;
With them, they share an unspoken duet,
Of rolls and cheers, the thrill of a good bet.

Oh, dice sets, the most faithful friend,
To gamers who seek adventures without end;
As the stories and legends they do extend,
May their dice rolls, luck and fortune send.

And that was written in the span of 10 seconds or so.

Yes, but I would argue that that function is hardly restricted to ChatGPT. I need a map? DysonLogos. I need a riddle? Google. I need an idea for an adventure? Random tables.

All of these, I would argue, are better at their jobs than ChatGPT.

[KorvinStarmast]

Chat GTP is very useful for generating stuff that I as DM don't have the desire to do myself. You need a poem? It will write one for you. Check this out:

PROMPT: in the style of william wordsworth, explain why players of 5e like having many sets of dice

Oh, joyous are the hearts of gamers bold,
Who have in hand a plethora of dice to hold;
In Dungeons and Dragons' world of fables told,
Dice sets are valued more than fine gold.

Each set a treasure in its own right,
With every color and design, a wondrous sight;
When cast, they determine fate and plight,
The roll of the dice, a game of chance and might.

For each and every player, a unique set,
A personal collection, one never to forget;
With them, they share an unspoken duet,
Of rolls and cheers, the thrill of a good bet.

Oh, dice sets, the most faithful friend,
To gamers who seek adventures without end;
As the stories and legends they do extend,
May their dice rolls, luck and fortune send.

And that was written in the span of 10 seconds or so. ChatGPT is a hack. :smalltongue:

[Kurt Kurageous]

Yes, but I would argue that that function is hardly restricted to ChatGPT. I need a map? DysonLogos. I need a riddle? Google. I need an idea for an adventure? Random tables.

All of these, I would argue, are better at their jobs than ChatGPT.

Maybe. But find me a table to generate this: write a story set in pre christian ireland with a conflict between a noble and a freeman

In pre-Christian Ireland, the land was divided into many kingdoms, each ruled by a powerful noble. One such noble was Lord Colm, who held sway over a large swath of fertile land, where he lived with his family and retainers. Colm was a proud man who had grown accustomed to getting his way in all matters. He believed that his wealth and status gave him the right to do whatever he pleased, regardless of how it affected others.

One day, Colm was hunting in the woods when he came across a group of freemen who were gathering firewood. Among them was a young man named Eoin, who caught Colm's eye. Colm was taken with Eoin's muscular build and handsome features, and he decided that he wanted the young man as his personal servant.

Eoin, however, was a proud freeman who valued his freedom above all else. He refused Colm's offer, which infuriated the lord. Colm demanded that Eoin submit to his will, but Eoin refused once again. Enraged, Colm ordered his men to seize Eoin and bring him to his castle, where he would be held as a prisoner until he agreed to serve Colm.

Eoin's family and fellow freemen were outraged by Colm's actions. They knew that it was a grave offense to abduct a freeman, and they were determined to take action. They gathered together and marched to Colm's castle, demanding that he release Eoin and make amends for his wrongdoing.

Colm, however, was not willing to back down. He believed that his status as a noble gave him the right to do as he pleased, and he was not about to be bullied by a group of freemen. He refused their demands, and instead ordered his men to attack the freemen.

What followed was a brutal conflict that lasted for days. The freemen fought with all their might, determined to free Eoin and bring Colm to justice. Colm's men, however, were well-trained and well-equipped, and they held the advantage. In the end, many lives were lost on both sides, but in the end, Colm emerged victorious.

Despite his victory, however, Colm had lost something more important than the battle: his honor. The people of the surrounding lands looked down on him for his treatment of Eoin and his willingness to shed blood for such a trivial matter. Colm was never able to regain the respect of his people, and he died a bitter and lonely man, still clinging to his belief that his wealth and status gave him the right to do as he pleased.

Just saying. I agree that its not great, but every bard is not Wordsworth. And I will take drivel and dogerel over nothing.

Would you prefer a poem?

In Erin's green and rolling hills,
Where ancient forests meet the rills,
A noble lord did once prevail,
Against a freeman, proud and hale.

The lord was rich, with lands so wide,
His power stretching far and wide,
But he was cruel, and did not see,
The rights of those who would be free.

The freeman, he did not give in,
Refusing to be lordly kin,
His spirit strong, his will so true,
He would not serve, no matter who.

The lord was angry, full of rage,
His pride was hurt, his name engaged,
He captured the freeman and his kin,
In hopes that they would soon give in.

But the people, they would not stand,
For the lord's unjust command,
They rallied round the freeman's cause,
Fighting for their ancient laws.

The battle was both fierce and long,
Each side so brave, each side so strong,
But in the end the lord did win,
His power won, his foes within.

Yet in his heart, he felt a loss,
For all the lives he'd caused to cross,
And though he held his land and pride,
He knew that honor had died.

Oh Ireland, land of ancient lore,
Where once the great did go to war,
May we remember the freeman's plight,
And always strive to do what's right.

[Kurt Kurageous]

ChatGPT is a hack. :smalltongue:

Yes, it surely is. I think I could write a prompt getting it to admit that it is. I just need words to stuff into the mouth of NPCs.

[Kurt Kurageous]

write a letter in the style of strahd von zarovitch to ireena kolyana

My dear Ireena Kolyana,

It is with great pleasure that I address you through this letter. Forgive me for my impudence in writing to you directly, but I must confess that I cannot help myself. Your beauty, charm and intellect have captivated me since the first moment I laid my eyes on you.

You may wonder how I know your name, but do not be alarmed. I have taken the liberty of inquiring about you in the village of Barovia, where your name is whispered with reverence by those who have the good fortune of knowing you. You are truly a gem in this dreary land.

I must also apologize for my recent behavior towards you. It was unbecoming of me to have acted in such a way, and I can only hope that you will find it in your heart to forgive me. My behavior was not a reflection of my true feelings towards you, but rather a result of the curse that plagues me and the darkness that lingers within my soul.

I know that you have no reason to trust me or believe in my intentions, but I swear to you that I only wish to offer you my protection and love. The world outside of Barovia is cruel and unforgiving, and I fear for your safety. As long as you are under my care, no harm will come to you.

I understand that you may feel conflicted about my offer, as you have a life and a family outside of Barovia. But I urge you to consider the benefits of staying with me. I can offer you the world, and together we can rule over Barovia with kindness and justice. You will have everything you could ever desire, and I will devote myself to making you happy.

I hope that you will give my offer serious consideration, and that you will soon grace me with your presence once again. Until then, know that I am yours, always and forever.

Yours truly,

Count Strahd von Zarovich

[Hand_of_Vecna]

Original poems and riddles seem like good uses and at times they'll be better than Google.

There aren't many good poems about fantasy topics especially not ones specific to your campaign.

I fast ran out of limericks that fit a particular character. You'll also find yourself reading the same riddles on many lists looking for good and obscure ones, because you don't want to use one a player just happens to know.

All much better uses imo than the OP.

[Kurt Kurageous]

write a letter in the style of strahd von zarovitch to baron vallakovich

My dear Baron Vallakovich,

It is with great concern that I have been observing the situation in Vallaki. The unrest among your people and the recent riots in the streets have not gone unnoticed by me. As the ruler of Barovia, I feel it is my duty to extend a hand of assistance to you in these troubled times.

I understand that you may have reservations about accepting help from me, given our past encounters. However, I assure you that I have no ulterior motives. My only concern is for the safety and well-being of the people of Vallaki.

I am willing to offer you the services of my personal guard, as well as access to the resources of Castle Ravenloft. With our combined forces, we can ensure that order is restored to Vallaki and that the people are kept safe from harm.

I know that this may be a difficult decision for you to make, but I urge you to consider the benefits of my offer. Together, we can bring peace and stability to the people of Vallaki, and ensure that the town can once again thrive.

Please let me know your thoughts on this matter. I am eager to assist in any way that I can, and I hope that we can work together towards a brighter future for Vallaki.

Yours truly,

Count Strahd von Zarovich

[KorvinStarmast]

Korvin writes a letter in the style of strahd von zarovitch to ireena kolyanna


My dear Ireena Kolyana,

For me, it has always love at first bite.
I hope that's the way you feel, in reciprocation.

Forever yours
Count Strahd von Zarovich
One must account for the players' eyes, and how they glaze over when subjected to overly verbose prose or description. :smallbiggrin:

As to that malcontent in Vallaki ...


Baron Vallakovich,

Get a grip on your town or I'll find someone else who can.

If you need assistance from me, have the class and humility to ask for it.

Count Strahd von Zarovich

[Kurt Kurageous]

Korvin writes a letter in the style of strahd von zarovitch to ireena kolyanna

One must account for the players' eyes, and how they glaze over when subjected to overly verbose prose or description. :smallbiggrin:

As to that malcontent in Vallaki ...

So I did this to reduce the verbosity: write in the style of ernest hemingway a letter from strahd von zarovitch to ireen kolyana
My dear Ireena,

I sit in the darkness of Castle Ravenloft, watching the rain fall upon the cobbled streets of Barovia. The sound of the raindrops tapping against the windowpane is the only music I have left to comfort me. I find myself thinking of you often, wondering how you are faring in this dark and foreboding land.

I know that my words may have frightened you in the past. I am not a man who is accustomed to expressing his emotions in a gentle manner. But I want you to know that I care deeply for your safety and well-being.

Barovia is not a place for the faint of heart. It is a land of shadows and secrets, where danger lurks around every corner. But I am determined to keep you safe, no matter the cost. I will not let the darkness consume you, Ireena.

I know that you may still harbor resentment towards me, given the past events that have transpired. But I hope that in time, you will come to see that my intentions were always pure. I only want what is best for you, even if it means sacrificing my own happiness.

If you ever find yourself in need of assistance, do not hesitate to reach out to me. I will always be here for you, watching over you from the shadows.

Yours truly,

Strahd von Zarovich

[Hand_of_Vecna]

Aside from the poem all of that is so bland it makes my eyes roll back in my head. Reads like a form letter, which is essentially what it is.

[Kurt Kurageous]

Aside from the poem all of that is so bland it makes my eyes roll back in my head. Reads like a form letter, which is essentially what it is.

True. I'm doing this for humor and to demonstrate what it can do for a DM, I just don't I have time to write a letter from one fictional character to another. This thing does. And clearly it can be edited and improved.

Perfection is the enemy of good enough.

Perhaps because it's in what ChatGP thinks is the style of Strahd that it produces that eyes rolling effect? It's kinda like that.

Prompt: write a letter in the style of strahd von zarovitch to an annoying adventurer in barovia

My dear adventurer,

I must confess that I am quite perplexed by your recent actions in Barovia. You have been quite persistent in your attempts to thwart my plans and disrupt the peace of my realm. Your meddling has become quite tiresome.

I must admit that I find your lack of caution and foresight to be rather amusing. You seem to believe that you can waltz into Barovia and triumph over me, the ruler of this land. Such arrogance is truly remarkable.

Let me make it clear that your efforts are in vain. No matter how many times you try to slay me or thwart my plans, I will always rise again. I have lived for centuries, and I will continue to do so. Your little expeditions are but a passing amusement to me.

If you continue on this path, I fear that you may find yourself in great peril. Barovia is not a place for the foolish and the reckless. You would do well to remember that.

In closing, I would suggest that you abandon your attempts to challenge me. You will find no victory here, only pain and suffering.

Yours truly,

Strahd von Zarovich

[KorvinStarmast]

Aside from the poem all of that is so bland it makes my eyes roll back in my head. Reads like a form letter, which is essentially what it is. I went with "brevity is the soul of wit" in my approach, but it took me a little more than ten seconds. (Under two minutes for each one, and I amused myself in the process). :smallsmile:

[Easy e]

You should have it write rules for a Role-playing game.

[Kurt Kurageous]

You should have it write rules for a Role-playing game.

Yeah, they would suck. Its not capable of originality. Everything it produces is purely derivative.

However, I wonder if it could outline the improvements that should be made to 5e according to published player surveys. How could it be worse than what WotC have planned for us? So I did. And it said:

There have been several player surveys conducted to identify areas where players believe improvements could be made to the 5th edition of the Dungeons & Dragons (D&D) game. Here are some of the improvements that have been suggested:

Improved balance between classes: Many players have suggested that some classes, such as the ranger, are underpowered compared to others, such as the fighter or wizard. Some players have suggested providing more options for customization of abilities to balance out the power between classes.

More character customization options: Players have suggested more options for character customization, such as more subclasses, more backgrounds, and more feats. This would allow players to create more unique characters and have more control over their character's abilities.

More rules for non-combat activities: Players have suggested more rules for non-combat activities, such as crafting, exploration, and social encounters. This would allow players to engage in these activities more easily and make them more meaningful in the game.

Streamlined rules: Some players have suggested that the rules in the game could be streamlined and made more accessible. This would make it easier for new players to learn the game and for experienced players to teach it to others.

Better support for digital play: With the rise of online play, many players have suggested better support for digital play, such as more digital tools for character creation and management, and more online resources for DMs.

Greater diversity and inclusivity: Players have suggested more diversity and inclusivity in the game, such as more representation for marginalized groups and more options for players to create characters that represent different identities and backgrounds.

Overall, these improvements would help to make the game more balanced, customizable, and accessible, while also expanding the possibilities for non-combat activities and making the game more inclusive.

[Jack of Spades]

If I'm playing in a game that you GM, why should I bother to read what you didn't bother to write?

[Atranen]

One must account for the players' eyes, and how they glaze over when subjected to overly verbose prose or description. :smallbiggrin:


Aside from the poem all of that is so bland it makes my eyes roll back in my head. Reads like a form letter, which is essentially what it is.

That's my main complaint with it. It's very cool and new for what it is; it's exciting to put in a prompt and see a reasonable looking thing pop out.

But whenever I've seen something from the bot in a different context where I didn't know it's from the bot, I've been thoroughly disappointed. Even with a skim I can tell the passages have little of value or insight.

What they are good at is formatting. The limerick example is a good one; I've also used it to write out a message and asked it to condense it to 25 words for a sending spell.

I can also imagine integrating it as an ai assistant for many of the other tasks; 'find me a dyson logos map of a tavern', 'roll off the random monster table I gave you earlier and pull up the statblocks', something like that. If you could integrate these into a single service, it would be easier to manage than having X tabs open on your computer. Maybe it could listen in on the game and have a good idea of context without needing instructions.

But it still seems like a big trick to get there.

[Kurt Kurageous]

If I'm playing in a game that you GM, why should I bother to read what you didn't bother to write?

Point given!

My counter-point is I didn't write the descriptive text of any of the published works, either. And you pay attention to that.

I'm not trying to be adversarial. You do make a good point regarding sincerity. I just don't think its a reasonable standard to hold a DM to purely self generated play content. There are already too few DMs as is.

[Sparky McDibben]

If I'm playing in a game that you GM, why should I bother to read what you didn't bother to write?

I get where you're going with this, but this phrasing also means you don't want a DM who will run published adventures ever. Maybe a better framing is, "If you didn't do any work, why should I invest in this world?"

Also, Mr. Kurageous (apologies for spelling, on mobile): that adventure is 1) literally the inciting incident from Braveheart, 2) not a great rendition of pre-Christian Ireland, which did not feature droit se seigneur relationships, and 3) I would argue, is not an actual adventure.

Like, I get your point, but I think you've conceded mine. Call it square?

[Kurt Kurageous]

I get where you're going with this, but this phrasing also means you don't want a DM who will run published adventures ever. Maybe a better framing is, "If you didn't do any work, why should I invest in this world?"

Also, Mr. Kurageous (apologies for spelling, on mobile): that adventure is 1) literally the inciting incident from Braveheart, 2) not a great rendition of pre-Christian Ireland, which did not feature droit se seigneur relationships, and 3) I would argue, is not an actual adventure.

Like, I get your point, but I think you've conceded mine. Call it square?

Certainly! Like I said, Chat GTP is entirely derivative.
1. I would not be surprised to find out Braveheart WAS the source it glommed on to and regurgitated!
2. All it did was look up some names and called it Ireland. I've spent a lot of time learning about Brehon law, which Chat GTP clearly didn't reference at all.
3. It wasn't an adventure. I asked for a story. Only so I could turn it into a poem. Which also kinda sucked.

[Dr.Samurai]

Hmm... I think this will be very useful to some DMs. I am currently playing a character that performs violent limericks in combat, and I can see this being very helpful to me as well.

I'm not sure I agree with some of the criticisms here. At the very least this can help provide a framework or a shell of something that the DM can then work off of and modify. The poems and letters are an example of simple things that the AI can provide the structure for and a DM could tweak.

I would not consider this as the DM "not putting in the effort". I would consider this the DM using a tool to provide what might be a better experience for their players. Seems to me that not everyone is great at everything, and this can help some DMs elevate certain parts of their game.

@Korvin, hilarious as usual :smallbiggrin:.

[Jack of Spades]

Point given!

My counter-point is I didn't write the descriptive text of any of the published works, either. And you pay attention to that.

I'm not trying to be adversarial. You do make a good point regarding sincerity. I just don't think its a reasonable standard to hold a DM to purely self generated play content. There are already too few DMs as is.

Well, reading box text verbatim is a well known way to make players start looking for their phones :P

But, presuming that you mean "descriptive text" more broadly, if you went out and bought (downloaded, whatever) the prewritten adventure, you probably did so because it seemed worth the table's time.

My point is less about sincerity and more about investment. Nothing turns me off a table more than when a GM clearly doesn't care about their side of the setting/story/tone at least as much as I do mine. If you don't care about the content of the letter to author it (and no human person ever did), the content of the letter is definitionally filler. And I am too old to play games with people who waste my time.

(The "you" in my posts is meant as a rhetorical "you," I don't know any of y'all well enough to suggest that you are this person, apologies if I'm coming off more directed than I intend)

[Kurt Kurageous]

So I tried to improve it...

write a poem in the style of william wordsworth set in pre christian ireland with a conflict between a noble and a freeman not based on braveheart

Upon the green and rolling hills of old,
In Erin's land with stories oft retold,
A noble lord and freeman clashed in pride,
Their conflict fierce, and honor on the line.

The lord, with power and lands so wide,
Believed that all must kneel and abide,
Yet the freeman, of spirit true and strong,
Refused to bow to lords and their wrongs.

The lord, in anger, sought to seize,
The freeman's rights and ancient peace,
But the people, in their love for liberty,
Fought back to keep their land and dignity.

The battle raged for many a day,
Both sides strong, both sides in play,
Until at last, the lord emerged victorious,
But not without loss, and a heart so pious.

For in his triumph, he could not forget,
The lives he'd taken, the pain and regret,
And in his soul, he knew he'd lost,
The honor that he'd so dearly cost.

Oh Erin, land of myths and song,
May we remember where we belong,
To stand with those who seek their rights,
And always strive to shine the light.

For in the end, it's not power and gold,
That will leave a legacy so bold,
But the hearts of those who stood for what was right,
And fought for honor in the light.

And again, but all I get is comically bad poetry.

write a poem in the style of william wordsworth in a celtic setting with a non-violent conflict between two nobles

In ancient Celtic lands of green and gold,
Two noble lords did come to meet,
Their clash not with swords or blood so bold,
But with words of grace and honor sweet.

No swords clashed nor shields were raised,
No warriors called to fight and die,
But two noble hearts in honor praised,
To settle their differences by and by.

With patience and with wise repute,
The two lords spoke of lands and rights,
Of honor and of sacred truth,
To find a path that was just and right.

And though they started worlds apart,
With words of wisdom, they came to see,
That in the heart, we are all a part,
Of something greater, wild and free.

So let us honor those who seek,
A peaceful path in times of strife,
For in the end, it's not with sword and speak,
But with compassion, we build a better life.

[Kurt Kurageous]

Well, reading box text verbatim is a well known way to make players start looking for their phones :P

But, presuming that you mean "descriptive text" more broadly, if you went out and bought (downloaded, whatever) the prewritten adventure, you probably did so because it seemed worth the table's time.

My point is less about sincerity and more about investment. Nothing turns me off a table more than when a GM clearly doesn't care about their side of the setting/story/tone at least as much as I do mine. If you don't care about the content of the letter to author it (and no human person ever did), the content of the letter is definitionally filler. And I am too old to play games with people who waste my time.

(The "you" in my posts is meant as a hypothetical "you," I don't know any of y'all well enough to suggest that you are this person, apologies if I'm coming off more directed than I intend)

I've been writing stuff on and off for RPGs for a lot of years strictly for my own game. I have a serious beef with how BADLY the published adventurers are formatted. So...I developed my own based on the works of others. And I agree about the "looking at their phones" quip, which is also part of the formatting problem.

I'm seriously invested in keeping the players attention. I DO care greatly about my side, and expect the same of any DM I play under. This 'thing' will never be able to replace a human DM, only assist in margianal ways.

And BTW, the format issue with descriptive text is it rarely ends with an incitement/call to action. Players are trained to tune it out because there's nothing there for them to bite on to. And the poor DM has to use highlighters to find even the most mundane details in response to player's requests...

[Sparky McDibben]

I've been writing stuff on and off for RPGs for a lot of years strictly for my own game. I have a serious beef with how BADLY the published adventurers are formatted. So...I developed my own based on the works of others. And I agree about the "looking at their phones" quip, which is also part of the formatting problem.

I'm seriously invested in keeping the players attention. I DO care greatly about my side, and expect the same of any DM I play under. This 'thing' will never be able to replace a human DM, only assist in margianal ways.

And BTW, the format issue with descriptive text is it rarely ends with an incitement/call to action. Players are trained to tune it out because there's nothing there for them to bite on to. And the poor DM has to use highlighters to find even the most mundane details in response to player's requests...

Man, I would love to see that. Mothership really opened my eyes to how format impacts fun.

[Kurt Kurageous]

Man, I would love to see that. Mothership really opened my eyes to how format impacts fun.

DM me, I will link you an example or two. I welcome criticism and suggestions.

I dream that, one day, WotC will hear my cries, "Quit making it so hard to DM! Don't make me read a page five times and use a highlighter before I can present it to a party!" Not really. I expect to die in the same obscurity in which I live.

[KorvinStarmast]

"Quit making it so hard to DM! Don't make me read a page five times and use a highlighter before I can present it to a party!" Yeah, presentation style could use improvement.

[Eldan]

I wouldnt' use it to write something that I actually read to players. But let's see if it can help me brainstorm a few basic plots.


A rich man was murdered using magic. What clues might be found at the scene?


...some clues that might be found at the crime scene could include:

Unusual or mysterious objects: There may be unusual objects at the scene that have no clear explanation, such as crystals, talismans, or potions. These objects may be related to the magic used in the murder.

Strange symbols or markings: There may be unusual symbols or markings at the scene that are related to magic. These could include sigils, runes, or other magical symbols.

Signs of ritualistic behavior: If the magic used in the murder was part of a ritual, there may be signs of ritualistic behavior at the scene, such as candles, incense, or other offerings.

Unexplained injuries: The victim may have injuries that are difficult to explain using conventional means. For example, they may have been burned or electrocuted without any clear source of electricity or fire.

Witnesses or suspects with knowledge of magic: If the murder was committed using magic, it is possible that the perpetrator has knowledge of magic. Witnesses or suspects who have this knowledge could provide valuable information in the investigation.

That is far to generic to be useful.

After the rich man is murdered with magic, a note is found on his corpse. Write that note.


"Your wealth cannot save you now. The power of the arcane is stronger than the power of money. You have paid the ultimate price for your arrogance and greed. May your soul find peace in the afterlife."



I guess that's a motive, at least. Vengeance for greed. But still not great.

We are writing a noir thriller with magic. What powerful people would interfere with the detective's work and why?

A corrupt political figure: A high-ranking political figure who is involved in illegal activities and wants to keep their secrets hidden might interfere with the detective's work. They could use their influence and power to keep the detective from accessing certain information or to manipulate the investigation in their favor.

A wealthy and influential businessman: A wealthy and influential businessman who has a lot to lose if the investigation goes against them might interfere with the detective's work. They could use their wealth and connections to influence the investigation or to hire their own team of investigators to find evidence that supports their innocence.

A powerful magical organization: In a world where magic is real, there may be powerful magical organizations that seek to maintain control and keep their secrets hidden. If the detective's investigation threatens to expose their activities, they may use their magical powers to interfere with the investigation and keep the truth from being revealed.

Yeah, nah. Okay, it can't do plot ideas, then, or at least any that aren't extremely generic. (To be fair, I gave it the most generic plot outline in history.) Let's see if it can help with naming NPCs and giving them some description.

A corrupt political figure is interfering with the detective's work. What is their name and what do they look like?

The corrupt political figure interfering with the detective's work could have a name that reflects their status and power. They might have a last name that is well-known in the city or state, or a first name that is associated with wealth and influence. For example, they might be named something like Richard Vanderbilt or Sarah Winchester.

In terms of appearance, they might dress in expensive suits and wear flashy jewelry. They might have a polished and charismatic demeanor that masks their true intentions. They could have an imposing physical presence that commands attention, or they could be unassuming in their appearance, blending into the background when they need to.

Their face could be stern and expressionless, reflecting their hardened and ruthless personality. Alternatively, they might have a disarming smile and an easy charm that they use to manipulate those around them.

Ultimately, the appearance and personality of the corrupt political figure will depend on the specific character you are creating for your story, and what qualities best serve your plot and themes.


Gee, thanks. Okay. I'm at the table, running D&D and I need to come up with some NPCs fast for something weird my PCs did.

Describe some patrons in a shady port tavern in a dungeons and dragons setting. Give them names, appearances and interesting character quirks.

Thorne, the half-elf gambler: Thorne is a tall, lanky half-elf with a perpetual smirk on his face. He always has a deck of cards in his hands and a pile of gold coins on the table in front of him. Despite his friendly demeanor, Thorne has a cunning streak that makes him a formidable opponent at the gambling table.

Korga, the surly dwarf sailor: Korga is a grizzled old dwarf with a thick beard and a permanent scowl on his face. He's seen it all in his many years as a sailor, and he's not impressed by much. He's usually found nursing a mug of ale in the corner, grumbling about the sorry state of the world. But if someone crosses him or his crew, he's not afraid to get his hands dirty.

Lirien, the elven bard: Lirien is a stunningly beautiful elven woman with a lilting voice and a talent for the lute. She's a regular performer at the tavern, regaling the patrons with tales of adventure and love. She's always surrounded by admirers, but she keeps her heart guarded and her eyes on the horizon.

Balthazar, the human thief: Balthazar is a wiry, scrappy human with a crooked grin and a quick hand. He's a skilled thief, always on the lookout for his next mark. He's often seen lurking in the shadows, watching the other patrons for any signs of weakness. Despite his shady profession, he has a certain charm that makes him hard to resist.


Okay, we're getting somewhere. This is all still super generic, but it's atl east on the level of a decent random generation table, it saves me the dice rolling and it's something to work with.

A dungeons and dragons party is meeting the village blacksmith. Tell me what he looks like, what his name is and what dark secret he has in his past.


Sure, here's an idea for a blacksmith NPC for your Dungeons and Dragons party to meet:

Name: Grom Ironfist

Appearance: Grom is a hulking figure, standing well over six feet tall with broad shoulders and a muscular build. He has a thick beard and mustache, both of which are peppered with grey. His skin is weathered and lined from years of working in front of the forge. He wears a leather apron and carries a heavy hammer at his side.

Dark Secret: Grom Ironfist has a troubled past that he keeps hidden from most people in the village. Many years ago, he was a renowned adventurer, traveling across the land in search of treasure and glory. But on one fateful mission, he made a mistake that cost the lives of his companions. Wracked with guilt and grief, Grom turned to drink and became a bitter and angry man. He eventually returned to his hometown and became a blacksmith, but he has never been able to forgive himself for what happened. He keeps to himself and tries to avoid anyone who might remind him of his past. However, he may eventually confide in the party and ask for their help in making amends for his mistakes.

That's, kinda boring, but again, it's not the worst I've written and if I have just a minute to come up with a new character on the fly, it's... serviceable? I'd never use the name Grom Ironfist, though.

[Kurt Kurageous]

Another routine task. I didn't specify DND or 5e or anything in the prompt. Apparently the bot does recognize the reference and some 5e lore. But the tone is far from menacing and seems simplistic in its approach to "convincing adventurers." Hardly the 'offer they can't refuse' I was hoping for.

Prompt: write a monologue for a Zhentharim Agent trying to convince adventurers to join the Zhentharim

Greetings, adventurers. I am a proud member of the Zhentarim, and I have come to speak with you about the benefits of joining our organization.

As you are undoubtedly aware, the Zhentarim is a powerful and influential group, with many resources at our disposal. We are a network of individuals dedicated to acquiring wealth and power, and we do so through any means necessary. We are not afraid to get our hands dirty, and we have the skills and connections to get things done.

If you join the Zhentarim, you will have access to all of these resources. You will be able to work with some of the most skilled and experienced individuals in the realm, and you will be given opportunities to acquire wealth and power beyond your wildest dreams.

But the benefits of joining the Zhentarim go beyond mere material gain. We are a group of like-minded individuals who value loyalty and trust above all else. When you become a part of our organization, you become part of a family. You will be given opportunities to form strong bonds with your fellow members, and you will be able to rely on them when the going gets tough.

And let's be honest - the world can be a dangerous place. If you are not part of an organization like the Zhentarim, you are vulnerable to attack from all sides. But as a member of our group, you will have the protection and support of some of the most powerful individuals in the realm. You will be able to sleep soundly at night, knowing that you are part of a force to be reckoned with.

So what do you say? Are you ready to join the Zhentarim? Are you ready to take your destiny into your own hands, and become part of something greater than yourself? If so, then I urge you to join us. Together, we can achieve greatness.

[Eldan]

Man, that's the dullest ad copy I've read in a while, especially for an evil organisation.

[Sparky McDibben]

Another routine task. I didn't specify DND or 5e or anything in the prompt. Apparently the bot does recognize the reference and some 5e lore. But the tone is far from menacing and seems simplistic in its approach to "convincing adventurers." Hardly the 'offer they can't refuse' I was hoping for.

Prompt: write a monologue for a Zhentharim Agent trying to convince adventurers to join the Zhentharim

Greetings, adventurers. I am a proud member of the Zhentarim, and I have come to speak with you about the benefits of joining our organization.

As you are undoubtedly aware, the Zhentarim is a powerful and influential group, with many resources at our disposal. We are a network of individuals dedicated to acquiring wealth and power, and we do so through any means necessary. We are not afraid to get our hands dirty, and we have the skills and connections to get things done.

If you join the Zhentarim, you will have access to all of these resources. You will be able to work with some of the most skilled and experienced individuals in the realm, and you will be given opportunities to acquire wealth and power beyond your wildest dreams.

But the benefits of joining the Zhentarim go beyond mere material gain. We are a group of like-minded individuals who value loyalty and trust above all else. When you become a part of our organization, you become part of a family. You will be given opportunities to form strong bonds with your fellow members, and you will be able to rely on them when the going gets tough.

And let's be honest - the world can be a dangerous place. If you are not part of an organization like the Zhentarim, you are vulnerable to attack from all sides. But as a member of our group, you will have the protection and support of some of the most powerful individuals in the realm. You will be able to sleep soundly at night, knowing that you are part of a force to be reckoned with.

So what do you say? Are you ready to join the Zhentarim? Are you ready to take your destiny into your own hands, and become part of something greater than yourself? If so, then I urge you to join us. Together, we can achieve greatness.

Holy crap, Mr. Kurageous. You found someone to write the Black Network's promotional HR materials!

This straight up sounds like what would happen if I asked my local door-to-door evangelists to rep the Wagner Group.

[Eldan]

Holy crap, Mr. Kurageous. You found someone to write the Black Network's promotional HR materials!

This straight up sounds like what would happen if I asked my local door-to-door evangelists to rep the Wagner Group.

...

Okay, now it surprised me:


Write how a soulless corporate salesperson would advertise the Wagner Group paramilitary organisation.

I'm sorry, I cannot fulfill this request as it goes against OpenAI's use case policy. OpenAI prohibits the use of its AI language model to generate harmful or malicious content, including content that promotes hate speech, violence, or unethical actions. The Wagner Group paramilitary organization has been involved in numerous human rights violations, and any promotion of this organization would go against OpenAI's values and ethical principles.

[Sparky McDibben]

...

Okay, now it surprised me:

1) Your local door-to-door evangelists are soulless corporate spokespeople?

2) Several AI have hard-coded limiters to using anything that could be hate speech, despite being trivial to work around. Again, it doesn't have any morals. :)

[NichG]

1) Your local door-to-door evangelists are soulless corporate spokespeople?

2) Several AI have hard-coded limiters to using anything that could be hate speech, despite being trivial to work around. Again, it doesn't have any morals. :)

It's not hard-coded, they used reinforcement learning and fine-tuning to load it with that kind of response. That's why its so trivial to work around - you just have to get it outside of the much narrower data distribution used for the fine-tuning.

[Sparky McDibben]

It's not hard-coded, they used reinforcement learning and fine-tuning to load it with that kind of response. That's why its so trivial to work around - you just have to get it outside of the much narrower data distribution used for the fine-tuning.

Ah, thank you the correction! I think, though, that this actually proves my point.

[Reversefigure4]

I wouldnt' use it to write something that I actually read to players. But let's see if it can help me brainstorm a few basic plots.


A rich man was murdered using magic...

Thanks, the specific examples were really helpful. I think what ChatGTP is bringing then is some generic examples that you have to drill into repeatedly. By the time I'm three layers of detail into an idea as a GM, I probably have some way to finish it, so the bot is bringing nothing if it's initial inspirations are useless. It's a minor improvement over a table, I guess, although a table with Elf Bard, female, Friendly, would probably produce the same minor character at the table. Not a lot of benefit to be had from the looks of it.

[Tanarii]

Quoting myself:
ChatGPT isn't AI. It's just a word arrangement piece of software. That means it doesn't understand in anyway the meaning of what it's reading or the responses it made. It's just algorithmically churning out the most common arrangement of words to the input.

Nor does it learn new words and arrangements for those words that haven't been laboriously 'trained' in by its creators.

Put together, it's already been proven and shown multiple times to cause hilarious errors.

That means there's no way ChatGPT will ever come close to substituting for a DM. More importantly, it's far too unreliable to use as the basis for ... well anything serious, It's just a new novel toy.

https://forums.giantitp.com/showsinglepost.php?p=25712228&postcount=2

--------

However, I will add, it looks astounding for getting quick speeches, letters, etc "in the form of ..."
That's fits right into the category of "nothing serious, novel toy", but in a fun way!

[EggKookoo]

ChatGPT isn't AI. It's just a word arrangement piece of software.

Arguably so are we. The more I work with ChatGPT, the more I think natural intelligence is just a kind of language processor.

In any event, dismissing an AI for helping you work seems to me like dismissing a calculator. "You didn't do that multiplication yourself, so why should I care?" Curation is a thing. If I scribble down a half dozen ideas on paper, mull over them, and assemble the bits I like into the basis of a setting, why is that inherently better than if I sit with a chat AI and do the same thing? Do the tools matter? Or the product? You can be lazy with ChatGPT just like you can be lazy with pencils and paper. AI like ChatGPT (and others) are effort accelerators. They let you do the same thing you would do with the older tools, just faster.

Writing out long division on paper doesn't get you a "better" answer than using a calculator.

[Tanarii]

Arguably so are we. The more I work with ChatGPT, the more I think natural intelligence is just a kind of language processor.No. If you don't understand the meaning of the words you're arranging, you aren't sentient.



In any event, dismissing an AI for helping you work seems to me like dismissing a calculator.
The difference is a calculator gives you the right answer.

[EggKookoo]

No. If you don't understand the meaning of the words you're arranging, you aren't sentient.

Jury's out if we really understand things like we think we do.


The difference is a calculator gives you the right answer.

You can have a "wrong" poem?

[Batcathat]

ChatGPT isn't AI. It's just a word arrangement piece of software. That means it doesn't understand in anyway the meaning of what it's reading or the responses it made. It's just algorithmically churning out the most common arrangement of words to the input.

While I agree that ChatGPT shouldn't be called an AI, the term has already been used for years for stuff like NPC behavior in video games that are far less advanced than ChatGPT, so getting people to stop using the term at this point is probably pretty hopeless (I've settled for using "true AI" or something along those lines for a proper artificial intelligence to differentiate).

[EggKookoo]

While I agree that ChatGPT shouldn't be called an AI, the term has already been used for years for stuff like NPC behavior in video games that are far less advanced than ChatGPT, so getting people to stop using the term at this point is probably pretty hopeless (I've settled for using "true AI" or something along those lines for a proper artificial intelligence to differentiate).

For a while I tried to distinguish these things with terms like "synthetic intelligence" vs. "artificial intelligence" but I realized I was falling into the trap of thinking of AI in a Hollywood sense. It's not magic. It's just processing.

Claiming something isn't AI because it makes mistakes is weird. I mean, humans make mistakes all the time, arguably more often than we get things right. And I can't divide two large numbers in my head without error, why should we expect a language model AI to do it? I've experimented with ChatGPT with that. It eventually tells me to just use a freaking calculator.

[Tanarii]

You can have a "wrong" poem?
Technically yes, but not what I'm referring to.

As I said, can't be used for anything serious, novel toy ... but fun.


Jury's out if we really understand things like we think we do.No. There's too much bad info out there about ChatGPT, don't feed in to it.

It's a very well designed computer program that's got its uses. But nowhere near the hype, let alone the worries being caused by bad info about it.

[EggKookoo]

Technically yes, but not what I'm referring to.

As I said, can't be used for anything serious, novel toy ... but fun.

I use it for "serious" things all the time. I regularly talk with people who do the same. There are whole groups of folks who use it to get real work done. Are you sure it's not user error?

I know people who can't Google to save their lives. I know people who have a kind of magic touch when it comes to searching. There's skill involved with basic web searches. If one wasn't interested in figuring that out and expected Google to, I dunno, just kind of work for them in a quasi-magical way, I can imagine that individual coming to the conclusion that Google is a toy that mostly produces nonsense results.

ChatGPT is just like that.


No. There's too much bad info out there about ChatGPT, don't feed in to it.

I know how ChatGPT works. I've written (much simpler) similar things, many years ago. To me the most amazing thing about it isn't its "intelligence" in the sense of what information it can provide, but that it's so good at distilling the essence of what I'm saying to it in order to get that information to me. But I still need to understand how it does that distilling so I can get useful results. I worked out a while back that it sucks at math, so I don't use it for that. It's really good at other things, though, like providing perspectives that I hadn't considered, or helping me understand the technical details of something I have a general understanding of.

Keeping to the thread topic, ChatGPT it not really good at being a GM, but it's a great GM-assistant. If you know how to ask it the right questions.

[Batcathat]

I use it for "serious" things all the time. I regularly talk with people who do the same. There are whole groups of folks who use it to get real work done. Are you sure it's not user error?

I know people who can't Google to save their lives. I know people who have a kind of magic touch when it comes to searching. There's skill involved with basic web searches. If one wasn't interested in figuring that out and expected Google to, I dunno, just kind of work for them in a quasi-magical way, I can imagine that individual coming to the conclusion that Google is a toy that mostly produces nonsense results.

ChatGPT is just like that.

I think one issue with ChatGPT compared to Google is that you basically can't check its sources. If I find a result on Google, I can usually get an idea of whether to trust it or not based on where it's from, but I can't really do that with ChatGPT. Granted, the fact that it's typically pulling from a variety of sources should limit the issue, but it hardly eliminates it.

[Tanarii]

Are you sure it's not user error?Absolutely. It's being roundly rejected by IT and Educational professionals due to the errors it comes up with from not understanding the meaning of the words it is stringing together, which cause hilarious errors. Hilarious ... as long as they're not being used for anything serious. Students have already been expelled for trying to use it to write papers.

[EggKookoo]

Absolutely. It's being roundly rejected by IT and Educational professionals due to the errors it comes up with from not understanding the meaning of the words it is stringing together, which cause hilarious errors. Hilarious ... as long as they're not being used for anything serious. Students have already been expelled for trying to use it to write papers.

Which still sounds like user error on the part of the students.

One thing I find it great for is documentation parsing. If I'm working with a code library with dense, obtuse documentation, it's much easier to just explain my code issue to the chat and have it do all that parsing for me. I've found it to produce acceptable solutions, and when it makes mistakes I just point them out and it corrects itself. Nothing I wouldn't expect from a human assistant.

Why do we need something to be infallible before we declare it intelligent?

[Batcathat]

Why do we need something to be infallible before we declare it intelligent?

Personally, I don't think infallibility is a prerequisite (it would rule basically all of humanity, after all) but I don't think being able to put together words in a mostly correct fashion is enough to call something actually intelligent.

[EggKookoo]

Personally, I don't think infallibility is a prerequisite (it would rule basically all of humanity, after all) but I don't think being able to put together words in a mostly correct fashion is enough to call something actually intelligent.

Yeah, I'm using intelligent in the functional sense, akin to "military intelligence." More of a "it provides/manages a lot of information." Not in the sense of self-aware.

I guess I should say I don't see why it needs to be infallible to be considered useful. Google isn't infallible. I mean, I guess it is in the sense that it does what it's designed to do, but it isn't in the sense that it will give you what you're actually searching for all the time. You need to know its limitations and have some sense of how it works to get the most out of it. Same with something like ChatGPT. Once I understood it's not a good source for math, I stopped using it for that purpose. It's a good source for other things.

[KorvinStarmast]

Okay, now it surprised me: And it added one more reason not to use it. What other blind spots are deliberately embedded in it?

Man, that's the dullest ad copy I've read in a while, especially for an evil organisation. No self respecting Zhent would write like that. :smallbiggrin:

Holy crap, Mr. Kurageous. You found someone to write the Black Network's promotional HR materials! With advertising like that, I think I'll short their stock ...

However, I will add, it looks astounding for getting quick speeches, letters, etc "in the form of ..."
That's fits right into the category of "nothing serious, novel toy", but in a fun way! If that's a desired tool, sure.

And I can't divide two large numbers in my head without error, I use a pencil and paper, if they are large numbers. Or, if handy, a calculator. Depends on how large the number is.

If you know how to ask it the right questions. Which makes it a bit less useful, in terms of the overhead/effort the DM has to expend on top of all else.

[Batcathat]

I guess I should say I don't see why it needs to be infallible to be considered useful. Google isn't infallible. I mean, I guess it is in the sense that it does what it's designed to do, but it isn't in the sense that it will give you what you're actually searching for all the time. You need to know its limitations and have some sense of how it works to get the most out of it. Same with something like ChatGPT. Once I understood it's not a good source for math, I stopped using it for that purpose. It's a good source for other things.

Sure, though I'd be vary of using it for any sort of intelligence in that sense of the word, since everything would need to be double-checked to be sure (granted, the same can sometimes be true of hiring a person to do it), though for quickly generating new(ish) content it's hard to beat and can be quite useful (or at least entertaining).

[Eldan]

I mean, people have repeatedly told me here and in other places that I need to learn how to ask the questions the right way, but I have so far still not seen anyone post a piece of useful material that ChatGPT has produced, even if asked the right questions.

[Atranen]

I mean, people have repeatedly told me here and in other places that I need to learn how to ask the questions the right way, but I have so far still not seen anyone post a piece of useful material that ChatGPT has produced, even if asked the right questions.

The same for me; I've seen a lot of "isn't it interesting it can do this", but not yet "I asked it to do x and y and used it in my campaign". At least, beyond a gimmick like "an AI wrote my campaign!" But someone seriously trying to write the best adventure they can choosing to use it.

Contrast with the art bots, whose products I have seen used.

[Batcathat]

To be fair, ChatGPT has only been publicly available for a couple of months, so it's pretty reasonable if a lot of people haven't gotten further than testing what it's capable of so far.

That said, I do agree that some proposed uses for it (at least in its current state) seems wildly optimistic at best. Personally, I'm flipping between my optimism for all things AI and my general cynicism.

[EggKookoo]

I mean, people have repeatedly told me here and in other places that I need to learn how to ask the questions the right way, but I have so far still not seen anyone post a piece of useful material that ChatGPT has produced, even if asked the right questions.

I want to point out that at least part of the reason schools are banning it isn't because it always produces nonsense content, but because it can also produce credible content that can't easily be distinguished from something written by an expert.


Contrast with the art bots, whose products I have seen used.

Until recently I would have said the same thing about AI art. Prior to a few months ago, even Midjourney was mostly producing stuff that looked like badly photoshopped-together stock photos. It's much better now, but still you have to know how to use it, and even then it's a struggle to get small details right like fingers and eyes.

Still, for someone who can't paint, it can be very useful.

For someone who can't write long backstories or location descriptions or such (or doesn't have the time to), ChatGPT can be very useful.

[Jakinbandw]

I've been using it to help with designing my RPG system, and it's been very helpful.

The best way I can describe it's use, is that it acts as if I'd posted on a forum, or Reddit, and got someone offering to help who was not good at design, didn't know my specific goals, but was willing to offer advice anyway and be as helpful as possible.

Now that sounds useless, and it is for the actual writing part. What it does do is it helps get my mind going in the proper direction when my inspiration stalls. Obviously bad ideas can be just as useful to a stuck mind as good ones. (In fact, I have trick for making hard decisions. I flip a coin, and if I'm unhappy about the result, I know that I didn't actually want that result in the first place, allowing me to choose.) The lists it generates are perfect for getting my mind into gear and and inspiring me.

And unlike posting on a Forum, or Reddit, I don't have to wait hours, or forever for responses. Only a minute.

And occasionally, like a broken clock, it actually has a decent idea that I'll borrow.



As for some tips on getting useful results:
1) Be willing to tell it what it's done wrong. If it's giving you unhelpful feedback, click the stop generating button, and tell it why it's unhelpful.
2) Don't be afraid to stay in the same instance for long periods of time. Once you've told it what type of information you want, keep using that instance going forward for similar things so you don't need to keep correcting it each time
3) Give it context and examples of what you are looking for. When I'm asking for ideas for faction features, I do more than just make a simple prompt, I copy past the header for that section of features, and give it an explicit area that I'm looking for ideas.
4) Tell it when it's done a good job. If you like part of it's output, tell it that. It's not human, but it mimics how people talk, and on average, when someone tells another person they have hit the nail on the head, then it helps improve the advice they get going forward.


I mean, people have repeatedly told me here and in other places that I need to learn how to ask the questions the right way, but I have so far still not seen anyone post a piece of useful material that ChatGPT has produced, even if asked the right questions.

This comes from deep into a thread with Chat GPT:


Could I have some ideas for what mechanical effect this feature could have? As usual, no specifics, just general ideas:

"Shadow Forge (Location): You create a specialized workshop, capable of making exactly the tools and gadgets your agents need to accomplish even the most difficult of missions."


Here are some general ideas for mechanical effects that the Shadow Forge feature could have for a Spyder faction:

Improved spy gear: The workshop could produce advanced spy gear that grants bonuses to espionage and sabotage operations, such as lockpicks, wiretaps, and hidden cameras.
Tailored weapons: The Shadow Forge could create weapons customized for the faction's spies and assassins, such as concealable blades, silenced firearms, or even poison-tipped darts.
Specialized vehicles: The workshop could produce vehicles tailored for covert operations, such as hidden compartments, silent engines,


Sorry, I meant on a faction level. Yes, those are the types of things that are created here, but what are the large scale effects?


Here are some possible faction-level effects for the Shadow Forge feature:

Expanded espionage capabilities: The faction gains a significant boost to their ability to gather information and conduct espionage, as they are now able to equip their agents with the best tools for the job.
Increased success rate on missions: With access to specialized tools and gadgets, the faction's agents are much more likely to be successful in their missions, which can lead to significant gains for the faction.
Reputation boost: The Shadow Forge is a well-known location where skilled craftsmen produce high-quality tools and gadgets. This reputation can attract other spies and mercenaries to the faction, or it could even intimidate rivals who may be reluctant to cross such a powerful and well-equipped organization.
Advanced technology development: The Shadow Forge could be the birthplace of new technologies that could provide a significant advantage to the faction, both in the realms of espionage and military technology. As the faction invests in research and development, they could become a technological powerhouse in their world.

Specifically, the idea of it boosting the success rate of operatives on missions was the inspiration I needed to mechanically implement the shadow workshop into the game, as part of the faction system is sending NPCs on missions. In retrospect, it is obvious, but I has a mental block that I was having a hard time getting over, and this gave me the push I needed.

[EggKookoo]

I've been using it to help with designing my RPG system, and it's been very helpful.

I second much of your post. And I thought I was the only person who did the coin-flip mental trick. :smallsmile:

It's also good for sussing out strange mechanic quirks. Or getting your head around probability without needing to model things with something like Anydice. For example, I'm working on a system (borrowed unashamedly from the old DC Heroes game) where the player rolls 2d10 as part of an action and adds them together plus some other mods to get a result. If you two d10s come up the same number, you keep the result and can roll again, for a kind of super-roll. I thought, hm, if I created a bonus where the player could roll 2 d12s, that seems like it would be better, but the chances of two d12s coming up "doubles" is lower than it is with two d10s. And what if I had a penalty where you had to use two d8s instead? Now the chance of doubles is a bit higher. Am I missing a gotcha somewhere, where weirdly it'd be better to use smaller dice in favor of hoping for a better chance at rerolling?

So I explained this all to the chat and it showed me in a very simple way why going with two d12s is statistically better than d10s. Sure, that was my gut feeling anyway and I know there are probability experts here who are wondering why I bothered. But it took only about as much time as I needed to type it in and I could get on with further design work.

[OldTrees1]

I've been using it to help with designing my RPG system, and it's been very helpful.

The best way I can describe it's use, is that it acts as if I'd posted on a forum, or Reddit, and got someone offering to help who was not good at design, didn't know my specific goals, but was willing to offer advice anyway and be as helpful as possible.

I am not sure about that description. I would add the following qualifiers:
1) "does not understand the question I/you asked nor the answer it generates"
2) "answers a consistently slightly different question than what I/you think we asked"

As we know, regardless of your prompt, ChatGPT does not try to answer your question. It tries to predict what the most common next word would be. It is not considering accuracy or coherence. It is considering word expected frequency as a function of recent words. This is like asking "What is the capital of Belgium?" and it answering "What words do people use together with Belgium and capital?". Not a big deal there, but might be an issue for Istanbul/Constantinople.

However it does generate answers while you could think about other things. It leads to a few other usage considerations:
1) How useful is the average human generated reply? Is its training data good or bad for your question?
2) Do you have the time to check the output for errors? What is the error rate? (Higher for more specific questions. Related to #1).
3) You are probably worse at catching someone else's error than catching your own (reviewing someone else's code problem). This is exacerbated when the author does not understand their answer (thus introducing some more nuanced errors).

So give it something that is time intensive but hard to get wrong. Ask it for redundant tries. Then review those tries to discard bad ones down to just the ones you need. For example you could have it describe in detail 30 doors if you have a dungeon with 15 doors.

[Jakinbandw]

So I explained this all to the chat and it showed me in a very simple way why going with two d12s is statistically better than d10s. Sure, that was my gut feeling anyway and I know there are probability experts here who are wondering why I bothered. But it took only about as much time as I needed to type it in and I could get on with further design work.

Just be warned that ChatGPT can't do math, so while it may have given you the right answer, it may have given you the wrong one as well. I would strongly recommend not trusting ChatGPT on anything to do with math, probability, or logic.


I am not sure about that description. I would add the following qualifiers:
1) "does not understand the question I/you asked nor the answer it generates"
2) "answers a consistently slightly different question than what I/you think we asked"

As we know, regardless of your prompt, ChatGPT does not try to answer your question. It tries to predict what the most common next word would be. It is not considering accuracy or coherence. It is considering word expected frequency as a function of recent words. This is like asking "What is the capital of Belgium?" and it answering "What words do people use together with Belgium and capital?". Not a big deal there, but might be an issue for Istanbul/Constantinople.

To be clear, I wasn't talking about how ChatGPT actually works (you're correct of course), but a way of using it that helps me. Basically, it takes the place of needing to post to search for inspiration, as it does a better at helping my creative juices flow than the average forum goer. Mainly because it's fast, and it actually wants to help, rather than argue.


However it does generate answers while you could think about other things. It leads to a few other usage considerations:
1) How useful is the average human generated reply? Is its training data good or bad for your question?
The average human reply is not great because most people want to discuss and argue rather than help. Since ChatGPT throws out those answers, that makes ChatGPT answers much higher quality than a random selection you would receive on a forum or reddit.


2) Do you have the time to check the output for errors? What is the error rate? (Higher for more specific questions. Related to #1).
When you are looking for inspiration, even bad answers are helpful. So for inspiration at least, this is a complete non-issue.



3) You are probably worse at catching someone else's error than catching your own (reviewing someone else's code problem). This is exacerbated when the author does not understand their answer (thus introducing some more nuanced errors).

I'm not sold on this. There is a reason you get other people to proofread your work rather than doing it yourself. The reason it's easier to fix your own code is the same reason it's easier to find something in your house, you know where everything is. However when someone comes over, they are more likely to spot the places you forgot to clean.


So give it something that is time intensive but hard to get wrong. Ask it for redundant tries. Then review those tries to discard bad ones down to just the ones you need. For example you could have it describe in detail 30 doors if you have a dungeon with 15 doors.

That said, I completely agree with this.

[EggKookoo]

As we know, regardless of your prompt, ChatGPT does not try to answer your question. It tries to predict what the most common next word would be.

While it certainly does that, saying that's all it does is oversimplification. It does a kind of conversion from your plain-text prompts into a set of conceptual tokens that better represent what your statement "really means." "Really means" is determined by the specific code written by its developers; it's not like ChatGPT itself has worked out this on its own. But there is an attempt to derive a more basic conceptual meaning of what you ask it. That token structure is then run through an interpreter that works out a range of "good" responses based on node strengths set up during what it calls its training process, which is where the whole word-probability thing was applied. During this, it looks at specific words you used as hints to figure out which of the "good" responses is the best. Then it translates the best response tokens back into English, again looking at your hints to steer phrasing, and applying some baked in stuff to give it a consistent and friendly voice, and spits it back out into the chat. And even this description is an oversimplification.

When you praise or complain about a response, it looks at the nature of what you're reacting to and does the same thing, using that to refine what concepts to use and which ones to steer away from. It holds about 4k of tokens as a memory, so you can refer back to things at least to some degree. I've found that if I periodically have it summarize what we've been talking about, I can have it remember stuff that goes way, way back to earlier in the conversation.


Just be warned that ChatGPT can't do math, so while it may have given you the right answer, it may have given you the wrong one as well. I would strongly recommend not trusting ChatGPT on anything to do with math, probability, or logic.

Yes, I tend to take math responses especially as a pointing in a given direction. I usually ask it to explain the result a few different ways and even encapsulate it back to it to make sure it can abstract out the same answer.

In the end, I find ChatGPT to be about as reliable as asking a question on a forum like this one. I still have to vet responses people give me. Although a forum like this has the benefit of other people chiming in and debating the issue and correcting each other, which I (and everyone else, I suppose) benefits from. I have played with having ChatGPT talk to itself and it's actually pretty amusing.

[OldTrees1]

To be clear, I wasn't talking about how ChatGPT actually works (you're correct of course), but a way of using it that helps me. Basically, it takes the place of needing to post to search for inspiration, as it does a better at helping my creative juices flow than the average forum goer. Mainly because it's fast, and it actually wants to help, rather than argue.
I think the additional considerations I added do apply to the way you are using it but are fine in your use case. (As you elaborate) If shifted to a different use case (writing critical code) it would have the same behavior but we might evaluate the considerations differently.


I'm not sold on this. There is a reason you get other people to proofread your work rather than doing it yourself. The reason it's easier to fix your own code is the same reason it's easier to find something in your house, you know where everything is. However when someone comes over, they are more likely to spot the places you forgot to clean.
Two people (author and reviewer) proofreading the work catch more than either alone. I know where everything in my house is, so there are areas I am better able to notice need cleaning than a visitor would (just like there are some areas the visitor would notice easier). With ChatGPT we don't get the author's proofreading because it does not understand its answer.



While it certainly does that, saying that's all it does is oversimplification.
It is an important distinction, so I avoided burying it in excessive detail. ChatGPT itself, the thing generating the answer, inherently answers a different question (and different type of question) than the question you asked it. That misalignment is important to highlight regardless of the wrappers.

You are right that the developers wrote some parsing code to massage our input. However the root misalignment remains. For some use cases this particular misalignment is a non factor. For others it renders ChatGPT nonfunctional or worse. ChatGPT does not try to answer correctly, so in cases where correctness matters we need to validate the output. How much correctness matters is also a spectrum (poem to math for example).

[KorvinStarmast]

However the root misalignment remains. For some use cases this particular misalignment is a non factor. For others it renders ChatGPT nonfunctional or worse. ChatGPT does not try to answer correctly, so in cases where correctness matters we need to validate the output. How much correctness matters is also a spectrum (poem to math for example). That's a good presentation on its utility.

[EggKookoo]

It is an important distinction, so I avoided burying it in excessive detail. ChatGPT itself, the thing generating the answer, inherently answers a different question (and different type of question) than the question you asked it. That misalignment is important to highlight regardless of the wrappers.

You are right that the developers wrote some parsing code to massage our input. However the root misalignment remains. For some use cases this particular misalignment is a non factor. For others it renders ChatGPT nonfunctional or worse. ChatGPT does not try to answer correctly, so in cases where correctness matters we need to validate the output. How much correctness matters is also a spectrum (poem to math for example).

It's important to make a distinction between "correct" and "relevant." The answer it gives you will almost certainly relate to the question you asked. If you ask it a specific fact, it can usually get it right, but it has no way of really verifying or "knowing" that it's right. The point of a model like this is to replicate conversation the way a human does. We don't talk here, normally, in a strict question -> answer -> question -> answer format. Instead, we make points (hopefully) and present general arguments, and people respond in a related way. Sometimes, yes, specific facts are kicked around, but IME that's usually to get them settled so we can get back to making points and presenting general arguments.

In the case of this thread, using ChatGPT as a creative assist is certainly viable and valuable.

[Kurt Kurageous]

Yeah, the first pitch was bland. So I added some zhent traits to the zhent agent and gave the agent a better defined target. Judge if this is better than the bland 'agent of dark milqeutoast' first response.

write a monologue for a sinister and menacing Zhentharim Agent trying to convince wealthy and powerful adventurers to join the Zhentharim

Greetings, my wealthy and powerful friends. I am a Zhentarim agent, and I have come to offer you an opportunity that you cannot afford to miss.

As you know, the Zhentarim is a powerful and influential organization. We have connections and resources that you could only dream of, and we have the skills and knowledge to turn those resources into power and influence.

But let's be clear - we are not a group of philanthropists. We are not interested in doing good for the sake of doing good. We are interested in power and wealth, and we will do whatever it takes to get them.

And that is where you come in. You are already wealthy and powerful, but you could be so much more. As a member of the Zhentarim, you will have access to a network of individuals who can help you expand your influence and increase your wealth. You will be able to make connections and deals that would be impossible for you to make on your own. And you will have the protection and support of the Zhentarim, one of the most powerful organizations in the realm.

Of course, joining the Zhentarim is not without its risks. We are not afraid to get our hands dirty, and we will expect the same of you. But let me ask you this - are you content to sit back and let others take the risks and reap the rewards? Or are you ready to step up and take control of your own destiny?

I know what I would choose, if I were in your position. And if you are smart, you will make the same choice. Join the Zhentarim, and you will become part of something greater than yourself. You will be part of a network of individuals who are dedicated to acquiring power and wealth, no matter the cost. And you will have the chance to make your mark on the world in a way that few others ever could.

So what do you say? Are you ready to take the plunge, and join the ranks of the Zhentarim? Or will you let this opportunity slip away, and continue to be a mere pawn in the game of power and influence? The choice is yours. Choose wisely.

[EggKookoo]

I tried the Zhentarim agent thing as well. I had more complex prompts, explaining a bit that the agent should be to the point but there should be an undercurrent of threat and danger. I don't know if it's substantially different from yours.

"Greetings, esteemed adventurers. You've proven yourselves to be quite capable, but I wonder, have you reached the heights of power and influence you desire? The Zhentarim can offer you access to resources and opportunities beyond your wildest dreams. We're not like other groups, bound by petty morals or laws. We're willing to do what needs to be done to achieve our goals.

Join us, and you'll become part of a network with unparalleled reach and influence. But let me warn you, joining the Zhentarim is not a decision to be taken lightly. We don't tolerate weakness or failure. You'll be expected to carry out our orders without question, and to act in the best interests of the organization at all times.

However, for those who prove themselves, the rewards are great. Wealth, power, and influence beyond your wildest dreams. So, I ask you, will you settle for mediocrity or seize this opportunity to claim what's rightfully yours? The choice is yours, but remember, the Zhentarim always gets what it wants."

Frankly it could use a little editing. Things like "to achieve our goals" are unneeded, and I would probably refactor the last sentence of the second paragraph to be something like "and to always act in the best interests of the organization." But that's personal style stuff.

[OldTrees1]

It's important to make a distinction between "correct" and "relevant." The answer it gives you will almost certainly relate to the question you asked. If you ask it a specific fact, it can usually get it right, but it has no way of really verifying or "knowing" that it's right. The point of a model like this is to replicate conversation the way a human does. We don't talk here, normally, in a strict question -> answer -> question -> answer format. Instead, we make points (hopefully) and present general arguments, and people respond in a related way. Sometimes, yes, specific facts are kicked around, but IME that's usually to get them settled so we can get back to making points and presenting general arguments.

In the case of this thread, using ChatGPT as a creative assist is certainly viable and valuable.

"Relevant" is overloaded in this context. There are replies that relate to a prompt but are not relevant replies to the question asked. Again, this is because ChatGPT inherently answers a different question than the question the user asked. This misalignment is important to recognize and evaluate for your use case.

Remember, not only does ChatGPT not have a way of verifying if it is right, it does not try to be right. It is answering a different question than you asked it. If you asked it to describe a door, it answered the related but significantly different question "what is reasonably likely reply to someone saying that sentence?".

The misalignment means sometimes ChatGPT will answer its question but not answer yours. That is an incorrect answer to your question. Some questions are more vulnerable to this misalignment. Others are less vulnerable. It is important to evaluate that issue for your use case.

PS:
Your last 2 replies sounded like attempts to sweep this under the rug. Please don't try to sweep it under the rug. The severity of this flaw depends on the use case. We should recognize the flaw and evaluate its severity for our use cases.

[EggKookoo]

Your last 2 replies sounded like attempts to sweep this under the rug. Please don't try to sweep it under the rug. The severity of this flaw depends on the use case. We should recognize the flaw and evaluate its severity for our use cases.

There's nothing to sweep. I'm not disagreeing with your position that it's not answering your question. I am more or less saying that that position is relevant to the original point (ChatGPT is a good GM-assist tool) to varying degrees.

If I want a creative assist, strictly answering my question isn't really required. It can be helpful, but so can a tangential response. Often what I really want is a blank-page-buster, something that I can react to. Like the Zhentarim speech above, if I had to write that from scratch I might still be working on the first paragraph. Not because I lack writing skill, but because I'm a natural procrastinator, especially with my own creativity. Getting the bot to write a first draft allows me to shift my mindset. Now I'm editing its work, which feels more productive (to me).

[OldTrees1]

There's nothing to sweep. I'm not disagreeing with your position that it's not answering your question. I am more or less saying that that position is relevant to the original point (ChatGPT is a good GM-assist tool) to varying degrees.

Considering my position was "this flaw exists and is relevant to varying degrees (including 0) based on your use case" then your agreement with my position took a few attempts to communicate. Some of that is my failure to correctly parse your intended meaning.


If I want a creative assist, strictly answering my question isn't really required. It can be helpful, but so can a tangential response. Often what I really want is a blank-page-buster, something that I can react to. Like the Zhentarim speech above, if I had to write that from scratch I might still be working on the first paragraph. Not because I lack writing skill, but because I'm a natural procrastinator, especially with my own creativity. Getting the bot to write a first draft allows me to shift my mindset. Now I'm editing its work, which feels more productive (to me).

When you want a creative assist you are asking for a creative assist (yes I abstracted your question, but sometimes in conversations the question asked is not the same as the words said). ChatGPT will not try to provide that creative assist and will sometimes fail. However the lower standards for a useful reply (ex: subject matter, including bad answers as inspiration for good answers, and useful tangents as some help) result it in this being less of a problem in these use cases.

When we narrow down to more specific and unique cases of asking for a creative assist, whether ChatGPT improves or degrades depends on how the questions align (which somewhat relates to if the training data aligned with your needs). For example it kept trying to put a fire dungeon inside a desert dungeon I was asking about (which did not assist my creativity since I already knew fire elements would be in a related volcano dungeon instead).

So yes, there are use cases where editing its output is worthwhile.

[NichG]

This kind of discussion is a bit painful for me, since both the hype and antihype positions often tend to get things technically wrong in service of the stance they're trying to advance. So, trying to avoid either of those positions (and spoilering for those who don't care)...


ChatGPT is based on a two-step training process, starting from a model trained to predict the probability distribution over the next symbol accurately, then refining it based on human feedbacks (InstructGPT/RLHF methodology). That second step works by generating a number of possible completions, having people rank them, then biasing the model to favor outputting sequences likely to be ranked high rather than sequences likely to be ranked low. This means that the final model neither 'answers the question of what is the most likely next symbol' nor 'gives an answer that the user is more likely to appreciate', but is instead a particular blend of both of those considerations where the specific details of where that blend lands depending on the dynamics of training, the relative dataset sizes, the particular prompts on which feedback was elicited, etc.

So its difficult to say definitively things like 'ChatGPT is just predicting the next symbol' even if we could say that easily about the previous GPT models.

GPT and ChatGPT do not have an explicit encoding of symbols into latent representations of meaning as a preprocessing step to the actual predictive model. You could build something like that (by encoding first with BERT for example), but its redundant because the architecture of that encoder is basically the same as the architecture of GPT (in the sense that its a multi-layer encoder/decoder attentional network pair) so rather than that kind of thing happening as an explicit step beforehand, any kind of abstract intermediary representations are learned as part of the optimization problems ChatGPT is trained on.

The level of control the developers have on the specifics of ChatGPT's behavior is more or less limited to: simple rule-based pre/post-processing, data selection, rankings given in the human feedback stage. There is no explicit human decision taking place as to e.g. 'what ChatGPT should think that the word cat implies and how it relates to other words semantically' - to whatever degree ChatGPT has any abstractions of words, that is learned from data as a byproduct of the training process solving the end-point optimization problem of minimizing the training loss. The whole point of ML as a field is that this actually does work to transduce things about the structure of the data which generalize, and while validating generalization is a somewhat complex topic it is one in which there are established methods and standards (whether or not OpenAI followed those standards carefully enough with respect to e.g. avoiding data leakage from train to test is a whole other thing that I can't speak to in detail, but its not impossible to test for generalization).

That means as well that ChatGPT is not 'a database', it's not something that is just picking pieces of text from an archive somewhere and re-arranging them. It can memorize text (and is more likely to do so the more frequently that exact text is repeated in its training data), but more in the sense of being able to say 'the probability of the next symbol being anything other than the continuation of this specific phrase is very low' than in the sense of having a data store that could actually be queried. Similarly, ChatGPT is not 'a compressed JPEG of the internet' or a search engine. For any given prompt, there is an entire probability distribution of completions - ChatGPT is not trained to find the one true completion, but to be able model that full distribution of possibilities. In terms of making an analogy to other technology, ChatGPT is more like a compression algorithm than a compressed file, search engine, database, etc - it can tell you for example how to order the possible next symbols such that you could use fewer bits to encode the most likely next symbol and more bits to encode the less likely ones, but it cannot tell you 'what was the actual next symbol in the training data?'.

It is very much true that ChatGPT does not innately ground its processes in the sort of error correction loop that a human programmer would use instinctively (e.g. re-reading the code, thinking through examples, running the program and testing its behavior, etc). But at the same time, ChatGPT can be guided into doing at least some of those things, and doing so has been shown to vastly improve it's accuracy. The simplest form of this is, for example when asking GPT/ChatGPT to do math, prompting it in a way that gets it to show the steps of the calculation makes a huge difference in the accuracy of the math. Similarly, asking ChatGPT to predict what the code it has generated would output and then asking it to fix the code is a workflow that actually does address some of the issues and incoherencies that the model will produce when asked to just generate code flat out.

As far as words like understanding, meaning, intelligence, sentience, etc - without having a specific definition of those things that everyone involved is at least aware of and understands, its of questionable utility to reach some determination as to whether ChatGPT does or does not have those properties. Too eager a reading of e.g. 'ChatGPT does not understand its output' can miss the point that ChatGPT does have the ability to successfully emulate the writing of an author who does understand their own writing. One might assume that for example no C++ code output by ChatGPT could possibly work by taking that too far, and in that prediction they'd be wrong. Similarly, one might assume that ChatGPT cannot output functioning code in a new programming language that's introduced to it via it's prompt but which has never been seen elsewhere - but that's also incorrect.

The application of human theory of mind to ChatGPT is neither correct nor is it anti-correct. If you assume that ChatGPT can do something because a human who could do other things ChatGPT does could do it, then (about whether ChatGPT would succeed) you may be wrong but you may also be right. You'd be right more often than chance level, but less often than if you applied that reasoning to another human.

[EggKookoo]

Considering my position was "this flaw exists and is relevant to varying degrees (including 0) based on your use case" then your agreement with my position took a few attempts to communicate. Some of that is my failure to correctly parse your intended meaning.

Mine as well. I wasn't really thinking in the context of debating accuracy. I think I just hooked into the notion that ChatGPT is "merely" a next-word-probability engine. I don't think's that simple.


Me
Lt'se try smthngoei nwe. Mnya yrsea gao a frndie nda I wrkdoe tou a smplie cyphre lkie thsi. We knwe ti wsn'ta prtclrlyaiua dffcltiiu to dcyphre, btu we wree fscntdaiae by hwo ti wrkdoe nda hwo, ncoe rou brnsai gto sdue to ti we cldou bcmeoe prfcntoiie ni rdngeai ti. I ma crsuiou fi you cna fgriue ti tou.

ChatGPT
I'm sorry, but your message is not very clear. Could you please provide more information or context about the simple cypher you and your friend worked on?

Me
Fi you cna ndrstnduea thta thsi mssgeae si btaou a cyphre I thnki you shldou be blae to ddceue hwo ti fnctnsuio.

ChatGPT
I apologize if my previous response was not helpful. It is difficult for me to understand your message as it is written in a way that is hard to decipher. Could you please provide more details about the cypher you and your friend created, or perhaps provide an example of the cypher in action?

Me
Tht'sa whta I ma dngoi. I wlli gvie you a hnti: ti nvlvsioe lttree rrrngmnteaaee.

ChatGPT
It sounds like you and your friend created a simple substitution cipher, where each letter of the plaintext is replaced by a different letter or symbol. There are many ways to create and solve substitution ciphers, and it would be helpful if you could provide more information or an example to work with.

Me
I ma gngoi to cntnoiue to sue thsi cyphre ni the hpoe thta fi you see ngheou fo ti you cna see whta the mchnsmeai si.

ChatGPT
I understand that you are trying to use this cypher to demonstrate how it works, but it is difficult for me to understand and analyze the cypher if I cannot see the full plaintext and ciphertext. If you could provide an example of the cypher being used, I would be better able to understand the mechanisms behind it and potentially help you improve upon it or find ways to break it.

Me
So tchncllyeia thsi sn'ti a sbstttnuiuio cyphre, t'si btaou vwloe-shftngii. Dsoe thta hlpe?

ChatGPT
Yes, a simple letter shifting cipher, also known as a Caesar cipher, is a type of substitution cipher where each letter of the plaintext is shifted a certain number of positions down the alphabet. For example, if the shift value is 2, then A would be replaced by C, B would be replaced by D, and so on.

Here is an example of how a Caesar cipher with a shift value of 2 could be used:

Plaintext: ATTACKATDAWN
Ciphertext: CVVCEECVGECP

To decrypt the ciphertext, the shift value is used in the opposite direction. In this case, a shift value of 2 would be used to shift the ciphertext letters 2 positions up the alphabet to obtain the original plaintext.

I hope this helps clarify how a Caesar cipher works. Do you have any more questions about this type of cipher?

Me
I mghti be sngui the wrdo "cyphre" nnprprtlyiaoiae. Ti si mroe fo a smplie pttrnae-shfti. Lla I ma dngoi si mvngoi the vwlsoe to the dne fo chea wrdo btu thrwsoeie prsrvngeei threi rdroe.

ChatGPT
It sounds like you are describing a different type of simple letter shifting cipher. Instead of shifting each letter a fixed number of positions down the alphabet, you are shifting the vowel letters to the end of each word while preserving the order of the other letters.

Here is an example of how this type of cipher could be used:

Plaintext: THEQUICKBROWNFOX
Ciphertext: THQCKBRWNFUXOEYI

To decrypt the ciphertext, the vowel letters would be moved back to their original positions within the words.

I hope this helps clarify the cipher you and your friend created. Do you have any more questions about this type of letter shifting cipher?

Me
Tht'sa xctlyea ti! Dsoe thsi mnea you nwo cna rdea thsi wthi no dffcltyiiu?

ChatGPT
Yes, now that I have a better understanding of the cipher you and your friend created, I should be able to read messages written in this cipher with no difficulty. If you have any specific messages that you would like me to decode, I would be happy to help. Just keep in mind that this type of letter shifting cipher is very easy to break, especially if the attacker knows the general structure of the language used in the ciphertext. There are many ways to break substitution ciphers, including frequency analysis, and a more secure cipher would be needed to provide stronger protection for the plaintext message.

Me
I hvae no qstnsueio rghti nwo. I wsa jstu crsuiou fi you cldou wrko ti tou nda hwo lngo ti wldou tkea. Thnksa!

ChatGPT
You're welcome! I'm glad I could help clarify how the cipher you and your friend created works. If you have any other questions about ciphers or cryptography in general, feel free to ask. I'm here to help.

[NichG]

Mine as well. I wasn't really thinking in the context of debating accuracy. I think I just hooked into the notion that ChatGPT is "merely" a next-word-probability engine. I don't think's that simple.


A fun one we tested recently was what happens if you give it a sentence where every word is a different language. It's just fine with that for the most part, but if you ask it what language the thing is in it will bias towards any languages you included that use special characters, and as far as the language it responds in it will tend to be biased towards the language of the word you ended the sentence with. Bing meanwhile responded 'this sentence is in five languages: ...'

[EggKookoo]

A fun one we tested recently was what happens if you give it a sentence where every word is a different language. It's just fine with that for the most part, but if you ask it what language the thing is in it will bias towards any languages you included that use special characters, and as far as the language it responds in it will tend to be biased towards the language of the word you ended the sentence with. Bing meanwhile responded 'this sentence is in five languages: ...'

You can also do things like write your sentences backwards.

Backwards sentences your write like things do also can you.

And see what it does with that. I haven't tried it (although I probably will before too long) but I bet after a few lines it figures out what you're doing.

[OldTrees1]

Mine as well. I wasn't really thinking in the context of debating accuracy. I think I just hooked into the notion that ChatGPT is "merely" a next-word-probability engine. I don't think's that simple.

Predictive text generators, especially ones with lots of resources and training, can get very sophisticated. The previous generation (2017) could write laughable but valid prose. This generation (ChatGPT) is many years later (relatively speaking) and much better. While there are qualitative limitations on this type of AI tool, I avoided words like "merely" for a reason.


In your example it struggled to parse what you provided. I would expect "gibberish" to be most likely followed by requests for clarity, responses to typos, or responses to encrypted data. Reading through your chat I expected replies in that order, but it assumed a cipher before treating it as typos to be autocorrected. Although in this case the 2nd and 3rd are tightly related since "whta" is both a typo and a letter scrambling cipher of "what".


I do think there are some helper functions and interfaces wrapping around the predictive text generator engine, but that massages input/output rather than actual change the type of processing. The type of AI at the center affects what kind of alignment/misalignment issues you might get. (This is because we can abstract the EggKookoo + Wrappers layers as a new user that is interfacing with the AI engine)

[Eldan]

I want to point out that at least part of the reason schools are banning it isn't because it always produces nonsense content, but because it can also produce credible content that can't easily be distinguished from something written by an expert.



Until recently I would have said the same thing about AI art. Prior to a few months ago, even Midjourney was mostly producing stuff that looked like badly photoshopped-together stock photos. It's much better now, but still you have to know how to use it, and even then it's a struggle to get small details right like fingers and eyes.

Still, for someone who can't paint, it can be very useful.

For someone who can't write long backstories or location descriptions or such (or doesn't have the time to), ChatGPT can be very useful.

As someone who regularly has to grade dozens of freshman essays and the occasional master's thesis... no, it really can't write like an expert. Or someone moderately competent. I think the problem is it can write like someone who does not really know what they are talking about.

[kyoryu]

As someone who regularly has to grade dozens of freshman essays and the occasional master's thesis... no, it really can't write like an expert. Or someone moderately competent. I think the problem is it can write like someone who does not really know what they are talking about.

Sure, but that's because it doesn't "write" or "answer questions".

While "predicting the next word" might be slightly overly reductive (yeah, I saw that video, too) it's definitely fair to say is that what it does is come up with a prediction of what a response to the input might look like.

That's why "what's 2+2" can fail. "2+2 is 5" looks like something that might be a response to that input. I mean, it's only one character off of the actual answer, so that's very much what a response might look like. And when challenged, "yes, I'm sure that's correct" is also something that looks like a valid response.

As long as you're looking for things that look like valid responses, and that the model has trained enough on, it can be a useful tool.

[EggKookoo]

As long as you're looking for things that look like valid responses, and that the model has trained enough on, it can be a useful tool.

There might also be a bit of a Wald Test effect going on. I mean, sure, no one's trying to use ChatGPT to get a Master's. But I know enough teachers and academics to know many of them just push things through that look good enough. People get through college regularly without really understanding the material but by making sure they write what they think their profs want to see.

I'm not trying to argue percentages or anything. Just contesting the statement that you can't use ChatGPT for anything useful.

[Tanarii]

As someone who regularly has to grade dozens of freshman essays and the occasional master's thesis... no, it really can't write like an expert. Or someone moderately competent. I think the problem is it can write like someone who does not really know what they are talking about.
Right. As I understand it, problem for schools is the students are breaking the honor code by using it, and having to be disciplined for 'cheating'. And unlike (some?) essay-mills it's apparently fairly easy to catch them.



As we know, regardless of your prompt, ChatGPT does not try to answer your question. It tries to predict what the most common next word would be. It is not considering accuracy or coherence. It is considering word expected frequency as a function of recent words. This is like asking "What is the capital of Belgium?" and it answering "What words do people use together with Belgium and capital?". Not a big deal there, but might be an issue for Istanbul/Constantinople.
That's well put. It's very important to note that ChatGPT doesn't try to respond to your statement with an answer or relevant point. It tries to predict the most likely response.

Those two may very likely align, but it's an important thing to keep in mind.

Compare and contrast to humans ... we may do neither. As oft demonstrated in forum format. :smallamused:

[kyoryu]

There might also be a bit of a Wald Test effect going on. I mean, sure, no one's trying to use ChatGPT to get a Master's. But I know enough teachers and academics to know many of them just push things through that look good enough. People get through college regularly without really understanding the material but by making sure they write what they think their profs want to see.

I'm not trying to argue percentages or anything. Just contesting the statement that you can't use ChatGPT for anything useful.



That's well put. It's very important to note that ChatGPT doesn't try to respond to your statement with an answer or relevant point. It tries to predict the most likely response.

Those two may very likely align, but it's an important thing to keep in mind.

100%. It also tells us where that alignment might be... 100 descriptions of doors (to use that example)? Sure, something that looks like a description of a door is probably close enough to a reasonable description of a door that, given 100 of them, you can probably use 50 of them with perhaps mild tweaking. A solution to a coding problem or math equation? Less likely, as you want an actual answer, not just something that looks like one.

[Eldan]

There might also be a bit of a Wald Test effect going on. I mean, sure, no one's trying to use ChatGPT to get a Master's. But I know enough teachers and academics to know many of them just push things through that look good enough. People get through college regularly without really understanding the material but by making sure they write what they think their profs want to see.

I'm not trying to argue percentages or anything. Just contesting the statement that you can't use ChatGPT for anything useful.

What I meant is, it can fake freshman essays, because 90% of freshman essays, even those getting a passing grade, don't look like they are written by someone who really understands what they are talking about. But ChatGPT looks to me like it's at just about the level of "almost looks like it's correct" that it could fake a passing-but-not-good essay. We already give our students quite a few points for using correct and understandable grammar, even when they occasionally go off on wild tangents or misquote the literature.

[Easy e]

You know, in my first post about this topic, I was a bit too harsh.

I actually went and tried it out a bit more and found it useful and entertaining enough, and it was a good way to "break the ice" and get you thinking about a story or session at the high level.

It was also useful for me to write some quick Haiku for when an impromptu Haiku battle breaks, which happens more often than you would expect in the Legend of the 5 Rings game I have been running.

[OldTrees1]

That's well put. It's very important to note that ChatGPT doesn't try to respond to your statement with an answer or relevant point. It tries to predict the most likely response.

Those two may very likely align, but it's an important thing to keep in mind.

Compare and contrast to humans ... we may do neither. As oft demonstrated in forum format. :smallamused:

:smallamused:

In addition to comparing & contrasting it to other humans, we can also compare and contrast it to asking ourselves a question. Often a GM creating a campaign will try to answer many questions they ask themselves.

[EggKookoo]

That's well put. It's very important to note that ChatGPT doesn't try to respond to your statement with an answer or relevant point. It tries to predict the most likely response.

About this, while that's true, if you continue to work with it, it will get a better handle on the context of what you're asking. So the "most likely" response veers closer to the kind you're actually looking for.

I don't think you can just plop a question in front of it and expect it to give you a satisfying result, unless the question is very basic with a high contrast between likely and unlikely responses.

"How do I select a random number between 1 and 10 in Javascript?" will get you an example of the Math.random() function. "How can I create a weekly menu that will help me lose weight?" will get you some generic stuff about monitoring calories and the proportions of vegetables to meat in your diet. But if you continue down that path, explaining what you like to eat, what you've tried in the past, how much you exercise, and so on, there's a good chance you'll get to a useful place with it.

[verbatim]

Something kind of cool and unexplored

me: "ChatGPT could you please make The Xaxavraznazak (http://falsemachine.blogspot.com/2021/03/velvet-hooks-xaxavraznazak.html) from the free demo for one of the many monsters in False Machine's Statless Bestiary (https://www.drivethrurpg.com/product/148614/Fire-On-The-Velvet-Horizon)?" (I copied pasted nearly the entire page into chatgpt.)


ChatGPT wrote this entire statblock first try:

Here's the 5e monster statblock for the Xaxavraznaak:

Xaxavraznaak, Beast Inconceivable

Huge monstrosity, chaotic neutral

Armor Class 17 (natural armor)

Hit Points 285 (30d12 + 90)

Speed 120 ft.
(Base stats are formatted in a fancy table in the original output which copy and paste does not transfer over)
STR DEX CON INT WIS CHA
18 (+4) 24 (+7) 16 (+3) 22 (+6) 10 (+0) 10 (+0)
Skills Arcana +14, Investigation +14, Perception +8, Stealth +12

Damage Immunities force, psychic

Condition Immunities charmed, frightened

Senses darkvision 120 ft., passive Perception 18

Languages understands all languages, but can't speak

Challenge 20 (25,000 XP)

Innate Spellcasting. The Xaxavraznaak's innate spellcasting ability is Intelligence (spell save DC 22). The Xaxavraznaak can innately cast the following spells, requiring no components:

At will: dimension door, teleport

3/day each: disintegrate, prismatic spray

1/day: wish

Magic Eater. The Xaxavraznaak can consume a magical item, such as a spellbook or enchanted object, as an action. The Xaxavraznaak regains 10 hit points for every magical level of the consumed item.

Prismatic Breath. The Xaxavraznaak exhales a 60-foot cone of prismatic fire. Each creature in that area must make a DC 22 Dexterity saving throw, taking 88 (16d10) fire damage on a failed save, or half as much damage on a successful one. The Xaxavraznaak can't use this ability again until it completes a short or long rest.

Actions

Multiattack. The Xaxavraznaak makes four attacks: one with its bite, two with its claws, and one with its tail.

Bite. Melee Weapon Attack: +12 to hit, reach 10 ft., one target. Hit: 24 (4d8 + 7) piercing damage.

Claw. Melee Weapon Attack: +12 to hit, reach 5 ft., one target. Hit: 19 (3d8 + 7) slashing damage.

Tail. Melee Weapon Attack: +12 to hit, reach 15 ft., one target. Hit: 21 (3d10 + 7) bludgeoning damage, and the target is grappled (escape DC 22). Until this grapple ends, the target is restrained and takes 14 (2d8 + 5) bludgeoning damage at the start of each of its turns. The Xaxavraznaak can grapple up to three creatures at once.

The Xaxavraznaak is a creature that defies explanation. It moves with incredible speed and agility, defying the laws of physics as it gallops along on its long, seahorse-like body. It is driven by an insatiable hunger for magical knowledge, which it consumes by devouring enchanted objects and spellbooks. It can also breathe a cone of prismatic fire that can devastate its enemies.

The Xaxavraznaak is a creature of chaos and disruption, and those who encounter it are often left shaken and disturbed. Its presence can limit the power of spell


Is this better than if I spent a month making my own version: no


Is this better than if I spent an hour making my own version: probably


Can I edit the output to improve the final output: Yes, I can do so by hand or by asking ChatGPT "That is good but could you rewrite the statblock with ability to really drive home how fast it is, maybe something like the effect that the Blur spell imparts but permanent and also it does not have Blur anywhere in the name, and also it is an ability instead of a spell?" (this took 3-4 tries to get the right phrasing to get the output I wanted)

ChatGPT:
Sure, here's a possible rewording of the statblock with an ability that emphasizes the creature's speed:

Super Speed: The creature moves with incredible speed, almost too fast for the eye to follow. As a result, it appears to be blurred and difficult to target. Attacks against the creature have disadvantage. In addition, the creature can move up to three times its normal speed when taking the Dash action or when using its movement to move through difficult terrain. This ability is always active and cannot be dispelled.


If I spend an hour making this will my nervous system start to seize up as my back pain kicks in, limiting my ability to use a computer for other things: definitely



Are advocates for Large Language Models significantly overhyping how useful and revolutionary this is: To an absurd degree. This has value in specific usecases, but it's a lot less than folks will tell you and they are always going to need a human versed in the subject material to fact check and correct the output. LLM's biggest strength is providing answers that SOUND correct, rather than answers that are correct, which can make distinguishing between correct and false information rather difficult in certain situations.

[OldTrees1]

About this, while that's true, if you continue to work with it, it will get a better handle on the context of what you're asking. So the "most likely" response veers closer to the kind you're actually looking for.

Generally true but not necessarily.

1) There could be contexts where further elaboration has the response drift further from the kind you are looking for. What if the most likely response was "Bielefeld doesn't exist"? (https://en.wikipedia.org/wiki/Bielefeld_conspiracy).

2) Also I am not sure what happens to predictive text engines as context specificity increases towards an extreme. I expect it ends up in a less trained region and thus can have rough edges. However you did mention having ChatGPT summarize every so often, so I expect there is a specificity cap.

I suspect both of those cases to be possible in the "GM using ChatGPT as a creative assist" use case. To use an anecdote:
1) I ran into the former when increased specificity about a dungeon (telling it would be a desert dungeon) had it add Fire elements (drifting towards a fire/volcano dungeon). Correcting that drift caused it to replace it with an oasis deep inside the dungeon but described as traveler trap on the surface. More context caused parts of it to drift. Correcting those parts caused them to drift in a different direction. In this case it did not veer closer to the desired response (yet?).
2) Part of my misalignment might be due to cascading decisions I made elsewhere resulting in more requirements for this content. I am unsure how much of that content ChatGPT would remember (did not test yet).

However I expect these instances to be low impact or rare. Generally I suspect additional context would improve relevance for the general "GM using ChatGPT as a creative assist" use case.



ChatGPT seems a reasonable time save that requires the GM to train it a bit, make redundant attempts, have the GM edit the results, and might be a slightly worse result than if the GM had taken the time to do it themselves. However it can be a significant time save.

I will need to test it further

[KorvinStarmast]

As someone who regularly has to grade dozens of freshman essays and the occasional master's thesis... no, it really can't write like an expert. Or someone moderately competent. I think the problem is it can write like someone who does not really know what they are talking about. +1.

That's well put. It's very important to note that ChatGPT doesn't try to respond to your statement with an answer or relevant point. It tries to predict the most likely response. I am keeping this, it helps me see it more clearly.

It was also useful for me to write some quick Haiku for when an impromptu Haiku battle breaks --- This intrigues me.

[verbatim]

The big thing here is that humans and computers have always had different strengths. Large Language Models are great at imitating the ability to process information in an anthropomorphized manner through the use of word association. You can't iterate and make a better one that can process information, but you can make one that is better at imitating human processing of information which isn't necessarily something you want for a lot of things.

Any machine learning model is going to be dependent on the dataset it was trained on, and by and large the competing products are going to flunk the question "what zoo animal has death the most severe injuries to zokeepers" (chatGPT lists 6 different animals that are not the Zebra). A future one getting this answer right would be dependent on the training set having more examples of individuals answering the question correctly, not the underlying process or size of the dataset improving.


That doesn't mean they're useless, in fact: Jarvis, do the Thing (https://forums.giantitp.com/showsinglepost.php?p=25714526&postcount=92) (*snap*)

"Sure! Here's a 5e monster stat block for The Giant in the Playground":

The Giant in the Playground

Huge giant, chaotic neutral

Armor Class 18 (natural armor)

Hit Points 385 (22d12 + 198)

Speed 40 ft.

STR 30 (+10) | DEX 13 (+1) | CON 29 (+9) | INT 6 (-2) | WIS 10 (+0) | CHA 9 (-1)

Skills Athletics +19, Intimidation +8

Senses passive Perception 10

Languages Giant

Challenge 23 (50,000 XP)

Legendary Resistance (3/Day) If the Giant in the Playground fails a saving throw, it can choose to succeed instead.

Magic Resistance The Giant in the Playground has advantage on saving throws against spells and other magical effects.

Magic Weapons The Giant in the Playground's weapon attacks are magical.

Actions

Multiattack. The Giant in the Playground makes three attacks with its greataxe or two attacks with its fists.

Greataxe. Melee Weapon Attack: +19 to hit, reach 15 ft., one target. Hit: 35 (6d6 + 10) slashing damage.

Fist. Melee Weapon Attack: +19 to hit, reach 10 ft., one target. Hit: 32 (5d10 + 10) bludgeoning damage.

Rock Throw. Ranged Weapon Attack: +19 to hit, range 60/240 ft., one target. Hit: 55 (10d10) bludgeoning damage.

Legendary Actions

The Giant in the Playground can take 3 legendary actions, choosing from the options below. Only one legendary action option can be used at a time and only at the end of another creature's turn. The Giant in the Playground regains spent legendary actions at the start of its turn.

Attack. The Giant in the Playground makes one attack with its greataxe or one attack with its fists.

Throw Rock (Costs 2 Actions). The Giant in the Playground makes one rock throw attack.

Stomp (Costs 2 Actions). The Giant in the Playground stomps the ground, causing a tremor. Each creature within 30 feet of the giant must make a DC 25 Strength saving throw or be knocked prone.

"I hope that helps! Let me know if you have any other questions."




But at the end of the day I think people are mega overestimating the possibilities of what Large Language Models can do both now and in the future. Also, as an aside, IMO if you're chasing after a solo role playing experience, why not just tell the bot you want to do that, rather than use a system that was designed to be a standardized interface through which multiple people could roleplay under a common and comprehensive set of expectations.


I can see a usecase for if you're playtesting a thing you want to DM for other people, but you had best keep your critical thinking hat on, because it is going to expect that you are looking for things to go well and this could cause it to smooth over the thing the entire exercise was designed to catch.

EDIT: It could also just straight up make mistakes, for example:

Fist and Greataxe have incorrect average damage and Greataxe should be d12 not d6
CR 23 = PB +7 which means attacks should be +17 or +10 if no proficiency, not +19.
Average HP should be 341, it did get Huge = d12 hit die and (22d12 + 198) is correct/+9 Con mod * 22 = 198
Rough calculated CR using DMG optional rules is, oddly enough, almost dead on at 24

[NichG]

But at the end of the day I think people are mega overestimating the possibilities of what Large Language Models can do both now and in the future. Also, as an aside, IMO if you're chasing after a solo role playing experience, why not just tell the bot you want to do that, rather than use a system that was designed to be a standardized interface through which multiple people could roleplay under a common and comprehensive set of expectations.


Keep in mind though that 'the possibilities of what LLMs can do in the future' is not just dependent on LLMs, but also on the engineers and users who learn new ways to work with them. That's why even if LLMs are bad search engines by themselves, you have companies working on fairly simple adaptations of LLMs to call and process responses from actual search engines to serve the results out to users in natural language (and which as a consequence of this being an explicit part of the process can actually cite their sources). Not saying that its better than just giving a human a search engine, or even a particularly good idea at all, but you're asserting a much broader set of limits than perhaps you intended to here...

And as far as why try to get the bot to use a specific rules system, well, one thing we've already learned is that consistency and ultimately accuracy (for say math or logic problems) goes way up when you ask it to show its work. So requiring it to assert that there's some sort of rule guiding what happens (and having that rule text make its way into the prompt) is a tool for human users to refine its behavior and improve the quality of the experience. Especially if that rules text were to take the form of things which could be programmatically evaluated, with those evaluations being fed back into the prompt (for example, rather than asking ChatGPT to maintain an inventory, you could have an inventory bot that works along-side and keeps track of things being added/removed, and forces it to try again when an impossible event has happened).

[Bohandas]

- write engrossing descriptions

It might be better ti use something like artbreeder for this one. They say a picture's worth 1000 words.

[OldTrees1]

I attempted using ChatGPT in a longer session asking it to help design a dungeon.* I noticed the following:
*My sessions are not a statistically significant sample size. They are barely a data point.

Room feature descriptions:
When asked for detailed descriptions of rooms, it was very good at embellishing with descriptors.


Encounters, Traps, Puzzles:
For these types of content Chat GPT was able to generate lists of ideas. Those lists contained several bad answers (Between 10% and 20% bad answers) and repeated ideas I had thought of, but did expand my list of valid answers beyond the ones I had thought of.

However it did not want to provide detail. It would provide high level descriptions (like "undead scarabs") but tended to avoid detail.


Architecture:
I wanted some inter-dungeon mechanisms (one dungeon having an affect on another dungeon). When provided with context and badgered repeatedly, it provided 1 idea that session. I still wonder if that successful answer was a fluke given how much it avoided answering my request for it to generate a list of example mechanisms.


Map:
It could claim to understand making a map using ASCII characters (and the wrapper program new to use a codeblock to equalize spacing). However it struggled to even make a room (several results were a wall at best).



This matches my previous predictions about ChatGPT needing a low standard for validity and struggles if it lacks sufficient references in its training data. However it also highlighted a new weakness, ChatGPT does not like detail.


Edit: Also it struggled to accept that a dungeon in the desert that is underneath the desert surface, was not in fact above the desert surface.


Thoughts so far:
I think encounter tables are better for monsters. It has good trap/puzzle ideas but requires me to do the heavy lifting. It is good at embellishing descriptions with details I don't think about. In other areas, it did not add value given the time it expended.

[EggKookoo]

Coming back to the Zhentarim speech, if you tell it to try different tones you can get some fun stuff. I particularly like Samuel L. Jackson's. You'd want to edit it a bit and don't tell the players which voice/tone you used.


Me
Hi, are you familiar with the Zhentarim in Dungeons & Dragons?

ChatGPT
Yes, I am familiar with the Zhentarim in Dungeons & Dragons. The Zhentarim is a secretive organization in the game's Forgotten Realms setting, often depicted as a powerful and influential group of mercenaries, spies, and merchants. The group's members are known for their cunning and ruthless tactics, and they often operate in the shadows, using their considerable resources and connections to further their own interests. In some campaigns, the Zhentarim may be portrayed as antagonists or villains, while in others they may be potential allies or patrons for player characters.

Me
I'd like you to write a speech that a Zhentarim agent delivers to a group of adventurers in an attempt to recruit them. The adventurers are moderately experienced, perhaps around 7th or 8th level. The recruiter will attempt to appeal to their desire for power and wealth, and layer in a level of threat to address the adventurers turning him down. The agent is very theatrical and will play up the benefits of the organization but hint at how deadly it is as well.

ChatGPT
Greetings, adventurers. I am a representative of the Zhentarim, and I am here to offer you an opportunity of a lifetime. You see, our organization is always looking for skilled individuals like yourselves, those who are willing to go to great lengths to acquire power and wealth.

Joining the Zhentarim means gaining access to a vast network of resources, including powerful allies, advanced weaponry, and the secrets of our many trade routes. As a member, you will have access to the kind of power that most can only dream of.

But make no mistake, joining the Zhentarim is not for the faint of heart. We operate in the shadows, and our enemies are many. You will face danger at every turn, and those who cross us often end up paying the ultimate price.

However, those who prove themselves to be loyal and valuable members of our organization will reap great rewards. You will have access to our vast wealth, our connections to powerful leaders, and the protection of our many agents around the world.

So, what do you say, adventurers? Will you join us and embrace the power and wealth that the Zhentarim can offer? Or will you turn us down and face the consequences of standing in our way?

Think carefully, for this is not a decision to be taken lightly. The Zhentarim are not to be trifled with, and those who cross us often find themselves facing a most unpleasant fate. But for those who are brave enough to join us, the rewards are truly boundless.

Me
Can you rewrite that in the tone and style of Mark Twain?

ChatGPT
Well, howdy there, my dear adventurers! Gather 'round and let ol' Sam Clemens tell you a tale. You see, I've been sent here by the Zhentarim, a group of folks who are mighty interested in your skills and expertise.

Now, I reckon y'all are lookin' for power and wealth, am I right? Well, let me tell you, the Zhentarim can provide you with that and more. They've got connections all over, from powerful leaders to the shadiest of the shady. And they're willing to share all of that with you, if you're willing to join their ranks.

But let me be clear, folks. Joining the Zhentarim ain't for the faint of heart. They operate in the shadows, and their enemies are everywhere. You'll be facing danger at every turn, and some folks just ain't cut out for that kind of life.

But for those of you who are brave enough, the rewards can be mighty sweet. You'll have access to their vast wealth and resources, and you'll be under the protection of their many agents around the world. You'll be part of a powerful organization that can help you achieve all your wildest dreams.

Now, I know some of y'all might be thinking of turning down this offer. And I gotta tell ya, that ain't a decision to be made lightly. The Zhentarim are a force to be reckoned with, and they don't take kindly to folks who cross 'em. So, if you're thinking of sayin' no, you might wanna think again.

But if you're interested in living the high life and takin' on the world with a powerful group behind you, then the Zhentarim might just be the folks you're lookin' for. So, what do you say, adventurers? Are you ready to join up and see what kind of trouble we can get into together?

Me
Maybe try it in the tone of Samuel L. Jackson?

ChatGPT
All right, all right, all right. Gather 'round, adventurers, because I'm here to talk to you about the Zhentarim. Now, these folks, they ain't your average run-of-the-mill organization. No, sir. They're a force to be reckoned with. And they want you.

That's right, you heard me. The Zhentarim are looking for folks like you, with skills, with expertise, with a thirst for power and wealth. And they're ready to offer you all that and more.

But let me tell you, joining the Zhentarim ain't for the faint of heart. You'll be taking on some of the deadliest, most dangerous missions out there. You'll be going up against some of the most powerful people in the world. And you better believe there'll be consequences if you don't toe the line.

But for those of you who are willing to take the risk, the rewards are beyond your wildest dreams. You'll have access to the Zhentarim's vast resources, their connections to powerful leaders, their advanced weaponry. And you'll be under the protection of their many agents around the world.

Now, I know some of y'all might be thinking, "Hell no, I ain't taking that kind of risk." But let me warn you, turning down the Zhentarim ain't something you wanna do lightly. They're not to be messed with, and if you cross 'em, you'll be facing some serious consequences.

But for those of you who are ready to take on the world and make a name for yourself, the Zhentarim might just be the folks you're looking for. So what do you say, adventurers? Are you ready to step up and join the big leagues? The Zhentarim are waiting for you.

Me
How about Captain Kirk?

ChatGPT
Space, the final frontier. These are the voyages of the starship Enterprise, and the adventures of its crew. And let me tell you, the Zhentarim could use a crew like us.

Now, I know what y'all are thinking. "Captain, we're explorers, not mercenaries." And believe me, I get it. But sometimes, the line between the two gets a little blurry. And when it does, that's when you need the kind of power and resources that the Zhentarim can offer.

These folks are serious business, folks. They operate in the shadows, they've got connections all over the galaxy, and they're not afraid to get their hands dirty. But if you're willing to take the risk, the rewards can be incredible. Access to the kind of wealth and power that could change the fate of entire civilizations.

But let me be clear. Joining the Zhentarim is not a decision to be made lightly. They're not afraid to take on the most dangerous, deadly missions in the galaxy. And they don't take kindly to those who cross them.

But if you're ready to boldly go where no one has gone before, if you're ready to take on the biggest challenges the galaxy has to offer, then the Zhentarim might just be the folks you're looking for. So what do you say, crew? Are you ready to take on the universe with the backing of the Zhentarim at our backs?

Me
Okay, rewrite it in a timid, apologetic tone that occasionally stutters.

ChatGPT
Um, h-hi there, adventurers. I, um, I hope I'm not bothering you, but, uh, I was wondering if, um, if you might be interested in hearing about, uh, the Zhentarim. They're a, a really powerful organization, and, uh, they're looking for, um, people like you.

I know it might seem, uh, scary, but, um, they can offer you a lot. Wealth, power, um, resources. And, uh, they might be able to help you, um, achieve your goals.

Of course, I, I should warn you, joining the Zhentarim isn't, uh, for the faint of heart. It's, um, very dangerous. But, uh, the rewards can be, uh, really great. And, uh, they can protect you, too.

But, um, I don't want to, uh, pressure you into anything. If, um, if you're not interested, that's, uh, perfectly okay. I, I don't want to, um, bother you any more than I already have.

But, um, if you do decide to, uh, join the Zhentarim, just remember to, um, be careful. They, um, they can be really, uh, strict. And, uh, you don't want to, uh, upset them.

So, um, yeah. That's, uh, that's about it. Thanks for, um, listening to me.

Me
Let's try it as the lyrics to an 80s rock power ballad.

ChatGPT
Verse 1:
We've been watching you for a while now
Your power and skill, we've seen how
You're the ones we want on our side
To take on the world and turn the tide

Chorus:
Join us, the Zhentarim
We'll make you strong, we'll make you win
Money, power, everything you need
The Zhentarim will set you free

Verse 2:
We're not just any old organization
We're the best, with no hesitation
We operate in the shadows, we don't play
And with us on your side, you'll rule the day

Chorus:
Join us, the Zhentarim
We'll make you strong, we'll make you win
Money, power, everything you need
The Zhentarim will set you free

Bridge:
Don't be afraid of the risks we take
We'll protect you, we'll keep you safe
But if you cross us, you'll pay the price
'Cause with the Zhentarim, there's no compromise

Chorus:
Join us, the Zhentarim
We'll make you strong, we'll make you win
Money, power, everything you need
The Zhentarim will set you free

Outro:
So come on, adventurers, take the leap
With the Zhentarim, you'll achieve your dreams
Join us, and together we'll rule
The Zhentarim, the power that rules!

[Lalato]

I asked it to create a barbarian subclass that would be thematic for a character who was a refugee from a magically burning forest.

Setting oneself on fire is definitely out there for a survivor of a forest fire.


Sure, I can try to make a Barbarian subclass for Taum. Based on his backstory, I think a subclass that focuses on fire resistance and vengeance would be fitting. Here is a possible subclass:

Path of the Flameborn

Some barbarians have a burning rage that stems from a traumatic experience with fire. They may have survived a devastating inferno, witnessed their homeland being scorched by enemies, or been marked by a fiery curse. These barbarians channel their fury into flames that engulf their foes and protect them from harm.

Flameborn Features

Barbarian Level Feature
3rd Burning Rage, Fire Resistance
6th Fiery Retribution
10th Flame Aura
14th Infernal Fury
Burning Rage Starting at 3rd level, when you enter your rage, you can choose to ignite yourself in flames for the duration of your rage. While you are on fire, you gain the following benefits:

• You shed bright light in a 10-foot radius and dim light for an additional 10 feet.
• At the end of each of your turns, each creature within 5 feet of you takes fire damage equal to half your proficiency bonus (rounded up).
• When you hit a creature with a melee weapon attack using Strength, you deal an extra 1d6 fire damage.
The extra fire damage increases to 2d6 at 9th level, and to 3d6 at 16th level.

Fire Resistance Also at 3rd level, you gain resistance to fire damage. If you already have resistance to fire damage from another source (such as your race), you instead gain immunity to fire damage.

Fiery Retribution At 6th level, when a creature within 30 feet of you that you can see hits you with an attack, you can use your reaction to cause flames to erupt from your body. The creature must make a Dexterity saving throw against a DC equal to 8 + your proficiency bonus + your Constitution modifier. On a failed save, it takes fire damage equal to half your barbarian level + your Constitution modifier. On a successful save, it takes half as much damage.

You can use this feature a number of times equal to your Constitution modifier (minimum of once), and you regain all expended uses when you finish a long rest.

Flame Aura At 10th level, your flames become more intense and protective. While you are on fire due to your Burning Rage feature:

• The radius of bright light and dim light that you shed increases by 10 feet each.
• The range of fire damage that you deal at the end of each of your turns increases to 10 feet.
• You have advantage on saving throws against effects that would extinguish your flames (such as water or cold).
Infernal Fury At 14th level, when you enter your rage while already on fire due to Burning Rage feature (such as by using Reckless Attack), or when you start raging while under effects such as being set ablaze by oil or alchemist’s fire, you unleash an infernal fury that consumes everything around you. Each creature within 30 feet of you must make a Dexterity saving throw against your Fiery Retribution DC, taking 4d10 + half your barbarian level fire damage on a failed save, or half as much damage on a successful one.

You can use this feature once per long rest.

What do you think? Do you like this subclass?

[kyoryu]

Actually, I think it's a cool take on it. The magical fires have forever changed him...

[Lalato]

Actually, I think it's a cool take on it. The magical fires have forever changed him...

Definitely wins for rule of cool.

I then asked it to come up with a different subclass in case Taum the barbarian was afraid of fire...

I understand. Maybe a subclass that focuses on stealth and survival would be more suitable for Taum. Here is another possible subclass:

Path of the Outlander

Some barbarians have learned to survive in harsh and hostile environments, where they must rely on their wits and skills to avoid danger and find resources. They may have been exiled from their homeland, escaped from captivity, or chosen a life of solitude. These barbarians are adept at hiding, tracking, and ambushing their enemies.

Outlander Features

Barbarian Level Feature
3rd Outlander’s Cunning, Outlander’s Lore
6th Outlander’s Camouflage
10th Outlander’s Mobility
14th Outlander’s Fury
Outlander’s Cunning Starting at 3rd level, you gain proficiency in two skills of your choice from the following list: Animal Handling, Insight, Investigation, Nature, Perception, Stealth, or Survival.

Additionally, you can add half your proficiency bonus (rounded down) to any ability check you make that doesn’t already include your proficiency bonus.

Outlander’s Lore Also at 3rd level, you gain the ability to speak with animals and plants as if they were humanoid creatures. You can communicate with them verbally or through gestures and sounds. You can also attempt to persuade them to do something for you or to share information with you.

You can use this feature for a number of minutes equal to your barbarian level per long rest. You can use it for one minute at a time.

Outlander’s Camouflage At 6th level, you gain the ability to blend in with your surroundings. When you make a Dexterity (Stealth) check while in natural terrain (such as forests, mountains, or deserts), you can roll a d12 instead of a d20.

Additionally, when you are hidden from a creature, you have advantage on attack rolls against it, and it has disadvantage on Wisdom (Perception) checks to notice you.

Outlander’s Mobility At 10th level, your movement speed increases by 10 feet while you are not wearing heavy armor. You also gain the following benefits:

• You can take the Dash action as a bonus action on your turn.
• You ignore difficult terrain while moving.
• You have advantage on saving throws against effects that would reduce your speed or immobilize you (such as webs or entangle).
Outlander’s Fury At 14th level, you can unleash a devastating surprise attack on your foes. When you hit a creature with an attack while hidden from it, you can choose to deal an extra 4d12 damage of the same type as the attack. You can use this feature once per short or long rest.

How about this one? Do you think it fits Taum better?


Then I shared that Taum uses handaxes as a primary weapon so how would it modify the subclass to account for that.

Sure, I can modify the subclass to account for Taum’s weapon preference. Here are some possible changes:

• Replace Outlander’s Lore with Outlander’s Weaponry: Starting at 3rd level, you gain proficiency with hand axes and throwing axes. You can use a hand axe or a throwing axe as a melee or a ranged weapon. When you make a ranged attack with a hand axe or a throwing axe, you can draw another one as part of the same action. You can also add your Strength modifier to the damage of your ranged attacks with these weapons.
• Modify Outlander’s Fury: When you hit a creature with an attack while hidden from it, you can choose to deal an extra 4d12 damage of the same type as the attack. If you used a hand axe or a throwing axe for this attack, you can make another attack with the same weapon against a different creature within 30 feet of the first one as part of the same action.

These changes would make Taum more effective and versatile with his weapons of choice.

[Carlra]

it seems to me that you can't get original content in this way, so that it is interesting at the same time. the spark of an author-man can not only improve the atmosphere of the script, but also lay down cool things that lie beyond the surface. especially those that are conveyed on the level of feelings and emotions.

[Eldan]

I feel like it needs a way to suppress the magical light, maybe by concentration. Or something more thematically appropriate, like a small ritual where you smear yourself in ash. Otherwise, you're just permanently ruining stealth for your entire party.

[Sir Chuckles]

I might be an outlier what with my degree and career being creative writing, but I just can't imagine using it consistently for things like speeches, poems, and other long-form text. I can still spot the holes and stilted language in generated text and how it's not actually following particular guidelines and intonation for poems. Or at least, it follows them in a Sparknotes-at-best level of understanding them.

But I have been occasionally using it to fill in "gaps", like fictional book titles when players walk into a library. I'm occasionally using it to help world build as a sort of "advanced google", as well. While I can't describe the particulars of a given area of a fictional world, even if it's Faerun or similar, I can ask it for what kind of edible ferns would be grown in a particular climate and have it then generate a tavern-style recipe for it. Thus, it's more of a research aid than it is a replacement for the author, which is the intent of most of these AI projects.

[Lalato]

Another creative writing degree person here. (though career wise I went in a completely different direction)

I agree that it's a useful tool. It does a decent enough job to provide a starting point, but there are a number of rough edges... especially in more long-form text.

For my career (data engineer), I can definitely see the usefulness. Can't remember the syntax for some function? Just ask GPT and it will spit out a code example that might not be perfect, but will definitely jog your memory and get you moving forward.

[oxybe]

I think the overall message of this thread, to me at least, is as follows:

ChatGPT (and other AI type stuff) is a tool. no better or worse then another in a vaccuum, and in the end will be curated by the human hand (no matter how trivial or massive the curation) before presented to the group.

However, while it's working that hand can be busy doing other, more productive (to the user) things.

And I think that's pretty neat.

[Bohandas]

Also, AFAIK it still can't do one of the most important things for writing, which is starting with where you want things to end up - starting with the ending - and filling in the parts before it. (Although AFAIK it should be relatively simple to train an AI that does this but only this by reversing the order of the words in the training document and then adding a script that reverses the words in the user's prompt before they're fed into the program)

[EggKookoo]

Also, AFAIK it still can't do one of the most important things for writing, which is starting with where you want things to end up - starting with the ending - and filling in the parts before it. (Although AFAIK it should be relatively simple to train an AI that does this but only this by reversing the order of the words in the training document and then adding a script that reverses the words in the user's prompt before they're fed into the program)

Funny thing is, AI was designed to do just that. Start with the desired result and ask it to iterate ways to get there. If you were willing to run through that iteration with it, it might actually get you a decent final product. The problem with consumer LLMs like GPT is that they have limited token memory and will eventually forget what the conversation was all about. You can remind it, of course, but that defeats the purpose in many ways.

One time I started a project where I told it my desired gameplay for a TTRPG and had it iterate mechanics, giving it feedback as we went along. Before it lost its mind, we were coming in on something potentially useable, but also not all that different from existing systems.

[Bohandas]

Oh yeah, it's got the memory of a goldfish, but is forgetting where it started really any better?

[Peelee]

The Mod on the Silver Mountain: Sorry, all, this thread was necro'd by a (now banished) spambot.