Quote Originally Posted by icefractal View Post
But the vast majority of TTRPG settings are genre fiction.

And IMO that's not by coincidence. A great writer can make virtually any premise compelling, but most GMs (including myself) aren't great writers. The typical situation is more like an amateur improv group with varying degrees of enthusiasm. The premise has to carry a lot more water.
Also, producing a compelling story for an audience is not the goal of D&D gameplay (and attempts to use D&D for this purpose tend to bend the rules is very substantial ways for this reason). The goal is for the participants to have a good time, usually in a fairly low-brow, lightly comedic way, and often with a large number of repeated, similar tasks with minor variation.

A wide range of options, notably, is a means to fight boredom. A game where the characters perform the same attack over and over in every fight because it represents the most optimal approach (there are many cRPGs like this, were the most efficient option is normal attacks + out of combat healing) rapidly becomes bland, especially given that TTRPG combat is inherently slow paced compared to video games - in an arena fight in something like Borderlands 3 each player might kill hundreds of opponents in 10 minutes. Doing that in tabletop might take days. D&D has a zillion monsters all with their own slightly different quirks because this provides variety to combat encounters.

The needs of a fun game and a compelling story are often in tension, this is one of the many reasons video game adaptations often fail. This is compounded in that a writer will often produce a fictional setting specifically to tell a single story. The world only needs to support that one scenario. A TTRPG setting however, needs to build dedicated fans who play many campaigns over time. That means producing new and different options for veteran players who have already used all the old ones. The problem with this is that option creep destroys both mechanics and fluff as they accrete.