Solving the "elf superiority problem"

[PhoenixPhyre]

But then saying "elves are rare and player character elves that can find new ideas faster are double rare snowflakes and those with high stats in addition are triple rare" might displease the player who wants to play an elf but not be recognised by other elves during their childhood as being the exceptionally fit elf that had ten ideas in a single week while most elves have one idea per week and being renowned for that.
Also how do you explain that those elves were supposed to be rare after your tenth elf player character fighter that got killed?

I think PP's concept here is more about solving the worldbuilding conundrum that can fairly be perceived in long-lived and ageless at the level of cultures, societies, and politics -- not so much about limited individual PCs.

PCs (and "adventurers" more generally) are an exceptional case. Specifically, PCs are drawn from the set of exceptions to the rules. For one thing, they grow really really really fast compared to everyone else. For another thing, their "potential" is indeterminate at the start, bounded above only by the limits of their reach. Most people, no matter how they work, can never reach "level 20". Heck, only a tiny fraction can reach a single class-level-equivalent power. Instead, the cap for a PC is based entirely on their own drive and reach for their goals. When they retire from adventuring (ie the campaign ends), they plateau pretty fast. So while active PCs, they can grow basically without limits. Once they're NPCs after a campaign ends, they're much more fixed.

The situation presented in the OP is really, really janky, in that while it does give an explanation for the long-lived races not dominating everything, it doesn't give a realistic outcome for how its proposed process would play out. Generalized plateaus work just as well with the underlying "normalized soul" explanation, but this then faces serious issues with the spectacular divergence that is the truly monstrous races like Dragons, given they are reliant on being well above average in most respects as a quite defining feature.

This can be worldbuilt around pretty easily by having the various scales within varieties form different distributions and play around with averages to fill in a lot of the flavor, such that every species has negligible differences in the theoretical average member, but the concentrations and emergent properties form all manner of stereotypes by the usual generalization of impressions method. For example, Elven longevity may give the appearance of on-average superiority, because their rolling totals get to be skewed to the ends of careers so much more heavily. You don't see the apprentices, because they're a smaller proportion of any given field.

And Dragons may specifically be the outcome of the upper end of a bi- or tri-modal distribution, such that being a Dragon is just the same as the various archmages, but that particular species almost always expresses this potential anatomically instead of professionally, such that you get ridiculously huge Great Wyrms instead of Dragon Archmages because all the supernatural capacity is taken up by their body.

There's also just not having science. Long-life superiority doesn't mean much of anything when your advancement is based on flat-out guesses, however educated they may be. Empiricism is a big thing for societies, and without it you don't exactly get very much advancement, because so many things are coming down to guesswork. The image of the experimenting Wizard is nothing like the image of the experimenting Scientist, and leaning on this with judicious application of Divination can keep the roots of the society firmly black-boxed and utterly immune to the exponential shenanigans that make spectacular longevity actually an advantage.

This then allows for Liches to progress with time, so there's a reason that people do it, and things being largely a black box with all sorts of obvious differences in the "usual" nature of the various races hides the fact you're not actually going to be any better than a fully trained mortal, you just have the time to be sure you reach that point, so while most Liches are Archmages, they're not any better than a mortal Archmage, and this is in no small part because the lower-quality Liches don't last very long. In choosing Undeath, they near enough guarantee survival to see their limits.

Yes, there are different distributions. But since dragons come from a different primordial stock, theirs and that of humanoids is quite differently allocated. And differently sourced. Dragons have huge amounts of potential energy, but most of it isn't theirs to begin with. Same with giants. Instead, they draw energy from other things. In the case of dragons, it's elemental energy channeled through their hoard (which may be physical or may be abstract). Without their connection to the greater pool of anima, they could never eat enough to have the energy to move. So theirs isn't learned anima, it's innate. In effect, they cheat, and pay a price for that cheating.

And dragons aren't particularly innovative. For most of them, it takes all they can do to hold the status quo. Some dragons learn "mortal" magic (becoming the metallics), but dragons don't generally develop their own spells. They are magic (channels for elemental anima in its rawest form). This is in part because of their ultimate origin--their ancestor race, the Wyrm, was the servant race to the Primordial of Destruction, of Ending. The titans shaped, the wyrm broke down those parts that were wrong. Once the titans and wyrm had their powers broken (long story), the races degenerated. But that legacy as agents of destruction rather than order (I intentionally don't use chaos here, that's different) is still with them. Their "genetic" makeup balks at building or inventing; even societies beyond the level of a Flight (basically an extended family group) are unusual. For most dragons, only their hoard matters--finding it (the life-or-death task of young dragons, and why they're so often the ones causing problems) protecting it, building it (by theft or domination, only rarely by trade or other "normal" means). Dragons hold grudges not because they're particularly vindictive, but because that's what they've got, along with habits. And old habits die really hard among the dragonfolk.

There are exceptions--there are always exceptions. The Landlord (adult silver) spends most of his time in human form, maintaining his real estate empire in a particular nation. That's because he hoards architecture. Buildings. He's more than happy to rent them out, but is pure death on anyone who causes property damage. The Prophet of Peace and his two companions in Bel's Kush have taken that nation as their collective hoard, and work continually to make it a place of peace and prosperity and order. They all adapt and change. So it's not a uniform thing--low probability events happen way more frequently than seems normal. But that's because humans are bad with probability.

As for liches, I know exactly why they don't progress. Becoming a lich involves anchoring your soul to an object and caging a Jotnar, an elemental (and beyond elemental) entity of negation and entropy, in its place and harnessing its drive to destroy for your own power. But you have to keep it fed (on souls and anima) lest the bonds weaken and it devour you. Eternal life and rebirth (if the phylactery is intact) at the cost of never being able to create again. All the anima you produce falls down the endless gullet of the jotnar. It's actually a refinement of the ritual that produced the first vampire lords (not these modern degenerate blood-suckers, creatures more powerful and more glorious)--instead of the constant hunger for life and sensation, you have dispassion and endless rebirth. Less likely to destroy your own mind, more stable. But the cost, your mortal capability for growth, is the same. That's the intrinsic price of letting a jotnar into your soul. You have become a gaping wound in reality where anima falls into the abyss.