I guess it comes down to return on investment.

That's probably closest you can get to an objective, or at least player-agnostic, metric of whether a game is good or not.

What do players (including the player in a DM/GM/Storyteller/whatever-the-title-is role) have to invest in a game?
- Time is one thing: time learning the rules; time playing the chargen minigame; time planning game sessions; time arguing over or looking up rules during a session.
- Emotional investment/buy-in is another (maybe).
- The cognitive demands of learning the game is another. This relates to time, but is also its own thing. Games that have more demanding mechanics just require more investment of players' cognitive resources.

What do players get out of it?
- Fun/enjoyment, insofar as this can be separated into fun/enjoyment to be had from the game itself versus fun/enjoyment from being at a particular table with other players.
- A sense of fulfillment/satisfaction, insofar as this is distinct from fun (maybe).
- As a negative, disappointment - that the game was unable to fulfill their particular fantasy for their character (insofar as that fantasy ought to have been realisable in that game), that some feature or mechanic that looked cool did not turn out to be so, that sort of thing

As long as the ratio of (positive return - negative return):investment is greater than 1, the game is good (although some are probably better than others). A game might be considered mediocre if that ratio is between 0 and 1. A game is bad if the ratio is 0 or less (because the returns are sufficiently negative).

Another thing to consider is a game's "universality", you might say. A game is better if it has "universality" - by which I mean that there is something in that game for any player whose preferences match the game's genre/theme/mechanics/default setting flavour. It is, of course, no strike against a game if there is a preference mismatch.

I suspect most RPGs as designed either have or have the potential to have "universality", but I think including the concept among our criteria does help deal with games such as That Game (that Stonehouse refers to obliquely), which was apparently designed not only to not have "universality", but in fact to be against "universality".

Whether "universality" ought to be considered on its own, or embedded in the "returns" of the game, I could not say. There may be other concepts that are similar to "universality", but my brain hurts too much right now to come up with any.

I think those are a decent conceptual basis for thinking about what makes an RPG good (or not).

Now, quantifying all of this - there's the rub. I'm not sure it's possible.