Well...

All characters starting with 6 20s would create some very odd imbalances.

Among other things, monks and barbarians now have a better AC than the heavy armor types, and you'll never convince one to wear clothes.

But I'm not sure it'd really break the game.