XP =/= old age wisdom. XP is a measure of the things you've had to learn to survive and grow powerful. A child who gets thrust into a jungle does not need to understand what he's doing in order to grow strong and ruthless. That doesn't necessarily translate into a greater insight on life and existence.
Only when you had time to mentally process all the things you've experienced (which, btw, have nothing to do with adventuring, as commoners become wise elders too, just not as worldly as adventurers), that you gain insight into the world and are able to dispense it as an elder.