Because it's easier to link appearance with physical attributes, while mental capabilities are set entirely arbitrarily. Warcraft goblins are geniuses, Golarion goblins... not so much. But it's pretty unintuitive if 3'6" tall, 35 lb halfling can be just as strong as 7', 350 lb musclebound hulk of an orc (or whatever).