I don't really think LA needs to be justified as slow learning. Unless you also want to have a lore explanation for leveling, which would simply lead to something like "Once you have 112 kobolds, you will have enough experience in the arcane arts to learn and cast spells of the third tier, such as the mighty ball of fire or bolt of lighting" as a fluff explanation.

It's not that the character learns slower, perhaps its just that the character joined the others earlier in his career. Afterall, what are the odds of 4 complete strangers to suddenly meet up, decide to adventure together and discover that not only do they all have a comparable degree of past experience, all of them seem to improve at the same "rate". Its not even the same rate though, where the barbarian learns to whack things just a bit harder, the wizard learns to change the shape of any willing creature, animate the dead and blink through vast distances with only a word.

I'd say you should just justify it as PCs with LA being just a bit fresher at the whole adventuring thing then their normal brethren or they may have relied on their racial gifts rather then on any learned skills. Who knows, maybe its a a kind of cosmic justice, a Half Dragon Fighter 1 may hit just as accurately and just as hard as a Fighter 4, but attacks seem to find their mark more often against him more often then against the regular fighter. Asuming HP is an abstraction, the blow that surpasses AC and deals 20 damage may be a near fatal blow that leaves the half dragon bleeding, but for the fighter it may leave him intact, not even connecting, but still represent a 20 point deduction from his HP value.