It's easy. Stop looking at what your 20 level build will look like. You're probably not getting there anyway.
Let's say your campaign only goes until 6th level. You won't cast a single sorcerer spell, but the sorcerer will have been tossing fireballs for a level already.