Yeah, don't worry about charisma unless you really want to roleplay a charming guy who is inherently skilled in social situations. But you could roleplay it anyway with 13 charisma, and the character would be charming anyway thanks to his spells.