That's cool, Shadow. I wouldn't rule that way, but I'd play it if it was offered.

I come from a primarily LARP background, where trying to cast a spell with while holding a weapon/shield is disallowed. You have to point at your designated target with an empty hand as part of the casting rules. Or, if a material component, like a spellball is required, it can be in your casting hand. But then, it's harder to gish up a LARP - they tend to frown on melee competent casters as being 'op'. Short sighted in my view, but people put a lot of emphasis on balance in pvp games.

At any rate, between a 3.x mindset and a LARP background, I find somatic components to be a particularly sticky wicket. YMMV, of course.

I'm not sure anyone's pointed this out, but Clerics can't cast spells in heavy armor without heavy armor proficiency either... that's a new trend.