Great for a background story but with a massive penalty.
You could try narrowing it down even more by saying that the penalty only applies to weapons that deal other types of damage.
Say a Blunt Weapon Reliance would gain a -1 or -2 on all attack with Piercing and Slashing weapons.
A Club and a Hammer are prety much alike in the way you wield them, but a club and a rapier...totaly differend.

Still a great Flaw.