Yes, that is how the rules work. It is not an interpretation, but the intended effect of the spell.
I think you're overreacting. Having a silinced team member going against enemy casters is a good tactic but it does not cause the game to grind "to a screeching halt" nor is it a fail-safe shutdown against casters.