But not pitchforks and fire. And swords from the local militia. Not to mention that old guy in the back who has his grandad's Enchantment-breaking Wizard-slaying sword left over from the two Witch Hunts they had a while back...
Even if you do manage to kill them all, the big bad good Warforged Paladin (named RoboCob, for some reason ), smashes you to pieces for killing all the townsfolk.
Anyway, you need to be Level 4 to have that spell (and Level 6 for Fireball), at least. You have to actually live that long to even get to Level 4. The common wizard who is fifteen or so, just learning his magic, wont have that. And, since in my world, just about every scroll ever seen is burned. A Wizard usually has a tough time finding spells to write down, and rarely, does he manage to have a spell he actually wants...
Anyone caught writing magic down and trying to control it (having a Spellbook, which wizards
need) is dead. In my world, a wizard wouldn't make it past level 2 before he's killed. Most of them get killed at 'level 0'.
It's how I can tell my PCs "You can't be wizards..." and feel fully justified.
And, is a good reason the two or three Wizards who are actually considered 'powerful' in my world have absolutely zero actual power. Which is what the OP is looking for.
They stay away from people. And don't. Do. Anything.