Scrolls taking the same action to cast as the spell they mimic is standard as of Rules Compendium.
That's how i've always played it and i've never had any trouble. Sure, they can be useful, but that's intended, not broken.
As others have mentioned the people who usually benefit the most from scrolls and wands are UMD users, not full casters. So if anything it improves balance instead of making it worse.
That's not a rule in the first place. Only staffs let you use your CL.