I'll leave mechanics to others. The first thing I'll say is that I wouldn't run a campaign like this, at least not without running it by my players first.
Gaining levels and power is one of the most exciting parts of the game. There's something very satisfying about watching your character grow and mature. You start taking away feats and powers and so forth, and your players are going to feel quite cheated.
Besides, there's a Chaos God running loose. You expect a bunch of level 10 mooks to deal with that?
If you don't want them to become exponentially, game-breakingly powerful from here on out, try to work in rewards that don't involve either XP or gold. Those things are the default goal, but if your players are willing to chase a different carrot, then the game can still be rewarding.