Take as many negative energy spells as you can when preparing. When he turns up hit him as as many as you can a round. Use targeted dispel magic or dimensional anchor to stop him teleporting. When you've weakened him with the negative levels send in the BSF. Let him whack him. If he shrugs of the negative levels, did you remember to use enough enervations?

Alternatively, avoid him for a few levels. When you can hit him with a metamagiked enervation then go after him, and use every spell you have to debuff him and remove his contingency. While this happens have the fighter trip him and the rogue deal as much damage to him as possible with his sneak attack.

Final tip: teach the bard to be a supporting character if he isn't already. Tell him as many bardic music tricks as you know.