Hm. Useful answers. I'm the DM in this situation, incidentally. We stopped in the middle of combat last game. Faced with an ambiguous rules situation, I had the will o' wisp withdraw, since it wasn't imperative that the thing fight it out.

Hmmm. Grapple. Assuming you could hit it - doesn't that require hitting its touch AC?

Though really for this exact case, that's not a problem. The PCs are around 10th and they're busy fighting the big boss's main minions. The will o' wisp is just a local nuisance/hanger-on. It has no loyalty to the boss. Having made at least one attack, it could retreat and have credibility later that it "fought bravely" if the big boss survives. It would rather the boss was gone. Too bad the PCs will probably try to kill it anyway afterwards. It's willing to bargain.