Long story short: if you get into the squares from which two attackers would flank in one round, you flank on your own.
EDIT: I think it's a great mechanic, I like it very much.
Yakk, I like your ideas, very original! I think I can make the second idea work with some tinkering.