You posted your answer while I was writing an answer to serp and to clarify my position. Hence the crossed wires.
As for your question - as I said, there should be unisex bathrooms in official buildings (courtrooms, police departments, etc.). In private places, the policy should be whatever the owner prefers - whether this be sorting by plumbing, by clothing or by saying no T's allowed. And the right to discriminate against a group applies to all groups, not specifically T's.