When the restrictions will make it hard to effectively play the game using that system.

Or

When the table agrees they're too stifling. Assuming the table came together before the decisions were being made. (E.g. a group of friends gathering to play as opposed to players deciding to join an open table game with predefined rules.)