We are not talking about Houdini. We are talking about my nimble waif making a physique check they should fail but you want to force them to pass.
The lumbering giant walking across a narrow ledge but falls off and has to catch themselves.
The nimble waif that is knocked down by a rampaging bull.
Our character's ability to fail at what they are not good at is as important as their ability to succeed at what they are good at. In order to play these characters the player would have to penalize the thematically inappropriate checks to fix the mechanical disparity between character concept and merged Physique attribute. However doing so is reinventing Str/Dex, so the real question is Physique XOR these characters.