I found Feynman's take on it to be persuasive; see if you agree. Essentially, it argues that physics is more than than the mathematics it depends on and extrapolation beyond what is mathematically proven appears to have predictive power. Why this is true is a fascinating question.