Yeah, this checks out. Places you last visited 50 years ago tend to have issues recognizing you.
Even liches or dragons or other long-lived entities would plausibly not recognize you; they probably never cared enough about your existence to remember you day to day, much less after decades. I could buy this.