My intuition would be something with ghosts. The village does not actually exist and is some kind of illusion produced by a supernatural entity..

There has to be a way to get down somewhere or the players can't interact with it. Maybe also hidden by an illusion