Main issue with non-combat encounters having any level of danger is that the system is set up to bypass the ac vs damage vs hp balance that combat uses to make sure PCs survive & succeed. It makes a "dangerous" non-combat scene like a regular combat scene where everyone has ac 15 and its all one-hit-KO attacks.

Combat doesn't work because of attacks being limited to a +4 to +15 range, it works because you have goid stat+prof attack vs 15-25 ac mitigated by level scaling damage vs hp and the players have hardcoded rule tools to manage their ac & hp. Non-combat you only have ability vs dc (same as attack vs ac but with lower PC bonuses and no extra attacks) but 100% effects instead of damage vs hp. Also unlike combat characters can't default to their best stat +prof 99% of the time, people will get stuck with non-prof int/wis/cha checks vs the usual 15-20 dcs.

So to have good non-combat dangerous encounters that gice xp you need to either make them more like combat or invent a new game structure/subsystem. Or you can just narrate & make it all up as you go, depending 100% on your GMing skills to make it work.