The main problem comes when you design your game in such a way that quest markers and the like are actually necessary. Morrowind's quest journal would often include directions to reach where you needed to go, because that was necessary with no quest markers and added some nice flavour to proceedings. (Would have been better if the journal wasn't such an awful piece of crap to use, of course, but that's more a UI thing than a quest marker issue). Whereas the likes of Skyrim doesn't bother with that noise, because why should it when you can just be led by the nose to your next destination? So in that case you have the opposite issue--you *have* to use the quest markers whether you want to or not!
I think the Witcher 3 actually had a nice compromise here--many quests would lead you to the general area you needed to be in via quest marker, but then you'd have to search within a small area for the actual important bit.