Going off the rails

[Calthropstu]

For those of you who have been paying attention, you'll know I have been running the Jade Regent campaign for a group. Last week, my players my players got frustrated because they had hit a wall comprised of 2 spirit nagas and an ogre mage riding a gorgon.
They decided they needed help to take them on so the wizard decides to teleport them to a distant city to do some recruiting.
Now I have created a semi-sandbox city for them, and it's getting out of control. The party is attempting to do far more than recruit followers and cohorts. They're jumping into things supposed to be covered in the next book of the ap in an area not covered by the ap, they haven't recovered a crucial item they lost in the dungeon and things are quickly spiraling off the rails.

I am tempted at this point to simply sandbox the rest of the adventure though since the ap is pretty much all grind grind grind.
Anyone have experience with ap's going off track? How did you handle it?
I have never run an ap before, I've always sandboxed adventures or drawn my own dungeons.

[timeeater14]

I don't know anything about Jade Regent, but one of the things that i like to do when players go off the rails is let them. And let them waste their time as the real threat gets closer to their real goals. If there's an artifact that they need to find, have a rival adventuring party steal it or have the villains get a hold of it. Above all, remember that the bad-guys aren't static. If they have a plot that destroys the world in 3 days, and the players go and muck about for 3 days, then the world should end. Game over.

[Talverin]

Not a bad idea, actually; that middle bit, I mean.

What if you had a rival group steal the item they were supposed to be after? Someone else puzzled it out and got the artifact back?

Suddenly the big bad confrontation the party has been winding up for has been stolen out from under them, and they no longer look like the big conquering world-savers. That just might get them angry enough to go haring off after the guys who stole the artifact.

It would help if, when they found them, the guys who stole it are themselves a fairly despicable and not-morally-defensible lot.

[John Longarrow]

For myself I don't run APs because too few really make sense with the folks I DM. APs are too.... limiting for what we like and never feel well done. They've always liked what I put out because it feels so much more connected to what they are interested in.

APs, to me, are a great place to steal ideas and some props. Running one? Naa... I'd get bored with that.

[Fizban]

When running a premade adventure, the players should know this, and have a general idea of what they're supposed to be doing so they can keep themselves on rails. If they agreed to play a module, they should accept it when you tell them they've gone far enough out of bounds that you can't keep up without re-writing the module, and get themselves back on track.

When running Red Hand of Doom I didn't have much in the way of problems but I was pretty stark about it. The module is Red Hand of Doom, you're supposed to stop the Red Hand of Doom, hey look you've found out how to stop them go do the stuff. I'm not too proud of how bluntly I told them the lich was out of their league, but it did keep them from forcing a fight that could have TPK'd them, and the worst thing that happened was when I went off the rails and had a former PC steal a bunch of loot and then come back to mess with them, the conclusion of which I should have expected (players running in despite being split up and low on resources, because of course they'd have a grudge). The common "sandbox" draws in RHoD are wanting to recruit/train/fortify the town ahead of the final battle, but none of these are actually supported and are generally unnecessary- I think I told them the town had already done everything that they could have helped with (having their own high level fighter, wizard, cleric) and they were fine with that explanation.

In World's Largest Dungeon I had to make another pretty blunt admission: one of the players had become obsessed with the idea of recruiting an orc that I'd said was alive but mindbroken and useless (just a bit of flavor rather than having the whole group dead). I made the mistake of commenting that there are actually rules for rehabilitating people from sanity loss, but I had no intention of letting them recruit "free" NPCs and had to say so directly, even if I'd managed to justify the orc's removal via random encounter roll in fairly short order.

In the player's seat, with the DM running Shadowdale: The Scouring of the Land, I eventually found out we'd jumped the rails quite a bit. I was adamant that we not go to the enemy's "fortress," because there was no way we could fight an army. It turned out the "fortress" was more of a guard post meant to be cleared out in a couple days as the first part of the adventure. Instead I'd dragged the party straight to the lich's sanctum (part 2-ish, meant to be after at least one level up), where we acquitted ourselves well enough thanks to the two more heavily optimized+homebrew''d characters. The DM then decided to go off the rails themselves, aiming for a larger and more appropriate scope given the original premise which was pretty nice, but unfortunately relying too heavily on classed humanoids that make for poor encounters against the optimized half of the team.
Back to advice for your game, well you're already experienced at sandboxing and the line "grind grind grind" implies you don't like where the module is heading either. So if you're fine with the altered/extra? workload then it sounds like that'll probably be more fun for everyone if you weren't committed to the idea of finishing the adventure path "properly."