Okay, how's this: The goblins got their hands on a very powerful artifact at some point and used it to create a one way wall that is nigh unbreakable. So when they get slaves, they just shove them through and the slaves can't get out. The slaves are then forced to mine rocks or whatever, sending it back via a slide they send the rocks down. If they don't meet their quota, they get no food. Might not be all the slaves, but it's a fun idea for all kinds of options to present the players. I'm sure you can imagine some.