Thats actually a good question... Maybe he's just spent all his time building things around the castle. It was a bad choice to send those kids anyway.
I really didn't like them adding in the scene where:
SpoilerFinn jumps in front of PB to deflect the sonic blast or whatever the hell he was shooting from his sword. I can only assume this is their way of drumming up suspicion that Finn isn't really over her, but a less cliché way would have been better.