Probably because he doesn't have the troops to fight off the invasion. Especially when 1200 men at arms that he could have used were taken by the nobles fleeing, plus their ninja. Maybe 30 paladins on the battlements would have helped, it might have prevented the walls being torn up so much. But maybe not. Not much help they could have been if the army charged like they did.