They do want to take over. But they have other more important things that require their personal attention, so they send some of their vassals to take care of it.
Actually, feudalism might be a good model for this situation. The people at the top have a claim to a large territory, but they can't control all of it directly themselves. So they basically lease it out to less powerful individuals who are given permission to use the fiefs given to them for their own use, but with the obligation to serve their liege when he calls upon them.