The abyss is the place where demons are the most numerous and dominant creatures there.

The devils' dominion lies in the nine hells of baator.

Of course, both types of fiends can also live on any other sort of evil plane (typically called the lower planes).

And then there's the third rather obscure type called yugoloths, or daemons, who stalk mostly on hades and gehenna, but also work as treacherous mercenaries for both devils and demons.