There are ways to choose your sacrifices so as to limit their impact on your population and its growth.

If you sacrifice the old and infirm who have already reproduced successfully and are likely to die shortly anyway this may actually allow for increased population growth through distribution of resources to the young. You can also do this via infant sacrifice of those who have minimal long-term viability due to obvious medical conditions (this is an alternative to the commonplace pre-industrial practice of infant abandonment).

If you sacrifice men rather than women, the impact on your population's growth potential is minimized, since the ability of women to bear children is the limiting factor. However, this will almost certainly lead to polygyny (though any culture that embraces human sacrifice probably already has this anyway).

If you can identify infertile people (via magic or some other means), you should sacrifice them preferentially over all others.

Quote Originally Posted by Catullus64
Or just... volunteers, no quote marks needed. It may seem implausible to a modern, but if it is genuinely believed (and, given that you've mentioned magic, possibly true) that these sacrifices bring divine favor and prosperity to the city, then I don't think it would be too difficult to find volunteers, especially in a time of crisis. If these sacrifices are important to the wellbeing of the community, than that community's value system will encourage volunteering. Lavish living leading up to the event, a special stipend (or even social mobility) for your family, a fancy parade, a deluxe tomb at the public expense; I can see all of these motivating a person to step forward.
Grim as it may be to discuss, it's also highly plausible for people in a society were sacrifice is an accepted practice to commit 'suicide by god.' Suicide is actually a distressingly common cause of death (there are a number of countries in the modern world with 2% or more of all deaths attributable to suicide) and while historical data is somewhat hard to come by, it may have been even more common in pre-industrial societies. If even 50% of the suicidal choose sacrifice as their method, that could easily be 1% of the total population.