Apparently I got the wrong impression from this quote:
It sounded like you expected to impose a huge penalty (3d6ish?) on attack rolls. I guess you really just meant that your AC is good enough that 1d6 is a relatively large penalty.
Good point about the Manglers vs. Darkness.
Once Wall of Fire is up, what's your plan for inflicting damage, in the "only 2 short rests" scenario? Can't Eldritch Blast now because you can't see the target. Synaptic Static? You may hit the PCs too (depends on positioning but it's only a 20' x 40' area) and penalize your own concentration saves.