Heh, I like how you put that. However, I was thinking that it wouldn't be too hard to justify mass warlocks. Just make it be a cultural/racial quirk unique to this island that you discover as you attempt to train the commoners. I think the archivist might stumble onto it or something.
Could be fey? Fey warlocks are always good for laughs.