On the 'fluff' side of things, i still think the original poster can have the "know's every spell" and "lazy" thing work.

Consider the learning of spells as a collection, rather than the acquisition of power/knowledge.
Theres no actual purpose to it. It's just something he does becuase he wants to finish his collection. Only the heroes run into him long after the collection is complete and he's been kicking back doing nothing for a while.
Maybe every now and then a new spell is researched and he finds the motivation to re-complete the spell list.

Given this, you've now got a powerful wizard who didn't become so to become so (if that makes sense). He's got some spells that make life easy and pleasurable so he just does that, keeping an eye on security at the same time.

As for weaknesses, he's a collector for the sake of collecting, so perhaps he has another collection (a rare item that the heroes might use to gain leverage). This also gives the heroes a reason to be interested - he's got some quests for them.
Alternatively (or in addition) he might show some mildly OCD behaviour. Patterns in his life that can be predicted and exploited. I've seen this work with someone who always ate in the same restaurant, at the same table, on a certain day - but you could play to hedonism and make it so that a large number of his spell slots are always taken by certain spells, making him much less formidable if he is unaware of a threat.