When I was building the character, I was torn between dipping and not dipping.
I did decide to dip for a few reasons though:
1 - It was a small party (me and one other player) so I was truly the only one on the frontline and felt like the increased AC - especially once magic armor/shields came into play - did help significantly more than the extra 2 HP/combat;
2 - Twilight's advantage on Initiative made so I was often going first even with 8 Dex. This is amazing for a Controller;
3 - With maxxed Int, I felt like Cha save prof was better than Int save prof. For rare saves, I thought it better to have both at a decent level rather having one be really good and the other be terrible (it did come up a couple of times);
4 - AoA is a really great upcast spell, especially at lower levels, so you don't miss out as much on having no highest level spell known;
5 - The extra source of Temp HP with Channel Divinity, even if a low amount, meant my Cpncentration was even harder to break in the lower levels.
I could definitely see this character going full Wizard, especially in a larger party. But I don't regret my Cleric dip one bit for him.