Well I don't see why they would need to lose pact augmentation, you may need to stat they can reselect pact augmentation ability once per day while bound to their master but that's all. And if you want to increase the appeal of binder level for those with the template maybe give them a +1 bonus to effective binder level to represent the fact that they studied the formal method of binding, it compensate for the LA.
That give use if we compare a Chosen of the Void 1/Binder 1 and a Binder 2 bith human with the same stat, elit array.
{table=head]Class|Str|Dex|Con|Int|Wis|Cha|HP|BaB|Fort|Ref|Will |Skill point|Abilities|Feat
Binder 2|12|13|14|10|8|15|10(2d8+2)|+1|+4|+1|+2|15|Soul binding (1 vestiges, level 1 or 2, EBL 2), pact augmentation (+1 AC), Suppress sign|Improved binding and one other
Chosen 1/Binder 1|12|13|16|10|8|17|6(1d8+2)|+0|+4|+1|+1|12|Soul binding (1 vestige, up to 4th level, EBL 3)|Favored vestige and 2 other
[/table]
And that without taking into account the additional bonus to an ability score because it vary depending on the vestige.
Its your template and as written its not bad at all but the for the ability to have a 4th level vestige ability at early level the trade off seem more than fair.