It does a few things. For one, a companion at max affection (10k/10k) supposedly has 15% efficiency (reduced time) and +5% crit chance on crafting. Secondly, affection unlocks companion conversations. These conversations give you backstory and the occasional side mission, plus experience! I've actually leveled once in a while based on completing several conversations in a row.
Another benefit to unlocking conversations is for your Legacy. Each type of companion has a specific bonus that you can unlock once per Legacy. For example, if you finish _all_ conversations with at least _one_ healer companion on _one_ of your characters in that legacy, you gain a bonus to all healing received on all characters in the legacy. I.e., finished talking to Doc? Then all of your characters get bonus healing! Plus, for _every_ companion whose conversations you finish, you get +10 Presence to each character in the legacy. As an example, I've unlocked 4/5 companions on my Consular, and 3/5 on my Juggernaut. So on my new Sniper I have +70 Presence, making leveling an alt that much easier! (Presence increases companion effectiveness in combat.)
I think there's another bonus or two that I've missed, but that's the bulk of it. This article is a pretty good look at gifts and increasing affection on your companions.