Quote Originally Posted by MeeposFire View Post
To go along with this while you certainly could say your familiar is the root of your pact that really makes a new question that you or your DM need to answer which is "why is your patron willing to adventure with you and further take orders from you" (the character may not know the reason why or even thought to ask but somebody really should figure out the motivation for this which thankfully can become a potentially fun plot point to bring up eventually).
Maybe the character is actually the familiar? It could almost be like having an intelligent sword that at certain times exerts its will over the character. With the right DM and player it could be awesome, but if not handled well will be frustrating and unsatisfying.