Ever went on a picnic and got attacked by fire ants? I refuse to believe ankhegs were inspired by anything else than that experience. They're giant, insectoid monsters that burrow up, bite, and then spit acid into your wounds. Lovely.
Three RHD, but they're magical beast type, so that's something. Ankhegs also have a huge strength bonus, large size, a very nice natural weapon and a sizable constitution bonus, as well as an enormous natural armor bonus. Oh, and they also have a burrow speed (that can be used to dig tunnels) and tremorsense.
On the other hand, they lack those (see what I did there?) so wielding weapons is going to be a problem for them and intellectually they're on par with toads.
To determine a good LA, let's do some comparing. On one side, we have a half-ogre with three levels of a full BAB class wielding a greatsword, on the other an ankheg. The half-ogre is dealing 3d6+16 points of damage at +6 to hit, the ankheg is dealing 2d6+16+1d4 at +9 to hit. The half-ogre hits slightly harder and crits somewhat more often. The ankheg will probably have higher AC and can burrow, the half-ogre has his class features. IMO, this is about even, so the ankheg gets +2 LA (man, I give a lot of monsters +2 LA: I should switch things up).