My way of countering that is two ways.
1. Scale
If a 10 is average, 12 is slightly above etc you have a decent measuring scale to begin with.
2. Find your flaws first
In my CS example, I have an INT of 12 and WIS of 14. However my STR, CON and CHA are all 8. If you look at yourself critically and address where you'd be lacking first, you are at least not a perfect does everything character.
Plus after looking at yourself more realistically you're less likely to inflate everything else beyond reason.
However, just because a few people disagree with a self analysis doesn't make you wrong. No one knows you better than you do. At the same time, some people may be overly confident in themselves and this translates to lack of confidence in others.
Also some people may hold a disliking, for example I know several people who regard me as above average intelligence. But there are a few people who dislike me who if they were making me would probably give me 8's or 6's for everything. Although getting second opinions and thoughts is nice when evaluating yourself, it is not final.
No one knows you better than you, and even if they did somehow they are still human with the possibility of being wrong.