I'm inclined to read it as a sort of chagrined/self-conscious "am I the only one who has no chemistry with Elliot?" moment. But I'm not terribly confident about that interpretation.