Its not just because she’s smart. The reason he says she’s annoying is because she insulted him after he only asked to hold her hand and then she tells him that he is of less importance and not as superior as her because she is smart. If she didn’t rub it in his face that she was smart and act as if she’s better than him then he wouldn’t care.
If you read the comic long enough, they say that they all look the same eventually…