How to deal with feeling proud of my own mindfulness?

It's a bit weird to explain. In daily life when I notice some though or emotion or when someone praises me or some other similar action and I notice that and don't cling on to it there is a inner voice praising myself for not getting attached. And if I "let that go" then again an inner voice praising that action and not getting attached to that feeling and this kind of creates a loop. It might sounds very weird or trivial but sometimes its very annoying. How should one deal with this? Just ignore it?