Finding it difficult to cultivate mudita(empathetic joy)

I am finding it very difficult to cultivate mudita for people who are happier and have it all. I am finding it easy to have empathy for those who are suffering. But my ego feels threatened when I encounter happy people who are not even practicing dhamma or meditation. How do I train my mind? How to look at such situation when people are in better situation than me?