Should we let go of the desire to help?

I don't think anyone that answered my questions about dharma or provided textual teaching about the dharma had any attachment to my wellbeing at all, and merely just "dropped pearls among the swine", as a christian would say. It is said that the Buddha did not want to teach Buddhism at all. I wonder if this is the way to go. I wonder if I should let go of my attachment to anyone's wellbeing. For example, my downstairs single mother neighbor drunk yells at her 9 year old child a lot, with severely slurred speech. I am attached to the emotional wellbeing of the child, and it makes me suffer to whitness the yelling. Should I let go of this attachment?