Is it wrong to tell maids to not chat when in my presence?

While I was working in the kitchen our two maids were loudly talking with each other and one was teasing the other, so I told them that when I'm around, doing something, I don't want to hear them talk. Is this unfair? There's a negative vibe to what I said, but given her job, I don't think it's too unfair to ask. But what's the Islamic perspective on what I did and the whole situation?