How did Martin Luther justify the separation from the Catholic Church?

Even though Luther found great abuses in the Church of his time, there is the clear idea in the Bible that the Church is one, Jesus even prays that his followers be in unity. ("I pray that they will all be one [...] " John 17:21) So how did Luther justify separating from Rome?