Are Catholic Bishops the successors of the Apostles alone or the Elders too?

In researching my answer for [this question](https://christianity.stackexchange.com/questions/35389/does-catholicism-confound-the-role-of-elder-and-priest) I started to think about the doctrine that the Bishops are the successors of the Apostles as being a bit unfair to the other folks. For instance, if St. James who wrote the epistle wasn't the brother of John or Jude, but actually a third James or a fourth James who was the Bishop of Jerusalem, would his Episcopacy necessarily have been conferred on him by the Apostles or just by virtue of who he was. Or were the elders the first successors of the 12 apostles regardless of their functions as presbyters?