Join a brother late to get benefit of fard salaah in congregation

When you come into mosque late for Zuhr, and someone is busy with salah, without you knowing if he is performing Sunnah Zuhr or Fard Zuhr, can you join that person and perform your Fard Zuhr and carry on after he salaam and still get the benefit of congregation (jama'a) salaah?