Why do adherents to the priesthood of all believers still have organized congregations?

I might be making some naive assumptions here, but I have a hard time seeing what the purpose of congregations are in the context of priesthood of all believers. As I understand it, priesthood of all believers basically says that all people have direct and equal access to God, as well as equal privilege and authority to minister. What then is the purpose of congregating? Why are those congregations well organized?