Should a Buddhist go to the gym?

I have to be honest with you, this question was already answered by an authority on Buddhism and long-time practitioner, which is the spiritual leader of the Sangha which I meditate with. And yet, I feel the need to listen to other opinions. The question is the one in the title. His main argument was that going to the gym is not beneficial in any way and it actually worsens us by feeding our pride, thus making the schism between ourselves and vain glory more difficult to realize. What do you think?