Reasons against childhood baptism

There is a belief that baptism (in Christianity) should be done later on life (e.g., adulthood), rather than in infancy. However, I do not see any reasonable (?) reason to NOT perform the baptism in infancy. One of the major drawbacks of credobaptism (the way that I understand things) is that the person is deprived of the Holy Spirit two-fold: though the lack of baptism itself, and through the ban for communion (baptism being a major prerequisite for the communion). So, shortly: what is inherently bad about baptizing infants?