How do we know numbers of past lives and rebirths?

I can understand how the Buddha and various arhats and the like can surmise the existence of past lives and rebirth. What I dont see is how they come up with actual numbers of times they have been reborn and the total number of times anyone is and how long the universe has been contracting and expanding and things like that? At some point isn't that delving into the unknowable?