The important thing to remember is that infinity is not a number; it is a concept.
Infinite means "without end". "in"= without or lacking. "fine"= end, finish.
Technically a thing is not truly infinite if it has a beginning, for then it has an "end", its beginning. However, there is often talk about something which begins and then lasts until infinity. Then again, there is nothing in science that supports such an idea.
Quantum loop gravity theory does espouse a particular facet of infinity, that matter that constitutes the universe has always existed and will always exist. This same theory goes beyond before the Big Bang and states that the matter which is our universe now existed before the Big Bang as a universe that was undergoing Universal Contraction, as opposed to Universal Expansion which we experience now.
Another definition or understanding of infinite is that something as simple a baseball cover can be considered infinite; specifically without end. If you were an ant crawling on the surface of a baseball, you would never encounter the end of the surface. Eventually it might dawn on you that you've been some place on the ball as before; but it certainly would be difficult to understand how it could be that you could end up where you started without turning around.
As stated at the beginning of this answer, infinity is not a number; therefore one cannot calculate with it. I say this even though "infinity" is mentioned several times in relation to mathematics. However, it is never properly utilized as a number; for it does not follow the laws that all numbers (counting, whole, real, positive, negative, irrational, and even imaginary numbers) do follow.
And since you are asking this in Astronomy & Space, a common question is whether our universe is infinite. The answer is: even though the universe is very large and even though one cannot reach its end (making it technically infinite), our universe is finite.
On the other hand, the super-universe in which our universe exists is literally infinite.