It's easy to forget that every experience you've had, someone's having that experience now. We put life in a chain, but it's a fabric. What you started off seeing and ended up seeing—another person has gone through the reverse, and feels just as sure of the new view.
I don't believe that now is all that exists. I don't see the past and future as illusions any more than the present is an illusion.
But what I'm getting at is that your past is someone's present. Your future is many's past.
I get the idea. The past only matters because the present was once it. But still...
Your memories are hooks on aspects of the present you do not see, but know are real.
To say now is the only time is to forget all the links.
