Why do we remember the past and not the future? Why do we perceive time as moving forward?
These are more serious questions than you might think. Most laws of physics don't really care if time moves forward or backwards, yet we see that time keeps slipping, slipping, slipping into the future. Somewhere, deep in our understanding of physics, we've missed something.
This stuff ain't easy to understand, either. Sean Carroll at Cosmic Variance has posted an Arrow Of Time FAQ, and it has some good, solid writing about this. Some of it will make your head hurt, but it's a good thing to be aware of. There are still some baffling questions we don't quite understand, and that's where a lot of the fun is!