This is a good question with an interesting answer. The crushed stones are what is known as ballast. Their purpose is to hold the wooden cross ties in place, which in turn hold the rails in place.
Think about the engineering challenge faced by running miles of narrow ribbons of steel track on top of the ground: They are subject to heat expansion and contraction, ground movement and vibration, precipitation buildup from rough weather, and weed and plant growth from underneath. Now keep in mind that while 99 percent of the time they are just sitting there unburdened, the remaining 1 percent they are subject to moving loads as heavy as 1 million pounds (the weight of a Union Pacific Big Boy locomotive and its tender).
Put all this together, and you have yourself a really, really interesting problem that was first solved nearly 200 years ago and hasn't been improved since!
The answer is to start with the bare ground, and then build up a foundation to raise the track high enough so it won't get flooded. On top of the foundation, you deposit a load of crushed stone with sharp edges (the ballast). On top of the stone, you lay down (perpendicular to the direction of the track) a line of wooden beams on 19.5-inch centers, 8.5 feet long, 9 inches wide and 7 inches thick, weighing about 200 pounds: 3,249 of them per mile. You then continue to dump crushed stone all around the beams, effectively locking them in place.
These beams are made of hardwood (usually oak or hickory), and impregnated with creosote for weather protection. In the U.S., we call them cross ties (or, colloquially, just railroad ties); in the U.K. they are known as sleepers, in Portuguese, dormentes. While 93 percent of ties in the U.S. are still made of wood, heavily trafficked modern rail lines are increasingly trying alternatives, including composite plastic, steel, and concrete.
Next, you bring in hot rolled steel rails, historically 39 feet long in the U.S. (because they were carried to the site in 40 foot gondola cars), but increasingly now 78 feet, and lay them on top of the sleepers end to end. They used to be joined by bolting on an extra piece of steel across the joint, but today are usually continuously welded end-to-end.
It would seem that you could just nail them or bolt them down to the ties, but that doesn't work because of the non-trivial movement caused by heat expansion and contraction along the length of the rail. So instead, the rails are attached to the sleepers by clips or anchors, which hold them down but allow them to move longitudinally as they expand or contract.
So there you have it: a centuries old process that is extremely effective at facilitating the movement of people and material over thousands of miles ... even though nothing is permanently attached to the ground with a fixed connection!
The ballast distributes the load of the ties (which in turn bear the load of the train on the track, held by clips) across the foundation, allows for ground movement, thermal expansion and weight variance, allow rain and snow to drain through the track, and inhibit the growth of weeds and vegetation that would quickly take over the track.
More questions on Trains: