I'm going to assume you mean elite swimmers here, and not swimmers in general. Before I answer, I have to voice my disagreement with the assumption behind the question - that most elite swimmers peak young. Just look at the average age of the Olympic teams over the past few quadrennia:. Swimmers now are going best times through their late 20s and early 30s. Yes, there are some swimmers who truly do peak very young (and they're seldom, if ever, males) but by and large, the most successful swimmers enjoy long careers, far into their 20's, perhaps 30's, and in the very rare case (ahem, Dara Torres) beyond. As the question states, athletic abilities really do peak later than the teenage years.
I can definitely see how one would think that swimmers peak early, given the recent WRs broken by 16-year-olds, but just because you're breaking world records doesn't mean you're at your peak. Yes, you're faster than anyone else in the history of the world (wow!), but that in no way means you've gone and blown your whole God-given sports success allowance all in one go.
But to answer your question, I have a few reasons why athletes may *appear* to peak early - and I'm referring here to the United States, specifically, since that's where my relative expertise is.
- Young athletes *can* rise to the top of the world heap. Swimming is one of those sports that isn't just about physical size and strength. If you have great technique and endurance, things that you can really develop before you're ready to put any muscle on/hit a growth spurt, you can actually compete with the best (except in the truly power-dominated events). So, because of this, young athletes with the right blend of strengths can rise to the top, and in some cases break world records. But to echo what I said above, that doesn't necessarily mean they're peaking.
- Freakishly young swimmers get a disproportionate amount of attention/hype. It's a storyline people love, for some reason. "16-year-old dominates the world." Wow! It's more exciting than "26-year-old dominates the world."
- The training:reward ratio for the sport is absurd. Training consists of spending hours staring at a black line on the bottom of the pool every day. (Or, I guess, if you're a backstroker, the sun.) Consider that most kids start swimming around age 8, and that's a LOT of hours by the time you're in your 20's. Continuing to swim past college (another point, below) requires the financial support to do so. Either you've got a full-time job and a deep, genuine passion for the sport - which is rare - or the incredible opportunity to support yourself with your swimming as a professional. The latter is rare, although not as rare as it used to be even a decade ago. The "average world-class swimmer" doesn't live like the average NBA player. If the incentives were different, I think you'd see a LOT more swimmers sticking it out later in life. As it is, I'm really impressed with the number of post-graduates still swimming, and I have to think it's in part because of the assistance you can get from USA Swimming as a member of the National Team and the rise of very strong post-graduate programs around the country. In any case, after you're done with college, into your peak years as an athlete, it's really, really difficult to justify continuing. You might actually hit your peak as a swimmer in your late 20's, but if you can't continue in the sport due to financial reasons or just lack of resources (like a great post-grad program nearby), then your peak is artificially shifted forward a few years. If we're going back several decades, though, there wasn't a whole lot of support for college swimming, especially women's swimming. Once you got through high school, there just wasn't anywhere to go.
- The NCAA. NCAA swimming is done in short course yards. All the international competitions that matter are done in long course meters. They're basically two different sports. So you spend four years focusing on dominating in short course (with brief 3-month summer interludes of long course competing) and, believe it or not, that turns most people into really great short course swimmers - with limited potential to extend their careers. You don't get sponsorships for winning NCAAs. You don't hear about the most successful college swimmers if you're not a college swimming fan (and if they aren't already on the world stage). You DO hear about the Olympic Gold Medalists and the World Record Breakers. There are certainly swimmers who compete at the highest levels in both college swimming and international competition, but they're actually quite rare. So, to the casual observer, I can see how it would appear that swimmers just kind of disappear when they hit college. There are some truly phenomenal college swimmers who just aren't as dazzling long course, where it "matters." Watch NCAAs and then come back and say that swimmers peak at 16!
- Training environments necessarily change dramatically as you grow up. Here, I'm talking specifically about making the jump from a child prodigy to college and beyond. Some athletes who rise to prominence very young have a great relationship with their coach and club team, and then they have to make a decision when they graduate high school (or before). Go pro and give up swimming in college, or go to college and chance it with a new coach, new team, and totally new environment? You just never know what's going to work and what isn't. There was a slew of young female swimmers in the last decade who went pro in high school, and it went very poorly for them, curtailing what otherwise would have been long and illustrious careers. This might be counter-intuitive, but there's a lot going on behind the scenes, and it's difficult to preserve what you had going on as a high schooler when you're older. The teenage stars who are able to keep performing at the highest levels find a training environment that works. It's a lot harder than it sounds.
- Injuries happen. I talked about this a little bit here: Injuries are certainly a factor in ending some careers prematurely - whether by an actual career-ending injury or by a condition that's worsened by swimming (backs and shoulders, I'm looking at you) that doesn't necessarily *force* swimmers into retirement, but *strongly encourages* it. As mentioned above, it's a LOT of hours of swimming. Some bodies weren't built to handle the day-in, day-out of the sport for decades. Mine sure wasn't. These days, though, lots of the older swimmers have figured out how to train in a sustainable way, so they can keep it up without getting injured. Pretty amazing.
So, I disagree that swimmers generally peak early, but there are some good reasons for the perception that they do. Plenty of factors conspire against keeping swimmers at the elite level throughout and after college. Some swimmers do indeed peak early, but there's usually something else going on behind the scenes.
More questions on swimming: