Answer by Dan Holliday:
So the reason is rooted in Gene Roddenberry's half-baked ideas about the future and his obsession with peace. Roddenberry was incredibly scarred by World War II, and he was obsessed with a future of peaceful exploration of the galaxy. While he accepted that there might be "others" outside of the Federation that were not idealists, for him the size of the United Federation of Planets' space was so vast that the Galaxy-class ships would have families for their pathfinder voyages.
When he conceived of Star Trek: The Next Generation back in 1986 (it was actually an offshoot of Star Trek Phase II), Roddenberry came up with a rulebook that is only talked about. First and foremost was that the Galaxy-class starships were rare—five in total—and that the backbone of the UFP's defenses would be its alliance with the Klingons (who would go to warlike berserkers if need be), the thousand or so Excelsior-class starships, and the incredibly crafty and creative Starfleet that could quickly resolve most "war" issues through diplomacy.
And that was the cornerstone of his vision for The Next Generation—that, in the future, the Federation would be so incredibly creative and dedicated to peace that this dedication would result in a society where diplomacy was always the factor that saved the day. Thus, crews dedicated to peace, science, and exploration would always find a diplomatic method of "saving the day."
Not all ships—in Roddenberry's vision—would have families. If you look at the early interior drafts of the Galaxy-class ships, you'll see his vision (which the producers kept in check) was for the Galaxy-class ships to be "city ships" that were the pinnacle of the UFP's "flag" and would always put such a positive foot forward that few would attack them, and when or if it did happen, then the crews of these ships would be so talented, that they could always weasel their way out without much of a fight.
He wanted the ships to be staffed with—essentially—a utopian crew who never fought and who always worked in perfect harmony. The drama would always be provided by external stimuli. The crew would resolve these issues through passion, intelligence, and teamwork.
His "families on ships" vision ended up filtering briefly into the beginning of Star Trek: Deep Space 9, because we saw—on that tiny Reliant-class vessel—that Sisko's wife and kid were onboard and needed to be saved. Roddenberry was part of the creative team of Deep Space 9, and as such, his initiating ideas were allowed to filter into it, but the battle of Wolf 359 was essentially portrayed (from the Sisko perspective) to highlight an end to "families on ships" idea.
Roddenberry's idea sucked. The moment he died, the producers backpedaled as quickly as possible from everything he wanted. And like George Lucas, he could not be trusted to materialize his dream because it was far too meta for most audiences to grasp. A TV show needs to make money, and to make money they need drama.
So, by the The Next Generation movies, Deep Space 9, and Star Trek: Voyager, ships were run more in line with a military organization than as some quasi-paramilitary scientific organization. The reason is simple: Despite what you see in the show, Starfleet is clearly a military organization. Not "quasi," not "semi." It's a full-fledge military organization, with military training, schooling, ranks, weapons, tactics, and design.
And with the addition of the Borg, Cardassians, and Dominion, the nail was driven into coffin of Roddenberry's "utopia" ideas. While Rick Berman and Brannon Braga ended up destroying the universe they saved from Roddenberry, their ideas were more or less sound in terms of getting away from the originator's crazy concepts about where humanity would go.
More questions on Quora:
- Star Trek (movie, TV and creative franchise): What is Star Trek all about?
- Star Trek: The Next Generation: Why didn't Picard destroy the Borg ship in Q Who episode as he knew the Enterprise's phasers could deliver real damage to the Borg ship?
- Star Trek (movie, TV and creative franchise): In Star Trek, is the Prime Directive moral?