Attending the taping of a late-night TV show is not a comfortable experience. First, you’ll stand in line on a city sidewalk for hours—chilled by wind, or baked by sun—being herded like a fatted calf to an abattoir. Upon arriving at the head of this line, you’re often tagged with a wristband and instructed to return in an hour so you can stand in a different line. When you finally enter the building, you’ll be squeezed into a musty back hallway or a barren anteroom. Here, you’ll cool your heels for a second eternity, pondering whether the free tickets these shows all hand out are somehow far too pricey.

Once you’re seated inside the studio (and warned never to look at your phone lest you be summarily ejected) you’ll often sit on your hands for another soul-sapping half-hour—watching production assistants and camera operators mill about on an otherwise empty stage. At some point, a warm-up comic will bound from the wings, accompanied by a brief fanfare of rock music.

This man will wear jeans and touch his hair a lot. He will probably tell you his Twitter handle. You’ll be so ecstatic that something—anything at all—is occurring that you’ll forgive him for being a chump. He will tell four jokes before commencing the warm-up process, which will mostly entail him commanding you to applaud for no reason, again and again, as he complains that your applause should be far more robust. You’ll then be forced to laugh aloud on cue—looking around in discomfort at the rictuses of your neighbors.

I’d been to a few of these things before. There was an early ’90s Leno taping in Los Angeles—they took audience suggestions on index cards pre-show in those days and, to my adolescent pride, Jay used a Dan Quayle joke that I’d submitted—followed by more recent visits to Fallon and Stewart. But as the final days of Letterman approached, I decided to see as many tapings as I could fit into a few weeks. I thought maybe that by observing these shows up close and in person, I might reveal the seams of their construction. What is the atmosphere like at each? Do we miss anything important when we watch at home? So I embarked on a late-night safari.

I headed to Late Night With Seth Meyers, where the guy who warmed up the crowd seemed visibly nervous, as if he would mess up the show by being lame. The five-piece band played desultory pop covers during breaks but, because regular bandleader Fred Armisen was absent, they weren’t allowed to interact conversationally with Meyers. This made them seem like a backup wedding combo that is tolerated but not beloved. Meanwhile, the most jazzed the studio audience ever got was when a young woman sitting in front of me spotted Lutz from 30 Rock in the wings. She yelped with glee, and Lutz happily gave us all a big wave.

At The Nightly Show With Larry Wilmore, I was seated next to some pleasant Minnesotans who had been waiting outside for a full three hours before the show even began. For their troubles, they got a warm-up comic making some feeble jokes about getting high. Poor Wilmore is still learning the ropes, so you can hardly blame him for botching some segments. But still, the flubs meant that each segment—his monologue, his interviews, and his end-of-show wrap-up—had to be immediately repeated. “Pretend you’ve never heard the jokes before and laugh louder,” a stagehand suggested. What stands out most when you watch the show live is its lack of fluidity. It’s just waiting … re-taping … waiting again.

John Oliver didn’t so much as glance at us, after his pre-show greeting. He sat at his desk and stared straight into a camera lens while reading from a teleprompter, delivering gag after gag as I disappointedly realized that we, the sad sacks in his tiny, tiered amphitheater, were serving as nothing more than a fleshy laugh track. Oliver was squarely focused on his real audience, by which I mean the TV/internet audience out there, instead of on the weirdo shmoes who’d been ushered in to fill up the seats.

None of these shows had much in the way of set design. Just a few haphazard tchotchkes and some wall decor that doesn’t show up in the camera frame anyway. Wilmore and Oliver play in small, black boxes inside buildings on desolate Midtown side streets. Meyers at least has exposed brick walls and some Rockefeller Plaza provenance: He boasted during his pre-show chat that Studio 8-G was previously home to luminaries like Phil Donahue and Rosie O’Donnell.

All three of these dudes did these brief audience meet-and-greets before the tapings began. They stood near the first row of seats and nervously parried a smattering of inquiries, acting deeply guarded, seemingly eager to dispatch with this chore and move on to the part of the show that actually goes on the air. Oliver’s production assistants repeatedly warned us not to ask anything “personal.”

I’d begun to wonder what we were doing there. I suppose a live audience helps the host know which kinds of jokes are generally working. But is that really helpful? Did they need the ambient noise of coughing and throat-clearing to make the set feel more alive? Wouldn’t a regular old laugh track and an occasional recorded, piped-in sneezing sound do the trick just as well, or better? It was hard to see why these shows went to the trouble of dealing with seats, or tickets, or us.

And then I saw Letterman. It felt almost regal to enter the Ed Sullivan Theater beneath a lit marquee on 53^rd and Broadway instead of through some unmarked, dented metal door on an industrial block of Hell’s Kitchen. The theater for the Late Show With David Letterman seats 450 people instead of 100, and its rococo balcony offers a glorious vista over the sprawling stage where the Beatles first performed in America. Instead of bathing us in washed-out, piped-in music, or a half-hearted pop quintet, Paul Shaffer and the CBS orchestra rocked us with a wall of sound, including a horn section that blasted riffs clear to the theater rafters.

It’s no surprise, of course, that the studio audience experience is radically different at the Late Show than it is everywhere else. An upstart talk show on Comedy Central is a very different breed than a late-night legacy brand on a major network. But still, it was hard not to sit in that Late Show studio audience and feel—for the first time—indispensable. Letterman puts on a show. He presents fully-rounded entertainment, and he feeds off the energy in the room. This is a big, brassy venue with a live orchestra, instead of a cramped black-box studio with somebody’s iPod plugged into ceiling speakers. And Letterman needed us there.

Compared to broad-shouldered, commanding Letterman, Oliver and Meyers seemed like slight, mincing fellows in overtailored suits. Dave appeared to delight in chatting with the audience, basking in our attention. He asked us where we were from, and darned if he couldn’t make a connection with everyone who spoke. He asked a Southern Californian about the status of the San Onofre nuclear plant, and quizzed a Chicagoan about the then-upcoming mayoral runoff election, all the while demonstrating an electric intellect and a well-informed, wide-ranging curiosity. He handed $50 cash to a woman sporting a vintage Letterman T-shirt he hadn’t seen in a while. He won us all over. He made us all love him, even though we loved him already.

Once the show began, stagehand Biff Henderson launched into a bit—just as he’s done for the last 30-odd years—racing out the theater doors, exploding onto the proscenium in that giddy, Letterman-y manner. Biff retrieved milkshakes from a nearby fast-food restaurant. Both Dave and Paul actually sipped their milkshakes even when the cameras weren’t rolling because, apparently, they just enjoy milkshakes.

I realized with a start that I’d been watching bandleader Shaffer, drummer Anton Fig, and bassist Will Lee play in and out of commercial blocks since I was in elementary school. They were still bringing it—and still betraying an unexplainable predilection for 25-year-old Living Colour songs. Three decades of spending their weekend evenings together. I found myself wondering what on earth they’d do when it was over.

Grandiose as it sounds, watching Letterman pace the stage, charisma still radiating, I couldn’t help thinking that this guy represents the last vestiges of the monoculture. The fortress of macro-entertainment has crumbled. The new late-night shows have no prayer of reaching all of America, all at once. They can’t rely on a docile audience that will patiently sit through the second celebrity guest and into the loopy, end-of-hour conversation with Fran Lebowitz, or the time-filling, willfully bizarre skit with Chris Elliot.

You can see it in the way the other hosts plod wearily through their audience interactions, passing time until the cameras roll again: They barely knew we were there. These shows are designed to chase likes and shares, to be easily chopped up into discrete grabs for elusive virality. There’s no need to put on a really big show in a really big theater when your end goal is a 30-second clip that will play in a tiny frame on someone’s Facebook feed. The studio audience is a vestige, too. But at Letterman, at least for one more week, a live taping still feels magical.