Thanks for the welcome, Steve. Good to meet you. I'll gladly respond to your questions in numerical order.
1) You ask: "Do you distinguish real differences between a fiction approach to story and a nonfiction approach?" Absolutely I do. The thing about David Simon is, he combines the best of both. I worked with him 25 years ago on our college newspaper, The Diamondback (University of Maryland). Even as an undergraduate, he had a full-blown writer's voice, just an extraordinary gift for language. But what he also had was the impulse to report, to investigate the workings of the world, which not all gifted writers have. (Those who have it can't necessarily do it well, just as many talented reporters aren't great writers, Bob Woodward being the classic example of the latter.)
So, in writing his nonfiction books, Homicide and The Corner, Simon combined a skillful reporter's urge to penetrate hidden worlds—be it the culture of police detectives or heroin addicts—with a novelist's ear for language and flair for spinning a tale. That killer combination applies as well to his fiction in The Wire. Everything is grounded in the raw materials of real-world reporting.
If his partner Ed Burns hadn't spent seven years teaching in the Baltimore schools, Simon wouldn't have tried to tell the story he's telling in Season 4. He'd lack the raw materials. How else would one know that middle-school kids are having sex in school lavatories? The girl-on-girl face-slashing is also based on something Ed Burns witnessed as a teacher, though in real life it took place in a lunchroom, not a classroom. (And it was Ed who ended up dropping Slasher Girl with a punch to the head.)
I kind of wish Simon would envision a sixth season of The Wire, dealing with the new wave of Hispanic immigrants. I don't know how this might plug into his theme of the failure of institutions, but I do believe it's the new chapter in the story of American cities. Over the last 20 years, places that have never had to deal with Hispanic immigrants are absorbing tens of thousands of Mexicans and Central Americans. Cities like Atlanta, Wichita, Indianapolis, Raleigh ... and, to a lesser degree, Baltimore, where the area known as "Spanishtown" didn't exist 20 years ago. (It had been the Polish immigrant enclave.)
But even if Simon wanted to tell a Hispanic immigration story, he wouldn't feel entitled to do so unless he reported it. That means tons of time spent hanging around real people to acquire those raw materials ... those voices. Difficult to do if you can't speak Spanish. The point is, this "nonfiction approach" frees up the artist within and explains why The Wire has the impact that it does.
In my case, I'm less of a born reporter, so I'm more inclined to just make up stuff.
2) Regarding the differences between writing for commericial television and for HBO, too much is made of the freedom to use profanity. After all, there are words you can print in the Village Voice that you can't print in the Washington Post. That doesn't mean the storytelling is any better in the Voice.
The big difference, as you suspect, is the absence of commercials. A decade ago, on shows like NYPD Blue and ER, you divided your story into five pieces: a teaser (before the opening titles) and four acts. Today, the broadcast networks generally demand a teaser and five acts, because the commercial breaks between acts were getting so painfully long. (I would guess that, over the last 15 years, the average "story length" of a given drama episode has shrunk from 47 minutes to 43.)
So, now you're chopping your story into six pieces. Which means you can't go more than seven, eight minutes without slamming on the breaks. So, you try to end each act in a way that'll keep viewers watching, and then, at the beginning of the next act, you're working to build up another head of steam ... only to slam on the breaks again.
The ability to tell a tale from start to finish without interruption allows for much denser, much more nuanced writing. The viewer is presumed to be paying closer attention. Multiple plays during the week are another benefit of HBO and Showtime. I happily check out episodes of The Sopranos, Deadwood and The Wire twice, confident of catching things I'd missed the first time. Broadcast TV will never be a home for shows like these, just as Top 40 radio was never the place for Coltrane.
3) You wrote: "David Mamet once said he doesn't deal with 'character arcs' because people don't really change."
I've always wanted to give that bozo a lesson or two in storytelling. Just kidding! Hey, I got jokes ... .
In series television especially, a "character arc" doesn't mean that the person changes. The character is the character; his circumstances change, and he must adjust. The Wire is a buffet of great character parts and great character performances. My favorite among favorites is Maj. "Bunny" Colvin in Season 3. In part because Robert Wisdom is a world-class actor, but mainly because the "Hamsterdam" story, to me, was about the folly of a man who dares to say, "I'm going to make the Nile flow this way, instead of that way." He messed with the forces of nature, and he reaped the whirlwind. I love the tragedy of that.
I feel lucky to have written for Season 4, where the focus isn't so much on servicing the continuing characters, but on those kids. Simon brought his writers together for three weeks in the summer of 2005 to beat out stories for the entire season. (Multicolored index cards on corkboards, the whole bit.) From the start, Ed Burns had the essence of those adolescent characters, based on kids he had taught.
Once the rest of us got a handle on Namond, Michael, and Randy—one who talks the talk but can't walk the walk; one who possesses true strength and leadership abilities, even if he doesn't assert them; one who isn't cut out for the street game at all—it was a thrill to imagine what the Fates would do with them. (Dukie, known in our early discussions as "Dirty Boy," emerged as a fourth among equals as time went on.) Wish I had taken meticulous notes, to chart how the whole thing evolved.
Concerns were voiced in the writers' room that all of the attention devoted to Carcetti and the political story would only steal time away from those kids, where the gold was. Simon, after letting everyone have his say, stuck with the heavy City Hall stuff. And it works. That's why he's the Man.
4) You ask: "What role, if any, do the actors play in lending realism to the series? To the dialogue? In other words, how much of what we see is on the page before shooting begins?"
It's all on the page. Simon has a sharp ear for dialogue, and he's spent years in West Baltimore, so he knows what those characters are supposed to sound like, more so than I, and more so than any given actor. So, the actors bring their actor stuff, that human emotional stuff, without the burden of having to make some fake-ass white-boy ghetto-speak sound authentic. All of those actors, I'm sure, are grateful for that.
I'm somewhat sympathetic to the racialist critique of white middle-class writers presuming to tell black ghetto stories. But in the end, good art trumps everything.