Why these books, why now? Boy, that's a hard one. Could be a simple matter of demographics: Boomers' kids are old enough to know what they like and say so, and there are enough of them to give their collective voice a certain volume. Or maybe it's a cyclical thing: Fantasy was last in fashion during the '70s, when (as you point out) Star Wars had its astonishing apotheosis. Is it coming back like platform shoes? Or maybe the explanation is Rowling's quick, enchanted pen--she's turned out three Potter stories in two years, leaving her readers no time to lose their appetites. Or could it be our ambivalence about the recent furious surge of moody technology, giving us apparently magical powers that we can't quite control? Or is someone at Scholastic a secret marketing genius? Or is Rowling herself a wizard? What's your pet theory?
I wonder whether it's true, as you claim, that children are less likely than adults to put up with being preached at. The authors of children's books on the whole certainly don't seem to think so. Classics like Little Women and The Secret Garden preach shamelessly without losing their passionate young groupies, even today. Kids, after all, get plenty of lectures ("discipline," "limits") from their beloved parents. The more recent crop of books continue to preach, though they promote psychological health rather than moral rectitude. American writers are far worse in this respect than the Brits--even Konigsberg and Fitzhugh, whom I love as you do, can occasionally sound like guidance counselors. Maybe one of the reasons Rowling hit it big with grown-ups is her refusal to drum in lessons. Instead, her books get their depth from a combination of allegory and genuine human interactions that haven't been pre-chewed. She lets her characters learn from their mistakes (or fail to do so). Adults like that--they're can dish out the preaching, but they can't take it.
One of the pleasures of reading Rowling as a grown-up, which you touch on, is seeing her develop as a novelist. The first book had a simplish dramatic trajectory: Harry is sure he knows who are the good guys and who are the bad guys, but he guesses wrong. In the second book, I thought at first that she was pulling the same trick, but I should have trusted her. The apparent villain really is a villain, of sorts--but she adds surprises. As we learn more and more clearly in each volume, there are lots of ways to help the forces of darkness: You can join them with all your heart, but you can also succumb to prejudice, or self-righteousness, or some apparently silly temptation. As Hagrid, the gruff-but-loveable caretaker, makes clear, you can love the wrong kind of folks (in his case, dragons and monstrous spiders). Or you can just be a vain goofball. In each book, the enemy's helpers become increasingly refined.
The series also has some of the excitement that Dickens' readers must have felt as they read his novels in serial form, knowing he was home working frantically on the next installment. Publishing the first books before she's written the later ones, Rowling hunts out bits and pieces from earlier on and puts them to new uses. The Whomping Willow, for example, which seems like a bit of inspired slapstick in Volume 1, has an important gesture to make in Volume 3, furthering the plot. One tiny character introduced in the first book swells to chilling proportions at the end of the third. It's like watching Rowling rummage around for mice and rats to enchant into horses and coachmen. Amazingly, none of it seems artificial.
I wonder how she's going to handle the Great Battle that comes with the genre, where Good meets Bad for something like a final showdown. So far, she's given us small (if important) skirmishes, and sketched out a grand battle past. The danger of series like this is that they can so easily collapse at the end, becoming too allegorically complex or too morally simple. How do you think she'll handle it?
And do you think Percy, Ron Weasley's perfect prefect/head boy/big brother, will play a villainous role in one of the next books? I hope so.