Then a friend suggested he run a jazz club. He leased a Boston hotel room and in 1950 opened the Storyville club. His drummer at the time was "Big" Sid Catlett, who had played with Louis Armstrong. When the trumpeter came to town, Wein asked Catlett to persuade Armstrong to drop in to Storyville. When one by one the band walked on stage, and finally Armstrong himself singing "Sleepy Time Down South", "I felt the electricity ... I said this is where I have to be, I have to be working with these people ... I made a decision that I had to work with the greats. I realised that you have to deal on a certain level ... I was losing money anyway, but I got to hang out with Art Tatum, Billie Holiday, Duke and Ella."
By the time Elaine and Louis Lorillard, the Storyville customers who were bored by their Newport summers, offered to put up $20,000, Wein had a contacts book to match his experience. Even so, he never predicted the festival's impact – no one had set up a commercial stage in a field before and there was huge press coverage – especially of the last day, when it poured with rain and photos of 5,000 people on a field listening to jazz under their umbrellas ran around the world. Wein's first bill included Billie Holiday re-uniting with Lester Young and, innovatively, had traditionalists and modernists sharing the same bill. "It was like a convention of the whole jazz world," he said. "Everybody came: managers, agents, the press, the critics."
The excitement of Newport's early years was caught in the 1958 film Jazz on a Summer's Day. Dave Brubeck and Gerry Mulligan, Ella Fitzgerald and Sarah Vaughn drew big crowds, as did Duke Ellington's comeback concert. There was also room for the new music of Cecil Taylor and Sun Ra.
A year later, in 1959, Wein produced the Newport Folk Festival, with Joan Baez and Bob Dylan headlining, and took the first of many Newport packages on tour. In 1960, he had to contend with newspaper headlines screaming "Riot at Newport Jazz Festival" after troublemakers made the most of the bars staying open late. The National Guard was summoned and Wein's festival suffered a one-year hiatus.
That year, his energies went into his consultancy and other festival productions. New Orleans was an early would-be customer but, in 1962, Wein turned the city down. "I said, 'I'd love to do a festival, but you won't allow black musicians to stay in the hotels.' They spent the whole day trying to get around their own laws." Much civil rights legislation later, Wein oversaw 1970's New Orleans Heritage Festival.
By this time free jazz had come and gone – Coltrane introduced his late-period expressionism to Newport in 1963 – and rock was steamrolling all. Wein's response was to book rock heavyweights such as Sly Stone and Led Zeppelin for the 1969 festival. But, he said, "It was the nadir of my career. I had no control. You couldn't produce with these rock and roll guys, they push you around. They were money."
The problem was that jazz was no longer self-sufficient. Sponsorship was the answer, and Wein got there first with a celebration of big bands awkwardly titled the "Schlitz Salute to Jazz"; "Kool Jazz" soon followed. Sponsors came and went. The Newport festival moved to New York and became multi-venue (another Wein innovation) before returning to Rhode Island a decade later. Wein became less involved and Newport developed into committee-driven predictability. In 2009 the festival's sponsors pulled the plug, and it seemed it was to be a victim of the credit crunch. Wein stepped in to save it, however, and has been back in charge ever since. He has lined up acts such as Michael Feinstein & Wynton Marsalis, Esperanza Spalding and Ambrose Akinmusire for this summer.
He goes to New York's jazz clubs three times a week, keeping tabs on rising musicians, and still performs himself. "I'm not necessarily more modern, but I get into modal playing sometimes, and swing a little differently." A recent line-up included the 79-year-old drummer Jimmy Cobb and 22-year-old Esperanza Spalding on bass. "I was 82," he said. "We went in and swung for two shows a night for six nights. We broke up the place. Never stopped, and so that's where I am with jazz."
Comparing past and present, Wein identified a peculiar paradox of jazz today. "There are 1,000 nights of jazz every month in New York, but the days when you could fill Carnegie Hall are gone. Jazz is bigger than ever, but there are no big names."
The audience has changed too. "When I had my club, people drank and they smoked, you could hear the tinkling of glasses. Now you go to a jazz club, it's like a church service", he said. "If you say something, people turn round and say 'sssshhhh'. I get very mad, I'm going there to have a drink and listen to some music."