If you start at the Bay Bridge and head west along most major streets in San Francisco, you’ll eventually get to a magical land of misery known as the Sunset. The name is a joke, and perhaps even a way to trick tourists: The sun rarely visits the Sunset, not even when it sets. The primary weather element in the Sunset is fog—thick, endless, depressive clouds of it that wash up from the ocean to completely saturate the land. I lived in the Sunset for a single, terrible year. Before I moved there, I used to be one of those snobby city-dwellers who’d look down on suburbanites who couldn’t handle San Francisco’s famously capricious climate. I’d heard the Sunset’s weather wasn’t great, but hey, how bad could it be?
It was bad. Too bad for me; after our lease was up, my wife and I moved to the suburbs. Looking back, what bothered me most wasn’t the terrible climate—though I did hate it—but the vast difference between the Sunset’s weather and the weather everywhere else. Whatever meteorological patterns applied in normal parts of San Francisco didn’t seem to apply to the Sunset, which meant that forecasts for the city held no sway there. If the weatherman said it was going to be 80 and sunny, it was probably 55 and cloudy at my house.
San Francisco’s weather is an extreme example of a common phenomenon in forecasting: microclimatic variances. We like to think of the weather as being one thing, but it’s not—the weather across town is often different, usually much better, from the weather where you are. It’s this disconnect that makes us hate weather forecasters, and it’s what makes forecasters hate the weather.
Now, finally, meteorologists have figured out how to deal with this variance. Last week Weather Underground, a pioneering weather site, launched a new forecasting algorithm that takes local variations into account. The site has been working on the new system for many years, but until last week the site’s default forecast for American locations still came from the National Weather Service. But now when you type in your zip code, you’ll get a forecast from Weather Underground’s proprietary system, called BestForecast, that’s tailor-made to where you are. There’s a good chance it will give you a more accurate picture of what next week will look like, and how much better it will be for your friends who live in better parts of the city.
Why is Weather Underground’s forecast better? It relies on more data. To tell you what’s going to happen next week, meteorologists have to know what’s going on today. Over the last few decades the National Weather Service and other agencies worldwide have built up a robust system of weather stations. There are about 1,000 National Weather Service weather stations in the United States, mainly at official locations like airports. A spokesman told me there are an additional 100 locations around the country where the NWS launches weather balloons twice a day. And finally, the service relies on volunteer reports from the public—a network of 11,000 “cooperative observers” who provide their data to the service, plus 300,000 people who’ve been trained to spot severe weather events and report what they see.
Weather Underground’s system takes most of this NWS data into account, and then it adds even more. In particular, the site has assembled a huge network of constantly updating automated weather stations. These stations are owned and maintained by weather enthusiasts—people who love to track precipitation in their own backyards. They agree to share their data with Weather Underground because the site offers free archiving; you can see what your station was reporting months or years ago, easily, from anywhere. And weather enthusiasts are a generous bunch—they love the weather so much that they want to share it with everyone, and they’ve been pestering Weather Underground for years to incorporate personal weather station data into its forecasts.