Content farms: What do they say about what we care about?

Commentary about business and finance.
March 25 2011 6:23 PM

Content Yawn

What content farms tell us about what we're interested in.

Content farms are to online media what tabloids are to print. Neither journalism nor advertising, they are a trashy and addictive product, sussing out what we really want in order to give us something we don't really need—and, in so doing, telling us something important about ourselves.

They work like this: An algorithm divines what people are searching for on the Internet, and what advertisers might pay to reach them. Content farms mostly ignore breaking news and other too-competitive searches, optimizing their content to fill the cracks neglected by retailers, newspapers, and other sites. They then spit out assignments to writers working for pennies a word, who churn out thousands of pieces of search-optimized infotainment on how to get a burger for $1 or how to make your own cat food or how to clip your toenails (watch out for the toe!) or hotels in North Dakota with hot tubs.

Few of us admit to reading this stuff, much less liking it. Even the search engines it is created for resist it: Last month Google  announced a change to its algorithm, designed to banish farmed content down into the nether regions of its search results. Still, content farms do get traffic. And their model seems compelling: They take what people are searching for and translate it into articles people are—or might be—curious enough to click on. The results speak to what we want to know, but aren't being told. As it turns out, our concerns are mostly petty, self-centered, or mundane.

To get an analysis of what content farms are farming, we crunched the numbers—and you can too. The following graph gives 30 of the most popular tags used by one of the biggest content farms. In the box below, type in any word or phrase to see its popularity and related tags. Click on the results to go to the articles tagged with the word or phrase.

A few words about our widget. First, it doesn't account for every content farm on the Web. In the past few years, companies like Demand Media have churned out more than 1 million stories or posts or bites or whatever you want to call them. So we focused on one year and one site: Yahoo!'s Associated Content in 2010. Slate's Chris Wilson and Angela Tchou constructed a bot that logged every tag—a word or phrase attached to every article, then picked up by search engines. The system collected data on more than 220,000 stories with more than 1 million tags.

With all that data compiled, we got to analyzing it ourselves. We expected to find a lot of articles catering to our one-track minds: Queries looking for pornography make up about a quarter of all Internet searches, after all. (Jeez Louise, guys!) Yet the tag sex comes in at No. 201 in popularity,  followed by sex tape (812), sexual (976), and sexy (1,014). Why so low? First, a content farm is obviously not the best place for a provider of adult content to advertise. Second, a content farm is obviously not the best place for a searcher of adult content to stay.

Rather, the top tags prove reassuringly banal: money (which appears 6,204 times in the tags), movie, show, school, family, students, business, game, years, and film. We want to know about our kids, our schoolwork, our travel, and our careers. We want to know about jobs, and what industries are growing.

Adding up and recombining the tags, one gets a better sense of what content farms are giving us. We want to know about news, but not just any news. Actual news sites—like Slateand the Huffington Post and Yahoo! News itself—have the real, newsy news covered. Publications are also increasingly savvy about performing some search-engine optimization of their own, making it harder for the farms to compete and pushing down their ad prices. Thus the big stories of 2010, like the Haiti earthquake and the midterm elections, do not make up much of the Associated Content canon. Likewise, celebrity sites have celebrities down pat—so not as many stories about Britney Spears and Lady Gaga as you would expect.

No, the news we want to know about—the underserved need—lies in part in film and television and entertainment. We want to know what is on, or what happened last night, or who went home, and it seems we need the content farms to tell us. The first proper noun that comes up in the widget is Idol, as in American. Dancing With the Stars comes in the top 100. The first company? Facebook. The first person? Sandra Bullock. Obama, for the record, ranks below American Idol contestant Lee DeWyze, Michael Jackson, and olive oil.

Then, we want to know about ourselves—specifically, our bodies. We want to know about scores of individual diseases and conditions, from discolored nails to distended bowels. We want to know about the health benefits of dozens of acts (retirement, breastfeeding, sex) and individual foods (bison, quinoa, sardines, cherries). We seem to want more speculative and stranger content than the medical sites are giving us.

Consider the tagged word cause or causes. It virtually always comes as part of a phrase: what caused a couple to break up, what caused your car to stall out in an intersection. But look at the most common words that go along with it: First, there is cause of weight gain. Some of the theories you see in the searches are: Prozac, MSG, exercise, sodium, IBS, diet soda, cereal bars, sugared sodas, antidepressants, and milk. Overeating and underexercising comes up less often than one would hope.

Then, we want to know the cause of: cancer, pain, and acne, followed by depression and hair loss. The list of ailments goes on: bad breath, baldness, death, disease, hair thinning, health, heartburn, hives, injury, cancer again, infertility, night sweats, poverty, vitamin, tinnitus, smoking, hiccups, migraine headaches, autism, back pain, blindness, dandruff, excessive thirst, hair loss again, loss, night sweats, and tinnitus again.

More than that, we want to know about our weight—how to control it, contain it, use herbs or foods or exercise to bring it down. The most commonly tagged individual food is fish (good for your diet, good for your brain). The word diet is tagged more than 1,000 times, and weight loss nearly 1,200; healthy,1,460, and eat, more than 1,500. The site provides thousands of articles about how to lose it.

Running through the tags and the articles, dozens of other strange optimizations arise. Associated Content has a corner on birthday searches, for instance. More than one article is tagged with any of the following: happy july 14th, happy july 17th, happy july 18th, happy july 19th, and so on. Unemployment comes up far less often than foreclosure does, though jobs come up far more often than both.

So what more do we know about the content farm from running through the database? It exists in the spaces that other sites neglect—answering the mundane questions we ask the Internet about our families, our friends, our bodies. It caters to our baser search instincts. What is the overall picture of us, painted by the content farm? We are, it seems, avid TV watchers who adore sports, pets, and our families, worry about our jobs, and suffer from hypochondria. But maybe none of us needed a content farm to tell us that.

Annie Lowrey is a contributing editor for New York magazine. She can be reached at annie.lowrey@gmail.com.