Read an Excerpt

Introduction

The tracking of top-seller lists is a national obsession. Our culture is a massive popularity contest. We are consumed by hits—making them, choosing them, talking about them, and following their rise and fall. Every weekend is a box-office horse race, and every Thursday night is a Darwinian struggle to find the fittest TV show and let it live to see another week. A few hit songs play in heavy rotation on the radio dials, while entertainment executives in all these industries sweat as they search for the next big thing.

This is the world the blockbuster built. The massive media and entertainment industries grew up over the past half century on the back of box-office rockets, gold records, and double-digit TV ratings. No surprise that hits have become the lens through which we observe our own culture. We define our age by our celebrities and mass-market products—they are the connective tissue of our common experience. The star-making system that Hollywood began eight decades ago has now spun out into every corner of commerce, from shoes to chefs. Our media is obsessed with what's hot and what's not. Hits, in short, rule.

Yet look a little closer and you'll see that this picture, which first emerged with the postwar broadcast era of radio and television, is now starting to tatter at the edges. Hits are starting to, gasp, rule less. Number one is still number one, but the sales that go with that are not what they once were.

Most of the top fifty best-selling albums of all time were recorded in the seventies and eighties (the Eagles, Michael Jackson), and none of them were made in the past five years. Hollywood box-office revenue was down by more than 6 percent in 2005, reflecting the reality that the theatergoing audience is falling even as the population grows.

Every year network TV loses more of its audience to hundreds of niche cable channels. Males age eighteen to thirty-four, the most desirable audience for advertisers, are starting to turn off the TV altogether, shifting more and more of their screen time to the Internet and video games. The ratings of top TV shows have been falling for decades, and the number one show today wouldn't have made the top ten in 1970.

In short, although we still obsess over hits, they are not quite the economic force they once were. Where are those fickle consumers going instead? No single place. They are scattered to the winds as markets fragment into countless niches. The one big growth area is the Web, but it is an uncategorizable sea of a million destinations, each defying in its own way the conventional logic of media and marketing.

iTunes Killed the Radio Star

I came of age in the peak of the mass-culture era—the seventies and eighties. The average teenager then had access to a half dozen TV channels, and virtually everyone watched a few or more of the same handful of TV shows. There were three or four rock radio stations in any town that largely dictated what music people listened to; only a few lucky kids with money built record collections that ventured farther afield.

We all saw the same summer blockbusters in the theater and got our news from the same papers and broadcasts. About the only places you could explore outside the mainstream were the library and the comic book shop. As best I can recall, the only culture I was exposed to other than mass culture was books and whatever my friends and I made up, and that traveled no farther than our own backyards.

Contrast my adolescence with that of Ben, a sixteen-year-old who grew up with the Internet. He's the single child of affluent parents in the tony North Berkeley Hills, so he's got a Mac in his bedroom, a fully stocked iPod (and a weekly iTunes allowance), and a posse of friends with the same. Like the rest of his teenage friends, Ben has never known a world without broadband, cell phones, MP3s, TiVo, and online shopping.

The main effect of all this connectivity is unlimited and unfiltered access to culture and content of all sorts, from the mainstream to the farthest fringe of the underground. Ben is growing up in a different world from the one I grew up in, a world far less dominated by any of the traditional media and entertainment industries. If you don't recognize yourself in the pages to come in this book, imagine Ben instead. His reality is the leading edge of all of our futures.

From Ben's perspective, the cultural landscape is a seamless continuum from high to low, with commercial and amateur content competing equally for his attention. He simply doesn't distinguish between mainstream hits and underground niches—he picks what he likes from an infinite menu where Hollywood movies and player-created video-game stunt videos are listed side by side.

Ben watches just two hours or so a week of regular TV, mostly West Wing (time shifted, of course) and Firefly, a canceled space serial he has stored on his TiVo. He also counts as TV the anime he downloads with BitTorrent, a peer-to-peer file-sharing technology, because it was originally broadcast on Japanese television (the English subtitles are often edited in by fans).

When it comes to movies, he's a sci-fi fan, so he's pretty mainstream. Star Wars is a passion, as was the Matrix series. But he also watches movies he downloads, such as amateur machinima (movies made by controlling characters in video games) and independent productions such as Star Wars Revelations, a fan-created tribute film with special effects that rival the Lucas originals.

Some of the music on his iPod is downloaded from iTunes, but most comes from his friends. When one of the group buys a CD, he or she typically makes copies for everyone else. Ben's taste is mostly classic rock—Led Zeppelin and Pink Floyd—with a smattering of video-game soundtracks. The only radio he listens to is when his parents turn on NPR in the car.

Ben's reading ranges from Star Wars novels to Japanese manga, with a large helping of Web comics. He, like a few of his friends, is so into Japanese subculture that he's studying Japanese in school. When I was in school, kids studied Japanese because Japan was a dominant economic power and language skills were thought to open up career opportunities. But now kids study Japanese so they can create their own anime subtitles and dig deeper into manga than the relatively mainstream translated stuff.

Most of Ben's free time is spent online, both randomly surfing and participating in user forums such as Halo and Star Wars discussion sites. He's not interested in news—he reads no newspapers and watches no TV news—but follows the latest tech and subculture chatter on sites such as Slashdot (geek news) and Fark (weird news). He instant messages constantly all day with his ten closest friends. He doesn't text much on his cell phone, but he has friends that do. (Texting is preferred by those who are out and about a lot; IM is the chat channel of choice for those who tend to spend more time in their own rooms.) He plays video games with friends, mostly online. He thinks Halo 2 rocks, especially the user-modified levels.

I suspect that had I been born twenty-five years later, my teenage years would have been quite similar. The main difference between Ben's adolescence and my own is simply choice. I was limited to what was broadcast over the airwaves. He's got the Internet. I didn't have TiVo (or even cable); he has all that and BitTorrent, too. I had no idea there was even such a thing as manga, much less how to get it. Ben has access to it all. Would I have watched Gilligan's Island reruns if I'd been able to build a clan with friends in World of Warcraft online instead? I doubt it.

TV shows were more popular in the seventies than they are now not because they were better, but because we had fewer alternatives to compete for our screen attention. What we thought was the rising tide of common culture actually turned out to be less about the triumph of Hollywood talent and more to do with the sheepherding effect of broadcast distribution.

The great thing about broadcast is that it can bring one show to millions of people with unmatchable efficiency. But it can't do the opposite—bring a million shows to one person each. Yet that is exactly what the Internet does so well. The economics of the broadcast era required hit shows—big buckets—to catch huge audiences. The economics of the broadband era are reversed. Serving the same stream to millions of people at the same time is hugely expensive and wasteful for a distribution network optimized for point-to-point communications.

There's still demand for big cultural buckets, but they're no longer the only market. The hits now compete with an infinite number of niche markets, of any size. And consumers are increasingly favoring the one with the most choice. The era of one-size-fits-all is ending, and in its place is something new, a market of multitudes.

This book is about that market.

This shattering of the mainstream into a zillion different cultural shards is something that upsets traditional media and entertainment no end. After decades of executives refining their skill in creating, picking, and promoting hits, those hits are suddenly not enough. The audience is shifting to something else, a muddy and indistinct proliferation of . . . Well, we don't have a good term for such non-hits. They're certainly not "misses," because most weren't aimed at world domination in the first place. They're "everything else."

It's odd that this should be an overlooked category. We are, after all, talking about the vast majority of everything. Most movies aren't hits, most music recordings don't make the top 100, most books aren't best-sellers, and most video programs don't even get measured by Nielsen, much less clean up in prime time. Many of them nevertheless record audiences in the millions worldwide. They just don't count as hits, and are therefore not counted.

But they're where the formerly compliant mass market is scattering to. The simple picture of the few hits that mattered and the everything else that didn't is now becoming a confusing mosaic of a million mini-markets and micro-stars. Increasingly, the mass market is turning into a mass of niches.

That mass of niches has always existed, but as the cost of reaching it falls—consumers finding niche products, and niche products finding consumers—it's suddenly becoming a cultural and economic force to be reckoned with.

The new niche market is not replacing the traditional market of hits, just sharing the stage with it for the first time. For a century we have winnowed out all but the best-sellers to make the most efficient use of costly shelf space, screens, channels, and attention. Now, in a new era of networked consumers and digital everything, the economics of such distribution are changing radically as the Internet absorbs each industry it touches, becoming store, theater, and broadcaster at a fraction of the traditional cost.

Think of these falling distribution costs as a dropping waterline or a receding tide. As they fall, they reveal a new land that has been there all along, just underwater. These niches are a great uncharted expanse of products that were previously uneconomic to offer. Many of these kinds of products have always been there, just not visible or easy to find. They are the movies that didn't make it to your local theater, the music not played on the local rock radio station, the sports equipment not sold at Wal-Mart. Now they're available, via Netflix, iTunes, Amazon, or just some random place Google turned up. The invisible market has turned visible.

Other niche products are new, created by an emerging industry at the intersection between the commercial and noncommercial worlds, where it's hard to tell when the professionals leave off and the amateurs take over. This is the world of bloggers, video-makers, and garage bands, all suddenly able to find an audience thanks to those same enviable economics of digital distribution.

The 98 Percent Rule

This book began with a quiz I got wrong. One of the things I do as the editor of Wired is give speeches about technology trends. Because I started my career in the science world and then learned economics at The Economist, I look for those trends first in hard data. And, fortunately enough, there has never been more data available. The secrets of twenty-first-century economics lie in the servers of the companies that are all around us, from eBay to Wal-Mart. Although it's not always easy to get the raw numbers, the executives at those companies swim in that data every day and have a great intuitive feel for what's meaningful and what isn't. So the trick to trend-spotting is to ask them.

Which is what I was doing in January 2004, in the offices of Robbie Vann-Adibé, the CEO of Ecast, a "digital jukebox" company. Digital jukeboxes are just like regular jukeboxes—a big enclosure with speakers and blinking lights, often found in bars—with the difference that rather than a hundred CDs, they have a broadband connection to the Internet and patrons can choose from thousands of tracks that are downloaded and stored on a local hard drive.

During the course of our conversation, Vann-Adibé asked me to guess what percentage of the 10,000 albums available on the jukeboxes sold at least one track per quarter.

I knew, of course, that Vann-Adibé was asking me a trick question. The normal answer would be 20 percent because of the 80/20 Rule, which experience tells us applies practically everywhere. That is: 20 percent of products account for 80 percent of sales (and usually 100 percent of the profits).

But Vann-Adibé was in the digital content business, which is different. So I thought I'd go way out on a limb and venture that a whopping 50 percent of those 10,000 albums sold at least one track a quarter.

Now, on the face of it, that's absurdly high. Half of the top 10,000 books in a typical book superstore don't sell once a quarter. Half of the top 10,000 CDs at Wal-Mart don't sell once a quarter; indeed, Wal-Mart doesn't even carry half that many CDs. It's hard to think of any market where such a high fraction of such a large inventory sells. But my sense was that digital was different, so I took a chance on a big number.

I was, needless to say, way, way off. The answer was 98 percent.

"It's amazing, isn't it?" Vann-Adibé said. "Everyone gets that wrong." Even he had been stunned: As the company added more titles to its collections, far beyond the inventory of most record stores and into the world of niches and subcultures, they continued to sell. And the more the company added, the more they sold. The demand for music beyond the hits seemed to be limitless. True, the songs didn't sell in big numbers, but nearly all of them sold something. And because these were just bits in a database that cost nearly nothing to store and deliver, all those onesies and twosies started to add up.

What Vann-Adibé had discovered was that the aggregate market for niche music was huge, and effectively unbounded. He called this the "98 Percent Rule." As he later put it to me, "In a world of almost zero packaging cost and instant access to almost all content in this format, consumers exhibit consistent behavior: They look at almost everything. I believe that this requires major changes by the content producers—I'm just not sure what changes!"

I set out to answer that question. I realized that his counterintuitive statistic contained a powerful truth about the new economics of entertainment in the digital age. With unlimited supply, our assumptions about the relative roles of hits and niches were all wrong. Scarcity requires hits—if there are only a few slots on the shelves or the airwaves, it's only sensible to fill them with the titles that will sell best. And if that's all that's available, that's all people will buy.

But what if there are infinite slots? Maybe hits are the wrong way to look at the business. There are, after all, a lot more non-hits than hits, and now both are equally available. What if the non-hits—from healthy niche product to outright misses—all together added up to a market as big as, if not bigger than, the hits themselves? The answer to that was clear: It would radically transform some of the largest markets in the world.

And so I embarked on a research project that was to take me to all the leaders in the emerging digital entertainment industry, from Amazon to iTunes. Everywhere I went the story was the same: Hits are great, but niches are emerging as the big new market. The 98 Percent Rule turned out to be nearly universal. Apple said that every one of the then 1 million tracks in iTunes had sold at least once (now its inventory is twice that). Netflix reckoned that 95 percent of its 25,000 DVDs (that's now 55,000) rented at least once a quarter. Amazon didn't give out an exact number, but independent academic research on its book sales suggested that 98 percent of its top 100,000 books sold at least once a quarter, too. And so it went, from company to company.

Each company was impressed by the demand they were seeing in categories that had been previously dismissed as beneath the economic fringe, from the British television series DVDs that are proving surprisingly popular at Netflix to the back-catalog music that's big on iTunes. I realized that, for the first time, I was looking at the true shape of demand in our culture, unfiltered by the economics of scarcity.

That shape is, to be clear, really, really weird. To think that basically everything you put out there finds demand is just odd. The reason it's odd is that we don't typically think in terms of one unit per quarter. When we think about traditional retail, we think about what's going to sell a lot. You're not much interested in the occasional sale, because in traditional retail a CD that sells only one unit a quarter consumes exactly the same half-inch of shelf space as a CD that sells 1,000 units a quarter. There's a value to that space—rent, overhead, staffing costs, etc.—that has to be paid back by a certain number of inventory turns per month. In other words, the onesies and twosies waste space.

However, when that space doesn't cost anything, suddenly you can look at those infrequent sellers again, and they begin to have value. This was the insight that led to Amazon, Netflix, and all the other companies I was talking to. All of them realized that where the economics of traditional retail ran out of steam, the economics of online retail kept going. The onesies and twosies were still only selling in small numbers, but there were so, so many of them that in aggregate they added up to a big business.

Throughout the first half of 2004 I fleshed out this research in speeches, the thesis advancing with each talk. Originally the speech was called "The 98 Percent Rule." Then it was "New Rules for the New Entertainment Economy" (not one of my better naming moments).

But by then I had some hard data, thanks to Rhapsody, which is one of the online music companies. They had given me a month's worth of customer usage data, and when I graphed it out, I realized that the curve was unlike anything I'd seen before.

It started like any other demand curve, ranked by popularity. A few hits were downloaded a huge number of times at the head of the curve, and then it fell off steeply with less popular tracks. But the interesting thing was that it never fell to zero. I'd go to the 100,000th track, zoom in, and the downloads per month were still in the thousands. And the curve just kept going: 200,000, 300,000, 400,000 tracks—no store could ever carry this much music. Yet as far as I looked, there was still demand. Way out at the end of the curve, tracks were being downloaded just four or five times a month, but the curve still wasn't at zero.

In statistics, curves like that are called "long-tailed distributions," because the tail of the curve is very long relative to the head. So all I did was focus on the tail itself, turn it into a proper noun, and "The Long Tail" was born. It started life as slide 20 of one of my "New Rules" presentations. I think it was Reed Hastings, the CEO of Netflix, who convinced me that I was burying my lead. By the summer of 2004 "The Long Tail" was not just the title of my speeches; I was nearly finished with an article of the same name for my own magazine.

When "The Long Tail" was published in Wired in October 2004, it quickly became the most cited article the magazine had ever run. The three main observations—(1) the tail of available variety is far longer than we realize; (2) it's now within reach economically; (3) all those niches, when aggregated, can make up a significant market—seemed indisputable, especially backed up with heretofore unseen data.

Tails Everywhere

One of the most encouraging aspects of the overwhelming response to the original article was the breadth of industries in which it resonated. The article originated as an analysis of the new economics of the entertainment and media industries, and I only expanded it a bit to mention in passing that companies such as eBay (with used goods) and Google (with small advertisers) were also Long Tail businesses. Readers, however, saw the Long Tail everywhere, from politics to public relations, and from sheet music to college sports.

What people intuitively grasped was that new efficiencies in distribution, manufacturing, and marketing were changing the definition of what was commercially viable across the board. The best way to describe these forces is that they are turning unprofitable customers, products, and markets into profitable ones. Although this phenomenon is most obvious in entertainment and media, it's an easy leap to eBay to see it at work more broadly, from cars to crafts.

Seen broadly, it's clear that the story of the Long Tail is really about the economics of abundance—what happens when the bottlenecks that stand between supply and demand in our culture start to disappear and everything becomes available to everyone.

People often ask me to name some product category that does not lend itself to Long Tail economics. My usual answer is that it would be in some undifferentiated commodity, where variety is not only absent but unwanted. Like, for instance, flour, which I remembered being sold in the supermarket in a big bag labeled "Flour." Then I happened to step inside our local Whole Foods grocery and realized how wrong I was: Today the grocery carries more than twenty different types of flour, ranging from such basics as whole wheat and organic varieties to exotics such as amaranth and blue cornmeal. There is, amazingly enough, already a Long Tail in flour.

Our growing affluence has allowed us to shift from being bargain shoppers buying branded (or even unbranded) commodities to becoming mini-connoisseurs, flexing our taste with a thousand little indulgences that set us apart from others. We now engage in a host of new consumer behaviors that are described with intentionally oxymoronic terms: "massclusivity," "slivercasting," "mass customization." They all point in the same direction: more Long Tails.

A Preview of Twenty-first-Century Economics

This book is partly an economic research project, with the help and involvement of students and professors from the Stanford, MIT, and Harvard business schools. It's partly the fruit of more than a hundred speeches, brainstorming sessions, and site visits with companies and industry groups that see the Long Tail changing their world. And it's partly a collaboration with the dozens of companies and executives who shared many megabytes of internal data, giving me an unprecedented view on the emerging micro-economics of markets in the online age.

What's fascinating about this moment is that the economics of the twenty-first century are already evident in outline form in the databases of the Googles, Amazons, Netflixes, and iTunes of the world. In those many terabytes of user behavior data is a clue to how consumers will behave in markets of infinite choice, a question that hadn't been meaningful until recently but has now become essential to understand.

Surprisingly, very few economists are looking at this data, mostly because they haven't asked (most of the academics I worked with are in business schools, only a few of them are economists). There are some exceptions—University of California Berkeley economist Hal Varian works part-time at Google, and auction-theory economists unsurprisingly love eBay—but they're rare. Some of the data in this book has never before seen the light of day.

Given the uncharted waters, I solicited a lot of help from experts in all corners. As an experiment, I worked through many of the trickier conceptual and articulation issues in public, on my blog at thelongtail.com. The usual process would go like this: I'd post a half-baked effort at explaining how the 80/20 Rule is changing, for instance, and then dozens of smart readers would write comments, emails, or their own blog posts to suggest ways to improve it. Somehow this wonky public brainstorming managed to attract an average of more than 5,000 readers a day.

In software, developers release early ("beta") versions of their code to their most avid users. In exchange for the privileged early look at the program, these users test it on their own machines, in their own way, and find errors that the developer missed. Such beta-testing is essential to creating robust software applications. My hope is that the same process—stress-testing many of my ideas in public—has led to a better, or at least sounder, book.

I should note here the difference between beta-testing ideas in public and actually writing a book in public. Although many have tried to do the latter—posting draft chapters online and sometimes even opening the text to collective editing—I chose to use the blog mostly as a public diary of my research in progress. The actual writing of the book, and most of the words in the following pages, I did offline.

Finally, one more note on parentage. Although I coined the term "The Long Tail," I can't claim any credit for creating the concept of using the efficient economics of online retail to aggregate a large inventory of relatively low sellers. That would be Amazon's Jeff Bezos, circa 1994. Most of what I've learned has come from talking to him, his counterparts at Netflix and Rhapsody, and others who have all been acting on this for years.

Those entrepreneurs are the real inventors here. What I've tried to do is synthesize the results into a framework. That is, of course, what economics does: It seeks to find neat, easily understood frameworks that describe real-world phenomena. Coming up with the framework is an advance in itself, but it pales next to the original inventions of all those who discovered and acted on the phenomena in the first place.

Excerpted from THE LONG TAIL by Chris Anderson. Copyright © 2006 Chris Anderson. All rights reserved.