Uncharted: Big Data as a Lens on Human Culture

by Erez Aiden, Jean-Baptiste Michel

Overview

Breaking open Big Data, two Harvard scientists reveal a ground-breaking way of looking at history and culture.

One of the greatest untapped resources of today isn’t offshore oil or natural gas—it’s data. Gigabytes, exabytes (that’s one quintillion bytes) of data are sitting on servers across the world. So how can we start to access this explosion of information, this “big data,” and what can it tell us?
 
Erez Aiden and Jean-Baptiste Michel are two young scientists at Harvard who started to ask those questions. They teamed up with Google to create the Ngram Viewer, a Web-based tool that can chart words throughout the massive Google Books archive, sifting through billions of words to find fascinating cultural trends. On the day that the Ngram Viewer debuted in 2010, more than one million queries were run through it.
 
On the front lines of Big Data, Aiden and Michel realized that this big dataset—the Google Books archive that contains remarkable information on the human experience—had huge implications for looking at our shared human history. The tool they developed to delve into the data has enabled researchers to track how our language has evolved over time, how art has been censored, how fame can grow and fade, how nations trend toward war. How we remember and how we forget. And ultimately, how Big Data is changing the game for the sciences, humanities, politics, business, and our culture.

Product Details

ISBN-13:	9781594487453
Publisher:	Penguin Publishing Group
Publication date:	12/26/2013
Pages:	288
Product dimensions:	5.90(w) x 8.30(h) x 1.20(d)
Age Range:	18 Years

About the Author

Erez Aiden is a Fellow at the Harvard Society and a former visiting faculty member at Google. He is the recipient of numerous awards, including the Presidential Early Career Award for Scientists and Engineers from Barack Obama. He lives in Cambridge, Massachusetts.

Jean-Baptiste Michel is a French and Mauritian scientist at Harvard University and a former visiting faculty member at Google. He was a 2012 TED Fellow and recently named one of Forbes “30 under 30.” He lives in New York.

Table of Contents

1 Through the Looking Glass 1

How many words is a picture worth? 24

2 G. K. Zipf and the Fossil Hunters

Burnt, baby, burnt 51

3 Armchair Lexicographerologists 53

Daddy, where do babysitters come from? 79

4 7.5 Minutes of Fame 81

One giant leapfrog for mankind 120

5 The Sound of Silence 122

Two rights make another right 150

6 The Persistence of Memory 152

Mommy, where do Martians come from? 182

7 Utopia, Dystopia, and Dat(a)topia 185

Appendix: Great Battles of History 213

Acknowledgments 238

Notes 243

Index 273

What People are Saying About This

From the Publisher

“One of the most exciting developments from the world of ideas in decades, presented with panache by two frighteningly brilliant, endearingly unpretentious, and endlessly creative young scientists.” – Steven Pinker, author of The Better Angels of Our Nature
 

Editorial Reviews

03/31/2014
Aiden and Michel gained widespread media attention when they first demonstrated their innovative use of the Google Books project, which made available more than 30 million books in digitized form—about one in every four books published. This "big data" is at the core of this fascinating glimpse into the pair's decade-long work and how "n the coming decades, personal, digital, and historical records are going to totally transform the way we think about ourselves and the world around us." Using a new scientific tool specially designed to be used with Google Books, the Ngram viewer, the pair were able to count words for "track certain kinds of cultural change over time" and to make "careful measurements that probe important aspects of our history, language, and culture." The result is like using a new kind of telescope that allows one to see more closely—and accurately—the evolution of words and how this reflects cultural change. A long appendix of charts provides a range of fascinating Ngram-based insights as well—such as the fact that the word "data" over the past hundred years has become more commonly used than the word "God." (Dec.)

Publishers Weekly

Harvard Society Fellow Aiden and Harvard University scientist Michel, recently named one of Forbes's "30 under 30," here address a hot topic: mining big data. Wondering what all those data on all those servers worldwide could tell us, they joined with Google to build the Ngram Viewer, a web-based tool that can chart words throughout the Google Books archive. The one million-plus queries run through the viewer since 2010 reveal startling cultural patterns on everything from how languages change over time to how art has been censored. Not just for geeks.

Library Journal

2013-11-04
The story of a remarkable scientific tool that uses big data sets to examine cultural trends in human history. In this debut, Aiden (Genetics/Baylor Coll. of Medicine) and Michel, founder of data science company Quantified Labs, describe research with big data that led to their teaming up with Google to develop the Ngram Viewer, an online tool that searches more than 30 million digitized books to reveal how words and phrases have been used over time. Launched in 2010 as part of Google Books, the viewer's search of ngrams (letter combinations) serves the needs of lexicographers and historians while providing endless diversion for others. Calling Google's digitized data "an unprecedented précis of humanity's cultural record," the authors show how such data can be made to reveal important changes over time, from when the early expression "the United States are" gave way to "the United States is" to how censorship can cause the sudden disappearance of particular words and phrases, such as "Tiananmen Square." Having met at Harvard, the authors began seven years ago to experiment with their new scope on historical trends to learn how English grammar changes, how people get famous, and how societies learn and forget. While recounting the copyright, privacy and other issues they faced in developing their tool, they offer fascinating insights into how dictionaries work, the half-lives of irregular verbs and the most famous people of the last two centuries (Hitler heads the list). In an appendix, some two dozen charts graph the relative frequency of use of certain words, such as "London" and "New York," since 1800. (New York began its ascendancy in 1911.) The authors also consider the moral issues raised by the prospect of a future in which personal, digital and historical records reveal more and more about human experience. A fun, revealing exploration of a new way to view the past.

Kirkus Reviews

Praise for Uncharted

“Aiden and Michel are big data pioneers, transforming how humanity thinks about itself. Uncharted is a magical, fun, fast and informative read. Every page brims with insight and humor.”
—Kenneth Cukier, co-author of Big Data: A Revolution That Will Transform How We Live, Work, and Think

“One of the most exciting developments from the world of ideas in decades, presented with panache by two frighteningly brilliant, endearingly unpretentious, and endlessly creative young scientists.”
—Steven Pinker, author of The Better Angels of Our Nature

“[A]musing, enlightening… Erez Aiden and Jean-Baptiste Michel show that our books are crammed with revelations about history, culture, economics, and politics that would even surprise their authors…. The resulting insights may shift our thinking about matters great and small." — Boston Globe

"Entertaining… This may be potato chips for intellectuals, but it is irresistible. You cannot eat just one ngram." — New York Times

“Erez and Michel are keen, lighthearted guides through their complex world… fascinating.” — San Francisco Chronicle

“Literary lovechild of: Raymond Williams’ Keywords: A Vocabulary of Culture and Society and Nate Silver’s The Signal and the Noise: Why So Many Predictions Fail—but Some Don’t…. Aiden and Michel have made fascinating discoveries about everything from the speed of fame to Soviet censorship. That they’re only just skimming the surface is hugely exciting.” — Slate

“[Aiden and Michel] offer fascinating insights… A fun, revealing exploration of a new way to view the past.”
—Kirkus Reviews

“[Aiden and Michel] reveal a wealth of historical nuggets… [E]ven math-phobic readers may glean some fascinating sociological tidbits.” – Booklist

 

From the Publisher