IDP2013 (VII): Duncan Watts: When does size matter? “Big data,” the Web, and social science

Notes from the 9th Internet, Law and Politics Congress: Big Data: Challenges and Opportunities, organized by the Open University of Catalonia, School of Law and Political Science, and held in Barcelona, Spain, on 25-26 June 2013. More notes on this event: idp2013.

Duncan Watts. Principal researcher at Microsoft Research and a founding member of the MSR-NYC lab.

Social phenomena arise when individuals interact to produce collective entities, when the collective begins to have a behaviour different from what was coded at the individual level: emergence. On the other hand, it is difficult to perform measurements both at the micro- and the macro- levels at the same time.

The Internet is not only changing society, but also changing the way we study society, just like the telescope did to observing the universe. The web has dramatically increased the scale, scope and granularity of data available to social scientists.

Example: music tastes

Why do so many TV shows, films, books, etc. fail? And why do some many TV shows, films, books that are very successful were initially rejected by “experts”?

It happens that our decisions are based on others’ (friends, family, etc.) decisions, and on a cumulative way: the more people like a movie, the more likely we will end up seeing it.

Evidence shows that inequality and unpredictability both increase in presence of social influence. Indeed, increased social influence yields increased inequality and unpredictability.

“Quality” still matters, on average, but lots of scatter over individual realizations (especially for high quality). So, social influence is more powerful than “quality”. Popular songs are more popular, but at the same time, which particular songs become the popular ones becomes harder to predict. The paradox of social influence is that individuals have more information on which to base choices, but the collective choice (i.e. what becomes popular) reveals less and less about individual preferences.

Example: Twitter

Twitter is ideally suited to study influencers.

But explanation and prediction are quite different things. If we look only at large cascades, will probably find patterns, hence will feel we have “explained” them. But there are many (the majority, actually) things that just did not went viral, just did not work, or just were not “meant to work”. It is very tempting to infer causality from “events”, but causality disappears once non-events are accounted for. Individual level predictions are unreliable, even given “perfect” information: instead of targeting the individual, who have to think stochastically, see the whole picture.

Two wrong assumptions:

  • there are influencers and non-influencers: everyone is, at some time, an influencer or is influenced by someone else.
  • this distinction is stable across time and context: we sometimes are influencers, we sometimes are influenced by others.

Viral vs. popular

There is a huge difference between “viral” and “popular”. It could be that popular things spread like viruses or spread because they were widely broadcasted.

Across all domains we find considerable diversity in structural virality among large events. Video, news and images are surprisingly similar, but petitions show evidence of being more “viral” relative to their size.

Surprisingly there is low correlation between size and virality. Knowing that something is popular tells you surprisingly little about how it went viral.


Jorge Salcedo: cannot we make predictions base on some patterns that we actually find? Watts: you may not be making predictions on what is going to happen, but you may on the different probabilities of some things happening or not. So, not saying what is happening next, but what is the probability of this thing happening next. You can predict probabilities, you cannot predict actual events. Even if it is precisely that what we would like to predict.


9th Internet, Law and Politics Conference (2013)

If you need to cite this article in a formal way (i.e. for bibliographical purposes) I dare suggest:

Peña-López, I. (2013) “IDP2013 (VII): Duncan Watts: When does size matter? “Big data,” the Web, and social science” In ICTlogy, #117, June 2013. Barcelona: ICTlogy.
Retrieved month dd, yyyy from

Previous post: IDP2013 (VI): Politics

Next post: IDP2013 (VIII): Social Movements

RSS feed RSS feed for comments on this post. TrackBack URI

Leave a Reply

Your comment: