[iDC] The wisdom of the few?

Thu Feb 28 07:35:59 UTC 2008

I think part of the confusion is that the authors of the original study 
contradict themselves. In their CHI paper 
(http://www.parc.com/research/publications/files/5904.pdf) they report 
results that support what Jeff is saying. But in a later blog post 
(http://asc-parc.blogspot.com/2007/05/long-tail-and-power-law-graphs-of-user.html), 
they note some problems with their original analysis. When those 
problems are corrected, the story flips entirely, showing that the 
amount of work done by the elites is either stable or slightly 
increasing. The Slate author cites both.

As I understand it, part of the problem was that in the CHI paper they 
defined 'elites' as the top 1% across the life of Wikipedia. But they 
realized later that to look at the trend over time they needed to define 
the top 1% for each given month, and then look at the work those people 
did in that month, and plot that. In other words, they needed a 
month-by-month list of elites rather than an all-time list.

My read is that both Chris (from Slate) and Jeff over-simplify a bit. 
Even reading through the CHI paper, which generally supports what Jeff 
is saying, there are conflicting signals. For example, on the one hand 
the 'elite' group seems to be making fewer edits, but on the other 
they're making larger edits on average.

One take-away from this, for me, is that we are at the edges of what 
these large-scale data-crunching projects are capable of understanding. 
In this case, the results turned out to be hugely model-dependent - 
something that in general should give us less confidence in both 
results. Not only that, but they lack the crucial qualitative component 
to tell us what's really going on in the social system. Even very 
careful analysis of diffs, user lists, and change logs can only say so much.

I tend to think that things with Wikipedia are as they have (mostly) 
always been. Smaller contributors do quite a lot of work, but organizing 
it, filtering it, and editing it, falls in stages to more specialized 
and powerful social groups within the Wikipedia community. In that 
sense, even though the Slate piece ended up being a bit too 
sensationalist for my taste, its title is pretty apt.

--Judd

--
Judd Antin
School of Information
University of California Berkeley
jantin at ischool.berkeley.edu
web: http://technotaste.com
blog: http://technotaste.com/blog

Howe, Jeff wrote:
> Attention to anyone who follows this link. Slate writer totally (and 
> inexplicably) misrepresents the PARC study that was supposedly providing 
> the evidence for his piece. My advice? Skip the article and go straight 
> to the study (attached), that reveals that the tide is starting to shift 
> again toward increasing participation on the part of the crowd, as 
> opposed to the few.
> 
> My email to Slate writer, fwiw:
> 
> Chris,
> 
> Thanks for the article. I hadn’t read the study, and found it valuable. 
> I believe you’ve misrepresented the conclusions Chi, et. al reached 
> though comments, my apologies. The prevalence of the 80/20 rule in 
> social media is an old story—got heaps of ink when Wales first revealed 
> that 2.5 percent of Wiki contribs were doing the heavy lifting back in 
> 04, but this study actually reveals something genuinely newsworthy (and 
> diametrically opposed to the angle on your piece):
> 
> The results suggest that although Wikipedia was driven by the
> influence of “elite” users early on, more recently there has
> been a dramatic shift in workload to the “common” user.
> 
> Now /that’s/ worth writing about. Sort of a closing of the arc. These 
> sites were championed as paragons of the Web’s democratic nature, then 
> the truth came out that a tiny percentage of “elites” were responsible 
> for most of the content, and now indications are showing that with 
> continued usage, the crowd is indeed taking over some of that burden.
> 
> I’d respectfully suggest you owe the good folks at PARC a correction, 
> but that’s between you and your ed.   
>  
> All Best,
> 
> Jeff Howe
> 
> 
> On 2/27/08 3:56 PM, "Steve Cisler" <sacisler at yahoo.com> wrote:
> 
>       Here's another critical view of a some so-called Web2.0 services
>     focusing on the dominance of the hyper-connected few:
>       
>       The Wisdom of the Chaperones Digg, Wikipedia, and the myth of Web
>     2.0 democracy.
>       By Chris Wilson
>       Posted Friday, Feb. 22, 2008, at 6:11 PM ET
>       
>       http://www.slate.com/id/2184487/
>     ------------------------------------------------------------------------
>     _______________________________________________
>     iDC -- mailing list of the Institute for Distributed Creativity
>     (distributedcreativity.org)
>     iDC at mailman.thing.net
>     https://mailman.thing.net/mailman/listinfo/idc
> 
>     List Archive:
>     http://mailman.thing.net/pipermail/idc/
> 
>     iDC Photo Stream:
>     http://www.flickr.com/photos/tags/idcnetwork/
> 
>     RSS feed:
>     http://rss.gmane.org/gmane.culture.media.idc
> 
>     iDC Chat on Facebook:
>     http://www.facebook.com/group.php?gid=2457237647
> 
>     Share relevant URLs on Del.icio.us by adding the tag iDCref
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> iDC -- mailing list of the Institute for Distributed Creativity (distributedcreativity.org)
> iDC at mailman.thing.net
> https://mailman.thing.net/mailman/listinfo/idc
> 
> List Archive:
> http://mailman.thing.net/pipermail/idc/
> 
> iDC Photo Stream:
> http://www.flickr.com/photos/tags/idcnetwork/
> 
> RSS feed:
> http://rss.gmane.org/gmane.culture.media.idc
> 
> iDC Chat on Facebook:
> http://www.facebook.com/group.php?gid=2457237647
> 
> Share relevant URLs on Del.icio.us by adding the tag iDCref