[iDC] Everything is Misc - extracts and intro

[David Weinberger]

Dear IDC'ers,

I'm a long-time lurker on this list, but I haven't contributed because I
find mailing lists intimidating, and this one particularly so. Yet,
Trebor's offer to try to turn the discussion to topics raised in my
book, "Everything Is Miscellaneous" (EIM) was too good to turn down.
Thank you, Trebor.

Trebor's asked me to introduce myself. I'm a Boston-based writer with a
particular interest in how the Web affects our ideas. (Well, that
narrows the scope to just about everything!) I was a co-author of "The
Cluetrain Manifesto" (www.cluetrain.com) (2000) and the author of "Small
Pieces Loosely Joined" in 2002 (www.smallpieces.com). I taught
philosophy until I was 36, but was not in a tenure-track position, so I
joined a high-tech company in 1986 as a marketing writer. I've been a
marketing guy ever since, although in the past few years I've been able
to spend much more of my time writing than marketing. I also am involved
in some political issues and campaigns. For the past few years, I've
been a Fellow at Harvard Law's Berkman Center for Internet & Society.

Given this list's interests, it might help to know that during my
philosophical "career," Heidegger was very influential. But I'm totally
out of shape as a philosophy guy, and, frankly, was always pretty
flabby.

Here are some Web coordinates:
My main blog: www.JohoTheBlog.com.
The EIM blog: www.EverythingIsMisc.com
A hype-y bio: www.hyperorg.com/speaker/bio.html
My newsletter: www.hyperorg.com

Trebor's suggested that I post the following two excerpts from
"Everything Is Miscellaneous." These are posted with permission of my
publisher, Times Books.

NEW PROPERTIES, NEW STRATEGIES, A NEW SHAPE OF KNOWLEDGE

[From Chapter 5, pp. 100-106. The first four chapters have tried to
convince the reader that how we order and classify our world has a
history, is always the result of our culture and interests, confers
power on those who get to do the classifying, and is complex and messy.
I've also introduced the idea that there are three "orders of order":
(1) Organizing the things themselves (books, photos...not Dinge an
sich!), (2) physically separating the metadata and organizing them
(e.g., catalog cards), and (3) digitizing both the content and the
metadata. The third order requires us to invent new principles of
organization.]

College students' silverware drawers, Delicious, Flickr, the BBC and
Wikipedia are miscellaneous in different ways, except for one thing:
How their content is actually arranged does not determine how that
content can and will be arranged by their users. In some cases -
Wikipedia, for example - no one even knows exactly where the raw
contents are. These examples are miscellaneous _because_ users don't
need to know the inner organization, _because_ that inner order
doesn't result in a preferred order of use, and _because_ users have
wide flexibility to order the pieces as they want, even and especially
in unanticipated ways. This means that the miscellaneous enables _all_
of the information contained in the set to be discovered over time.

But this also means the miscellaneous doesn't much resemble our
traditional view of knowledge. Knowledge, we've thought, has four
characteristics, two of them modeled on properties of reality and two on
properties of political regimes.

As we've seen, the first characteristic of traditional knowledge is
that just as there is one reality, there is one knowledge, the same for
all. If two people have contradictory ideas about something factual, we
think they can't both be right. This is because we've assumed
knowledge is an accurate representation of reality, and the real world
cannot be self-contradictory. We treat ideas that dispute this view of
knowledge with disdain. We label them "relativism" and imagine them
to be the devil's work, we sneer at them as "postmodern" and
assume that it's just a bunch of French pseudo-intellectual gibberish,
or we say "whatever" as a license to stop thinking.

Second, we've assumed that just as reality is not ambiguous, neither
is knowledge. If something isn't clear to us, then we haven't
understood it. We may not be 100% certain whether the Nile or the Amazon
is the longest river, we but we're confident one is. Conversely, if
there's no possibility of certainty - "Which tastes better, beets or
radishes?" - we say it isn't a matter of knowledge at all.

Third, because knowledge is as big as reality, no one person can
comprehend it. So we need people who will act as filters, based on
education, experience and clear thinking. We call them experts and we
give them clipboards. They keep bad information away from us and provide
us with the very best information.

Fourth, experts achieve their position by working their way up through
social institutions. The people in these institutions are doing their
best to be honest and helpful, but, until humans achieve divinity, our
organizations will inevitably be subject to corrupting influences. Which
groups get funded can determine what a society believes, and funding is
often granted by people who know less than the experts: The fate of a
DNA research center may rest with Congresspeople who couldn't tell a
ribosome from a trombone.

The way we've organized knowledge has been largely determined by these
four properties of knowledge. We've tried try to settle on a single,
comprehensive framework for knowledge, with categories so clear and
comprehensive that experts can put each thing in its proper place.
Institutions grew to maintain the knowledge framework. Their ability to
certify experts and to vouch for knowledge made them powerful and
sometimes rich. So, when the miscellaneous shakes our certainty in the
nature of knowledge, more than the future of the card catalog is at
stake. Because a third order miscellany is digital, not physical, we no
longer have to agree on a single framework. Things have their _places_,
not a single place. We get to create our own categories, ones that suit
our way of thinking. Experts can be helpful, but in the age of the
miscellaneous they and their institutions are no longer in charge of our
ideas.

These are big changes, but perhaps the most urgent one is this: Over the
course of the millennia, we've developed sophisticated methods and
processes for developing, communicating and preserving knowledge. We
have major institutions - serious contributors to our culture and our
economy - devoted to those tasks. We're good at it. Now we have to
invent new ways appropriate to the new shape of knowledge. We are doing
so at a pace unparalleled in our history.

Three new strategic principles are emerging, severing the ties between
the way we organize physical objects and ideas.

FILTER ON THE WAY OUT, NOT ON THE WAY IN. A friend of mine who worked at
the Harvard Business Review tells amusing stories about the "slush
pile," the unsolicited manuscripts that arrive every day. Harvard
Business Review is a sober journal of research and ideas, yet people
submit poetry, short stories, and arty photographs. My friend's job was
to go through the slush pile to see what, if anything, was worth passing
along for serious consideration. She was a gatekeeper, a filterer, a job
that makes sense when the economics and physics of paper force us to
make decisions about what knowledge we will publish and thus preserve.
We rely on experts such my friend to spare us from having to wade
through the slush pile on our own.

But, when anyone can publish at the press of a button, the social role
of gatekeepers changes. For example, from the outside, the "blogosphere"
looks like a self-indulgent pool of slush that wouldn't get past the
usual publishing filters. While the economics of publishing ensure that
most blogs indeed wouldn't be let through the gates, the aggregate
value of all the blogs in the "long tail" (to use the term Chris
Anderson made popular in his book of that name) - each perhaps of
interest only to a few people -  is incalculable. This is an inversion
of the old model. In a world of parsimonious access to paper, filters
increase the value of what's available by excluding the slush. But in
the third order, where there's an abundance of access to an abundance
of resources, filtering on the way in _decreases_ the value of that
abundance by ruling out items that might be of great value to a few
people. Filtering on the way out, on the other hand, increases the value
of the abundance by locating what's of value to a particular person at
a particular moment. For example, a young physics professor at McGill
University, Bob Rutledge, started an electronic bulletin board that
posts new findings for any research as soon as it can be summarized.
Rutledge doesn't apply criteria to decide for the reader whether the
research is important enough to be included (though only active,
professional astronomers can register to post to the site). It's up to
each reader to be the filterer. Similarly, the Public Library of
Science's biology journal, a peer-reviewed but free online resource,
started PLoS One in November 2006. "The idea is to take the
editorializing out of the peer review process," says Hemai
Parthasarathy, the managing editor. So long as a paper is "sound," it
will be published. If it's good science, _someone_ may find it useful.
So long as the user has good tools for finding what she needs - and this
is a task many are working on - filtering on the way out vastly
increases our shared potential for knowledge.

PUT EACH LEAF ON AS MANY BRANCHES AS POSSIBLE. In the real world, a leaf
can only hang from one branch. In the first order of organization,
there's no way around that limitation. In the second order, most
cataloging systems have provisions for listing books under more than one
heading, but the physicality of the second order still usually demands
that one branch be picked as the primary one and there is a limit on the
number of secondary listings.

In the third order, however, it's to our advantage to hang information
from as many branches as possible. If you get a new Casio digital camera
to sell in your online store, you'll want to list it under as many
categories as you can think of, including cameras, travel gear, Casio
products, graduation gifts, new items, sale items, and perhaps even
sports equipment. Hanging a leaf on multiple branches makes it more
findable by customers. Unlike in the second order, this doesn't make
your e-store disorganized or messy. It makes it more usable…and more
profitable.

EVERYTHING IS METADATA AND EVERYTHING CAN BE A LABEL. In a store, it's
easy to tell the labels from the goods they label, and in a library the
books and their metadata are kept in separate rooms. But it's not so
clear online. If you can't remember the name of one of Shakespeare's
plays, go to the search box at Google Book, type "Shakespeare tragedy,"
and you'll see a list of all of them. Click on, say, _King Lear_ and you
can read the full text, including the famous line, "How sharper than a
serpent's tooth it is to have a thankless child!" Now suppose you want
to know where the quotation "How sharper than a serpent's tooth" comes
from. Type the phrase into the search box and Google will list _King
Lear_.  Simple, but in the first case you used Shakespeare's name as
metadata to find the contents of a book and in the second you used some
of the contents of the book as metadata to find the author and title. In
the miscellaneous order, the only distinction between metadata and data
is that metadata is what you already know and data is what you're trying
to find out.

In the first two orders of order, we've had to think carefully about
which metadata we'll capture because the physical world limits the
amount of metadata we can make available: A book's catalog card has to
hold far less information than does the book itself. In the third order,
not only can every word in a book count as metadata, so can any of the
sources that link to the book. if we want to help our customers or users
find information, we'll try to make as much of usable as metadata as we
can.

This not only makes sites easier to use, it vastly increases the
leverage of knowledge. Think of what we can do with just the few words
that fit on a second-order card or label. Now that everything in the
connected world can serve as metadata, knowledge is empowered beyond
fathoming. We not only can find what we need based on whatever slight
traces we have in our hand, we can see connections that would have
escaped notice in the first two orders. The power of the miscellaneous
comes directly from the fact that in the third order, everything is
connected and therefore everything is metadata.


GIVE UP CONTROL. Build a tree and you surface information that might
otherwise be hidden, just as Lamarck exposed information left hidden in
Linnaeus' miscellaneous category of worms. But, a big pile of
miscellaneous information contains relationships beyond reckoning. No
one person or group is going to be able to organize it in all the useful
ways, hanging all the leaves on all the branches where they might be
hung. For example, iTunes shows users a branch that pulls together
albums by a particular artist, but the millions of playlists that users
have made there find relationships that the organizers of iTunes could
not possibly have foreseen, from techno versions of children's songs
to tracks played at someone's third wedding. iTunes simply cannot
predict what people are going to be interested in, what a song is going
to mean to them, and what connections they're going to see. Some of the
combinations will be of passing value only to one person, but other
people may find their world changed by how a stranger has pulled
together a set of songs to express a mood, an outlook, or an idea.

That's why it's so powerful to let users mix it up for themselves. Go
into a real world clothing store and try pulling everything in your size
off the racks and into a shopping cart so ou can go through it in an
orderly fashion. After all, that's the rational way to proceed.
Everything that's not your size is just noise, a distraction. Yet,
within ninety seconds you'll be thrown out of the store and firmly asked
not to return. On line, on the other hand, we just naturally expect to
organize digital information our way, through tags, bookmarks,
playlists, and weblogs. And then we add to the information a site
provides us by disagreeing with it in our own reviews. Users are now in
charge of the organization of the information they browse. Of course,
the owners of that information may still want to offer a prebuilt
categorization, but that is no longer the only - or best - one
available. Put simply, the owners of information no longer own the
organization of that information.

Control has already changed hands. The new rules of the information
jungle are in effect, transforming the landscape in which we work, buy,
learn, vote and play.

           * * *

THE CONUNDRUM OF CONTROL

[From Chapter 7, "Social Knowing," pp. 131-133]

In February 2005, Michael Gorman, the president of the American Library
Association, lambasted weblogs in the association’s flagship magazine,
Library Journal:

      A blog is a species of interactive electronic diary
      by means of which the unpublishable, untrammeled by
      editors or the rules of grammar, can communicate their
      thoughts via the web…

      Given the quality of the writing in the blogs I have
      seen, I doubt that many of the Blog People are in the
      habit of sustained reading of complex texts. It is
      entirely possible that their intellectual needs are
      met by an accumulation of random facts and paragraphs.

Some librarians—especially those who were also Blog People—were
outraged. "An example of irresponsible leadership at its worst," wrote
Sarah Houghton on her blog, Librarian in Black. "Excoriating ad hominem
attacks wrapped in academic overspeak," blogged Free Range Librarian
Karen Schneider, adding, “No citations, of course.” The best title of a
blog post had to be "Turkey ALA king" by Michael D. Bates at BatesLine.

Gorman brushed off his critics, citing his “old fashioned belief that,
if one wishes to air one's views and be taken seriously, one should go
through the publishing/editing process.” How fortunate for Gorman that
he heads an organization with its own journal.

But then, Gorman is hardly alone in his skepticism about online sources.
In October of the same year, Philip Bradley, a librarian and Internet
consultant, said in The Guardian that Wikipedia is, theoretically “a
lovely idea,” but “I wouldn’t use it, and I’m not aware of a single
librarian who would.”

Robert McHenry, a former editor-in-chief of the Encyclopedia Britannica,
summed up his analysis of Wikipedia:


      "The user who visits Wikipedia to learn about some
      subject, to confirm some matter of fact, is rather
      in the position of a visitor to a public restroom.
      It may be obviously dirty, so that he knows to exercise
      great care, or it may seem fairly clean, so that he
      may be lulled into a false sense of security. What
      he certainly does not know is who has used the facilities
      before him."

If these experts of the second order sound a bit hysterical, it is
understandable. The change they're facing from the miscellaneous is deep
and real. Authorities have long filtered and organized information for
us, protecting us from what isn't worth our time and helping us find
what we need to give our beliefs a sturdy foundation. But, with the
miscellaneous, it’s all available to us, unfiltered.

More is at stake than how we’re going to organize our libraries.
Businesses have traditionally owned not only their information assets
but also the organization of that information. For some, their business
_is_ the organization of information. The Online Computer Library Center
bought the Dewey Decimal Classification system in 1988 as part of its
acquisition of Forest Press. To protect its trademark, in 2003 the OCLC
sued a New York City hotel with a library theme for numbering its rooms
with Dewey numbers. Westlaw makes a good profit providing the standard
numbering of court cases, applying proprietary metadata to material in
the public domain. But just about every industry that creates or
distributes content - ideas, information or creativity in any form -
exerts control over how that content is organized. The front page of the
newspaper, the selection of movies playing at your local theater, the
order of publicly-available facts in an almanac, the layout of a music
store, and the order of marching bands in the Macy’s Thanksgiving Day
Parade all bring significant value to the companies that control them.

This creates a conundrum for businesses as they enter the third order.
If they don’t allow their users to structure information for themselves,
they’ll lose their patrons. If they do allow patrons to structure
information for themselves, the organizations will lose much of their
authority, power, and control.

The paradox is already resolving itself. Customers, patrons, users and
citizens are not waiting for permission to take finding and organization
information. And we’re doing it not just as individuals. Knowledge—its
content and its organization—is becoming a social act.

----------

And thus ends the longest post to a mailing list ever!

I look forward to the discussion, although the way it's worked out, I'll
be on dialup starting on Sunday or Monday, and thus may be somewhat
laggy in my own participation.


David W.

David Weinberger
Harvard Berkman Center for Internet & Society
self at evident.com
www.JohoTheBlog.com
www.EverythingIsMiscellaneous.com