Really interesting stuff, John! Definitely agree with you on the "necessary but not sufficient" formulation. <br><br>>>But the issue we’d like to discuss with the list is what a system with
the same goals--ongoing, deep evaluation of complex learning--would look
like if it were designed to work on the same scale as, say, the Khan
Academy. Is peer feedback sufficient to meet those goals? If so,
quality would somehow need to be controlled so that it doesn’t turn into
a stream of YouTube comments, and if not some other method would have
to be used to deal with large volumes of students. <br><br>What strikes me is that there are different types of peers--some peers perhaps more equal than others. In a community of practice model there are fellow beginners, who have one type of feedback to offer, then there are people just ahead of you--like the sophomore, junior, senior to your freshman, who have a different type of feedback (less grounded in immediate understanding of what you're going through and more grounded in knowledge and experience), and then graduate student/TA/professor with a more sophisticated offering still. <br>
<br>One can imagine a scalable system that incentivizes feedback according to the experience and sophistication of the person offering it, and thus its likely value to the user. Maybe it's a "freemium" model where learners give and receive feedback freely as a condition of participation up to a certain level of experience, and the most experienced participants receive other kinds of incentives (even money?) in exchange for offering the most detailed, sophisticated, time-consuming forms of feedback. <br>
I often think back to my summer studying capoeira where the most
experienced students took on more and more responsibilities instructing
the beginners, as an honor--but only the mestre gets paid. <br><br>Of course there are other technological ways of encouraging quality control on a large system that depends for its value on freely offered feedback. These are all over the net. TripAdvisor, Amazon, eBay, Quora, Yelp are all good examples--Yelp in particular, again for the way it incentivizes its best providers of feedback, making them a recognized part of a community, allowing the raters to earn ratings. LinkedIn with its endorsement structure another one to look at.<br>
Maybe you need a system of badges, tags or profile keywords so you can ask a
native Brazilian to read your Portuguese paper or a nationally ranked
chess player to check out your game or someone with a stellar Github
rating to look at your code. <br>a<br><br><br><br><div class="gmail_quote">On Mon, Sep 5, 2011 at 3:35 PM, John Bell <span dir="ltr"><<a href="mailto:john@novomancy.org">john@novomancy.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi all,<br>
<br>
As Jon said, we'd like to think about how community sites and online social mediators can contribute to self education. Sites like Khan Academy, the various Open CourseWare implementations, and many others have done a tremendous job of opening up information access, but--as was mentioned in some recent discussions on this list--that is only a necessary condition for DIY education, not a sufficient one. Once a student finds the information they’re after they have to do something with it and evaluate the result, and this is where most online offerings fail. Unless they are part of an organized class, feedback is generally limited to automated quiz grading (if even that) and thus the course is limited to either easily measured skills or very shallow evaluation of more complex ideas.<br>
<br>
In order to enable DIY education and not just DIY skill acquisition, I think we need to crack the problem of scaling evaluation and discussion to the same level that we’ve scaled information access. DIY students have to find or create support systems to provide that kind of feedback on their own now, leaving them adrift in a sea of ten-minute videos. Without evaluatory feedback students don’t get the chance to iteratively work through ideas that require more depth than simple skills. A site like quora where students can ask questions may be a good start, but it is more useful for in-process guidance than evaluating the application of what has been learned.<br>
<br>
Our approach to this problem at Still Water has been to build a tool for peer evaluation of iteratively created projects called The Pool (<a href="http://pool.newmedia.umaine.edu/" target="_blank">http://pool.newmedia.umaine.edu/</a>). The Pool provides mechanisms for giving both text comments and numeric ratings on in-progress work--and in The Pool, all work is "in-progress" because it doesn’t recognize that a project is ever definitively finished. We based its feedback model on critique instead of grading in an attempt make an online, asynchronous system that goes beyond grading multiple guess quizzes.<br>
<br>
Since its inception The Pool has expanded from one class to multiple classes at the University of Maine, and in the past couple years to a handful of universities, including UC Berkeley, UC Santa Cruz, and USC. So far it's harbored 1200 participants exchanging 9000 reviews. This cross-campus feedback has made for some interesting discoveries about cultural differences between students on different campuses.<br>
<br>
The current implementation of The Pool is not scalable to larger groups of students for some fundamental technical and interface reasons. But the issue we’d like to discuss with the list is what a system with the same goals--ongoing, deep evaluation of complex learning--would look like if it were designed to work on the same scale as, say, the Khan Academy. Is peer feedback sufficient to meet those goals? If so, quality would somehow need to be controlled so that it doesn’t turn into a stream of YouTube comments, and if not some other method would have to be used to deal with large volumes of students. Would the sort of social interaction created by such a system move a site like Khan out of the skill acquisition space and closer to the more rounded goals of academia? If not, what would?<br>
<font color="#888888"><br>
- John<br>
</font><div><div></div><div class="h5"><br>
On Sep 5, 2011, at 12:57 PM, Jon Ippolito wrote:<br>
<br>
> Thanks to Trebor Scholz and Caroline Buck for inviting my Still Water colleague John Bell and me to contribute to this discussion. This week we'd like to flip the Self / Community coin upside down to consider the community underpinnings necessary to promote DIY learning.<br>
><br>
> First our bios:<br>
><br>
> John Bell is a Web developer and data artist based in Maine. He has contributed to the development of The Pool, a system for fostering and documenting distributed creativity in digital arts; released several open-source web authoring tools; and given birth to an artificial intelligence that accidentally committed suicide. Many of his projects focus on trust in online communities and maintaining intellectual integrity in environments where there are few consequences to ignoring it. His work has been featured in Wired online and he presented CodePlay@UMe at Ars Electronica's Electrolobby Kitchen in 2003. He holds an Intermedia MFA from the University of Maine and is currently a research fellow for the University's Still Water lab, working on the Variable Media Network's "Forging the Future" project.<br>
><br>
> I'm an artist, writer, and curator (<a href="http://three.org/ippolito/" target="_blank">http://three.org/ippolito/</a>) who co-directs Still Water with Joline Blais. I see my career goal as building creative networks that can hold up to media monopolies (<a href="http://three.org/openart" target="_blank">http://three.org/openart</a>), accelerated obsolescence (<a href="http://variablemedia.net/" target="_blank">http://variablemedia.net/</a>), and co-optation by academia (<a href="http://thoughtmesh.net/" target="_blank">http://thoughtmesh.net/</a>).<br>
><br>
> With those introductions out of the way, I'm passing the ball to John to frame some questions about whether DIY education can be ramped up to the scale necessary for crowdsourced feedback, without degrading into lectures or mobocracy.<br>
><br>
> Looking forward!<br>
><br>
> jon<br>
> ______________________________<br>
> Still Water--what networks need to thrive.<br>
> <a href="http://still-water.net/" target="_blank">http://still-water.net/</a><br>
><br>
<br>
_______________________________________________<br>
iDC -- mailing list of the Institute for Distributed Creativity (<a href="http://distributedcreativity.org" target="_blank">distributedcreativity.org</a>)<br>
<a href="mailto:iDC@mailman.thing.net">iDC@mailman.thing.net</a><br>
<a href="https://mailman.thing.net/mailman/listinfo/idc" target="_blank">https://mailman.thing.net/mailman/listinfo/idc</a><br>
<br>
List Archive:<br>
<a href="http://mailman.thing.net/pipermail/idc/" target="_blank">http://mailman.thing.net/pipermail/idc/</a><br>
<br>
iDC Photo Stream:<br>
<a href="http://www.flickr.com/photos/tags/idcnetwork/" target="_blank">http://www.flickr.com/photos/tags/idcnetwork/</a><br>
<br>
RSS feed:<br>
<a href="http://rss.gmane.org/gmane.culture.media.idc" target="_blank">http://rss.gmane.org/gmane.culture.media.idc</a><br>
<br>
iDC Chat on Facebook:<br>
<a href="http://www.facebook.com/group.php?gid=2457237647" target="_blank">http://www.facebook.com/group.php?gid=2457237647</a><br>
<br>
Share relevant URLs on <a href="http://Del.icio.us" target="_blank">Del.icio.us</a> by adding the tag iDCref<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><b><span style="color:rgb(51, 204, 255)">New ebook!</span></b><b> </b><a href="http://edupunksguide.org/" target="_blank">The Edupunks' Guide</a><b><br>Fast Company column</b> <a href="http://www.fastcompany.com/user/anya-kamenetz" target="_blank">Life In Beta</a><br>
<b>Tribune Media column</b> <a href="http://www.tmsfeatures.com/columns/business/personal-finance/savings-game/" target="_blank">The Savings Game</a><br><b>Book</b> <a href="http://www.amazon.com/DIY-Edupunks-Edupreneurs-Transformation-Education/dp/1603582347" target="_blank">DIY U: Edupunks, Edupreneurs and the Coming Transformation of Higher Education </a><br>
<b>Blog</b> <a href="http://diyubook.com/" target="_blank">DIYUbook.com </a><br><b>Twitter </b><a href="http://twitter.com/#%21/anya1anya" target="_blank">@Anya1anya</a><br><br>