[iDC] Data Literacy and Cultural Analytics

Wed Oct 12 15:02:49 UTC 2011

Comments inline...

On 10/11/11 11:54 PM, Lev Manovich wrote:
> Andreas
>
>
> my answers are below:
>
>> Personally, I would also find it more
>> challenging and interesting to write the data scrapers that "harvest" image
>> sets from the web or come up with new analysis algorithms that calculate new
>> dimensions for existing images.
> And what are you going to do with the lots of scraped images and lots
> of extracted features? Thats why we create ImagePlot - it takes images
> and their features and visualize relations between images.
>
I don't dispute the merits and creative opportunities ImagePlot creates. 
A visualization (or more generally, transformation) tool is obviously 
needed to make such data collections accessible to the viewer/user. When 
I find some time, I may try to code up a visualization tool myself and 
contribute it to your group.
>
>> Let's say I write an algorithms that
>> determines the entropy (randomness) in an image ... How would you enable
>> someone downloading ImagePlot to apply this to their images?
> ImagePlot takes as its input a directory of images and information
> about their organized in a simple tab delimited file. So a user can
> use any software to extract additional features - you just need then
> to add new columns to a data file.
I've experimented with ImagePlot and know how it works. My point was 
merely, that the prescribed data files are not a good way to _share_ 
such information. Community based operations such as adding new or 
derived dimensions - i.e. a markup a-la wikipedia or augmentation via 
Matlab algorithms - are not part of the design of this tool. You may 
call this a feature request (or critique), which would - if available - 
make ImagePlot-style data explorations more accessible to people and 
increase the quality of the available image data.
> We developed a system which uses Matlab to extract 400 features, and
> we use ImagePlot to visualize the results.
>
>
>
>> Conversely, how
>> would I acquire interesting content sets that span millions of images?
> There is one single solution.
> With Flickr and other social media sites, its pretty simple to
> download large images sets using their API.
> With some other sites, you can use web harvesting programs. For
> instance, we used SiteSucker to download one million manga pages.
>
I understand the opportunities - I have done it commercially in the past 
(see http://ica.blogs.com/en/2006/06/pictures_and_th.html) - but can 
attest that it is not always simple to get "cool" or legal content sets. 
Again a feature request (or critique), would be to add some "cloud 
based" data set storage to the tool to enable sharing of image sets 
amongst ImagePlot users.

Also merely attempting to "harvest" the web may get you into trouble 
(i.e. see the precedences listed on 
http://en.wikipedia.org/wiki/Web_scraping#Legal_issues). I feel that 
this should enter more the into the documentation, because these issues 
are a cornerstone of "Data Literacy" these days.

Thanks,
Andreas
>
>
> On Tue, Oct 11, 2011 at 9:23 PM, Andreas Schiffler
> <aschiffler at ferzkopp.net>  wrote:
>> Hi Lev,
>>
>> aren't the data collections equally (or maybe even more) important than the
>> processing tools?  More and more image content ends up in walled gardens
>> (i.e. the guy who made a Firefox scraper plugin to allow peoples to download
>> their OWN images from their Facebook accounts and got a cease-and-desist
>> letter from FB's lawyers) or has some other funky legal "chain" attached to
>> it which can be used to block or monetize even derived works (i.e. Time
>> Magazine never complained?). Personally, I would also find it more
>> challenging and interesting to write the data scrapers that "harvest" image
>> sets from the web or come up with new analysis algorithms that calculate new
>> dimensions for existing images. Isn't the more interesting problem to solve
>> one of enabling the "community" to get access to content and allow them to
>> markup the data algorithmically. Let's say I write an algorithms that
>> determines the entropy (randomness) in an image ... How would you enable
>> someone downloading ImagePlot to apply this to their images? Conversely, how
>> would I acquire interesting content sets that span millions of images?
>>
>> Thanks,
>> Andreas
>>
>> On 10/2/11 11:34 AM, Lev Manovich wrote:
>>> Greetings
>>>
>>> I am Lev Manovich and my talk at the forthcoming conference will be
>>> called  "Data Literacy and Cultural Analytics"
>>>
>>> Here is what I plan to address:
>>>
>>>
>>>
>>> The joint availability of numerous large data sets on the web and free
>>> tools for data scraping, cleaning, analyzing and visualizing enable
>>> potentially anybody to become a citizen data miner. But how do we
>>> enable this in practice?
>>>
>>> What are the necessary elements of “data literacy”?
>>>
>>> How do we inspire students in traditionally non-quantiative fields
>>> (art history, film and media studies, literary studies, etc.) to start
>>> playing with big data?
>>>
>>> One the limitations of the existing popular data analysis and
>>> visualization tools is that they are designed to work with numbers and
>>> texts – but not images and video. To close this gap, In 2007 we have
>>> established Software Studies Initiative (softwarestudies.com) at
>>> University of California, San Diego. The lab’s focus in on development
>>> of new visualization methods particularly suited for media teaching
>>> and research.
>>>
>>> In my presentation I will show a sample of our projects including
>>> visualization of art, film, animation, video games, magazines, comics,
>>> manga, and graphic design. Our image sets range from 4535 covers of
>>> Time magazine to 320,000 Flickr images from “Art Now” and “Graphic
>>> Design” groups, and one million manga pages.
>>>
>>> In September 2011 we released ImagePlot -  free software tool that
>>> visualizes collections of images and video of any size. I will discuss
>>> how we use ImagePlot in classes with both undergraduate and graduate
>>> students to create collaborative projects which reveal unexpected
>>> cultural trends and also make us question our existing concepts for
>>> understanding visual culture and media. You can download software
>>> here:
>>>
>>>
>>> http://lab.softwarestudies.com/p/imageplot.html
>>>
>>>
>>> best,
>>>
>>> Lev
>>>
>>> ---------------------
>>> Dr. Lev Manovich
>>> Director, Software Studies Initiative, Calit2<softwarestudies.com>
>>> Professor, Visual Arts Department, UCSD<visarts.ucsd.edu>
>>>
>>> email: manovich.lev at gmail.com
>>> lab: softwarestudies.com
>>> www.manovich.net
>>> twitter.com/manovich
>>> www.facebook.com/profile.php?id=668367315
>>> youtube.com/user/softwarestudies
>>>
>>>
>>> Mailing address:
>>> University of California, San Diego,
>>> Lev Manovich, Visual Arts Department,
>>> 9500 Gilman Drive. #0084, La Jolla, CA 92093-0084, U.S.A
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Sep 30, 2011 at 9:19 PM, Trebor Scholz<scholzt at newschool.edu>
>>>   wrote:
>>>> Dear Lev,
>>>>
>>>> All participants introduced their contribution to the summit on the iDC
>>>> mailing list.
>>>>
>>>> Join the discussion by posting to idc at mailman.thing.net
>>>>
>>>> The discussion leading up to MobilityShifts:
>>>>
>>>> May 2011
>>>> Call
>>>> https://lists.thing.net/pipermail/idc/2011-May/004532.html
>>>>
>>>> June 2011
>>>> Public Library Thread and Introductions
>>>> https://lists.thing.net/pipermail/idc/2011-June/thread.html
>>>>
>>>> July 2011
>>>> Public Library Thread continued + The Future of the Humanities  +
>>>> Hackademia as New Structural Transformation of the Public  Sphere
>>>> https://lists.thing.net/pipermail/idc/2011-July/thread.html
>>>>
>>>> August 2011
>>>> Discussion: The Edupunks' Guide + Are We Changing?
>>>> https://lists.thing.net/pipermail/idc/2011-August/thread.html
>>>>
>>>> September 2011
>>>>   From Digital Natives to Digital Outcasts + Can DIY Education be
>>>> Crowdsourced? + The Aims of Education
>>>> https://lists.thing.net/pipermail/idc/2011-September/thread.html
>>>>
>>>> (initiated and moderated by Trebor Scholz)
>>>>
>>>>
>>>> best,
>>>> Trebor
>>>> =
>>>>
>>>> R. Trebor Scholz, PhD
>>>> Director of the conference series The Politics of Digital Culture
>>>> Chair, MobilityShifts
>>>> The New School
>>>> 65 West 11th Street, New York, NY 10011
>>>>
>>>>
>>>>>>> Lev Manovich<manovich.lev at gmail.com>    09/30/11 3:08 PM>>>
>>>> Jennifer
>>>>
>>>> I am very sorry for not being in touch - classes just started at UCSD
>>>> this week.
>>>>
>>>> yes of course I am participating.. I am finalizing the abstract and
>>>> will send it to you later today
>>>>
>>>>
>>>> I was not aware that you are making travel arrangements for the
>>>> participants - so I already booked both flights and the hotel.
>>>> I am arriving Dec 12 and departing Dec 16.
>>>>
>>>> However, its very possible that you have a better rate in the hotel
>>>> you are working with - which one is it, and what is the rate?
>>>>
>>>>
>>>> again, my apologies for not being in touch
>>>>
>>>> best
>>>>
>>>> lev
>>>>
>>>>
>>>>
>>>> On Fri, Sep 30, 2011 at 11:22 AM, Jennifer Conley Darling
>>>> <conleydj at newschool.edu>    wrote:
>>>>> Dear Lev Manovich,
>>>>> I'm following up as we have not gotten a response from you regarding the
>>>>> confirmation of your date&    time, nor a title and description of your
>>>>> talk.
>>>>> We also have not made any flight arrangements for you. I am currently
>>>>> holding a hotel room for three nights (Oct 13-16) in your name and will
>>>>> need
>>>>> to release these at the beginning of next week if you will not be
>>>>> joining
>>>>> us.
>>>>> Please let us know per the below email if you can accommodate the
>>>>> timing.
>>>>> I look forward to your response.
>>>>> Jennifer
>>>>>
>>>>> On Fri, Aug 5, 2011 at 10:16 AM, Jennifer Conley Darling
>>>>> <conleydj at newschool.edu>    wrote:
>>>>>> Dear Lev,
>>>>>> I'm so glad to have you on board for MobilityShifts. The full
>>>>>> conference
>>>>>> runs October 10-16. We have you slated to speak in speaker series on
>>>>>> Friday,
>>>>>> October 14 from 10-12pm in Wollman Hall located at 65 West 11 Street.
>>>>>> If you'd be so kind as to send a brief description and title of your
>>>>>> talk,
>>>>>> we would like to include this in our program, which goes to print late
>>>>>> next
>>>>>> week.
>>>>>> Thank you so much and don't hesitate to reach out with questions.
>>>>>> --
>>>>>> Jennifer Conley Darling
>>>>>> Conference Producer
>>>>>> MobilityShifts: An International Future of Learning Summit
>>>>>> October 10-16, 2011
>>>>>> The New School
>>>>>> 65 West 11 Street
>>>>>> New York, NY 10011
>>>>>>
>>>>>> On Thu, Aug 4, 2011 at 3:18 PM, Lev Manovich<manovich.lev at gmail.com>
>>>>>> wrote:
>>>>>>> Yes I confirmed already - very happy to participate
>>>>>>>
>>>>>>> what dates will be the conference - 14-16 ?
>>>>>>>
>>>>>>> and my talk?
>>>>>>>
>>>>>>>
>>>>>>> best,
>>>>>>>
>>>>>>> Lev
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 4, 2011 at 12:15 PM, Losh, Elizabeth<elosh at mail.ucsd.edu>
>>>>>>> wrote:
>>>>>>>> Hi Jennifer,
>>>>>>>>
>>>>>>>> Last I heard Lev was on board, but I am cc'ing him.
>>>>>>>>
>>>>>>>> Liz
>>>>>>>>
>>>>>>>> Sent from my iPhone and thus more likely to include nonstandard
>>>>>>>> spelling or syntax.
>>>>>>>>
>>>>>>>> On Aug 4, 2011, at 11:23 AM, "Jennifer Conley Darling"
>>>>>>>> <conleydj at newschool.edu
>>>>>>>>   >    wrote:
>>>>>>>>
>>>>>>>>> Hi Liz,
>>>>>>>>>
>>>>>>>>> Trebor had asked me to follow up with you regarding Lev's
>>>>>>>>> participation in MobilityShifts. Have you been able to get
>>>>>>>>> confirmation from him? If so, we would like to begin a direct
>>>>>>>>> communication with Lev regarding location, A/V needs, etc.
>>>>>>>>>
>>>>>>>>> Please let me know where things stand as soon as possible.  Thanks
>>>>>>>>> so much.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Jennifer Conley Darling
>>>>>>>>> Conference Producer
>>>>>>>>> MobilityShifts: An International Future of Learning Summit
>>>>>>>>> October 10-16, 2011
>>>>>>>>> The New School
>>>>>>>>> 65 West 11 Street
>>>>>>>>> New York, NY 10011
>>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Jennifer Conley Darling
>>>>> Conference Producer
>>>>> MobilityShifts: An International Future of Learning Summit
>>>>> October 10-16, 2011
>>>>> The New School
>>>>> 65 West 11 Street
>>>>> New York, NY 10011
>>>>>
>>>>>
>>> _______________________________________________
>>> iDC -- mailing list of the Institute for Distributed Creativity
>>> (distributedcreativity.org)
>>> iDC at mailman.thing.net
>>> https://mailman.thing.net/mailman/listinfo/idc
>>>
>>> List Archive:
>>> http://mailman.thing.net/pipermail/idc/
>>>
>>> iDC Photo Stream:
>>> http://www.flickr.com/photos/tags/idcnetwork/
>>>
>>> RSS feed:
>>> http://rss.gmane.org/gmane.culture.media.idc
>>>
>>> iDC Chat on Facebook:
>>> http://www.facebook.com/group.php?gid=2457237647
>>>
>>> Share relevant URLs on Del.icio.us by adding the tag iDCref
>>>
>>