Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 18 2011


All recipes 30 minutes to cook

The other week we quietly added two tutorials of a new kind to the site, snuck in behind a radical site redesign.

They’re instructive recipes, which anyone with a modicum of programming knowledge should be able to easily follow.

1. Introductory tutorial

For programmers new to ScraperWiki, to a get an idea of what it does.

It runs through the whole cycle of scraping a page, parsing it then outputting the data in a new form. For a simplest possible example.

Available in Ruby, Python and PHP.

2. Views tutorial

Find out how to output data from ScraperWiki in exactly the format you want – i.e. write your own API functions on our servers.

This could be a KML file, an iCal file or a small web application. This tutorial covers the basics of what a ScraperWiki View is.

Available in Ruby, Python and PHP.

Hopefully these tutorials won’t take as long as Jamie Oliver’s recipes to make. Get in touch with feedback and suggestions!

December 03 2010


Views part 1 – Canadian weather stations

(This is the first of two posts announcing ScraperWiki “views”. A new feature that Julian, Richard and Tom worked away and secretly launched a couple of months ago. Once you’ve scraped your data, how can you get it out again in just the form you want?)

Canadian weather stations

Clear Climate Code is a timely project to reimplement the software of climate science academics in nicely structured and commented Python. David Jones has been using ScraperWiki views to find out which areas of the world they don’t have much surface temperature data for, so they can look for more sources.

Take a look at his scraper Canada Climate Sources. If you scroll down, there’s a section “Views using this data from this scraper”. That’s where you can make new views – small pieces of code that output the data the way you want. Think of them as little CGI scripts you can edit in your browser. This is a screenshot of the Canada Weather Station Map view.

It’s a basic Google Map, made for you from a template when you choose “create new view”. But David then edited it, to add conditional code to change the colours and letters on the pins according to the status of the stations.

This is the key powerful thing about ScraperWiki views – even if you start with a standard chart or map, you have the full power of the visualisation APIs you are using, and of HTML, Javascript and CSS, to do more interesting things later.

There’s more about ScraperWiki and the Canada weather stations in the posts Canada and Analysis of Canada Data on the Clear Climate Code blog.

Next week – part 2 will be about how to use views to output your data in the machine readable format that you want.

September 16 2010


What I read today…

March 16 2010


SXSW 2010: Rushkoff’s Ten Commandments

3428784075_72ea756861_mIt’s widely dubbed the geeks spring break but the heaving human mass descending on Austin Texas for another year’s future scoping at South-by-South West is a much broader church. Film makers, artists, journalists, writers and musicians mingle with programmers, developers, gamers and interaction designers for five days of content, camaraderie, context and cocktails. It’s where people meet to discuss the future before they go out and build it. And if you think that might be SxSW over stating its position maybe Doug Rushkoff, speaking at one of the opening sessions could persuade you other wise.

Rushkoff’s stinging attack on contemporary society’s uncritical consumption of all things ‘web’ was delivered to an initiated congregation perhaps looking for more affirmation than conflagration. Program or be Programmed built on Postman’s thinking applying his frequently referenced idea that ‘to a man with a pencil the world looks like a book’ and translates it for the digital age. His prophetic warning came in the form of Ten Commandments reminding SxSW worshippers that binary code demands a much more absolute view of the world than the analogue legacy it is fast replacing. The clue is in the title – ‘Binary’. It’s either a ‘one’ or it’s a ‘zero’. There’s no room for an analogue ‘maybe’ or ‘almost’. Instead, binary code and the programmers using it offer a series of pre-programmed choices requiring a yes or a no answer. Think about the average Facebook profile where users indicate whether they are in ‘in a relationship’ or ‘looking for a relationship’. If they’re ‘religious’ or not. It’s a tick box questionnaire limited by the toolset, and maybe the imaginations of people who use code to deliver a product. They cannot afford to allow anyone to consider the millions of other permutations or lifestyle choices they haven’t been offered. There’s a statistical impossibility in any attempt to present the creation of boxes for the individual who may be ‘pseudo religious but only occasionally depending on the weather and/or mood’.  And so the overly simple tyranny of binary code shapes its masters in the same vein as the old Churchillian quote about us shaping our buildings, which thereafter shape us. Rushkoff cautions us against the mindset of the programmer presenting their machine coded choices as cold rational solutions without alternative. Commandment number four: ‘you may always choose none of the above’.

Here are the rest of Rushkoff’s Ten Commandments to address the biases of digital media for your edification:

  • Time: Thou shalt not always be on.
  • Distance – Thou shalt not do from  a distance what can be done face to face.
  • Scale – Exalt the particular (resist the temptation to ‘scale-up’).
  • Discreet – Thou may’st always choose none of the above.
  • Complexity – Thou shalt not always be right.
  • Corporeal – Thou shalt not be anonymous.
  • Contact – Thou shalt remember the humans.
  • Abstraction – As above not so below.
  • Openness – Thou shalt not steal
  • End Users – Programme or be programmed

If all that sounds a bit too academic SxSW hosts hundreds of practical sessions. A cursory glance through the 256 page programme lists workshops, seminars and keynotes on citizen journalism, augmented reality, design collaboration, Javascript Architecture (whatever that is), iPhone apps and web 2.0 marketing. Among the more interesting titles: What if your phone had five senses? Augmented Reality Games and Women. Neuroscience and Marketing. Exploiting Chaos. Can Wikipedia survive Popular Success and Community Decline?

Then there are the parties, or should that be ‘networking opportunities’ which I suspect is how the marketing people refer to them when they’re talking to their Financial Directors. 17 floors up sipping Martinis and watching Austin’s high-rises dissolve into the night sky is only the start of a nightly ritual lasting long into the wee small hours and playing out in dozens of downtown venues. For the FD’s wondering whether the money they spend hosting these bacchanalian extravaganzas is well spent rest assured it is. But then, as a sharply dressed advertising type person passes me another cocktail I guess I would say that wouldn’t I.

Picture by Designbyfront on flickr

Enhanced by Zemanta
Tags: News Views
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!