Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

April 02 2013

14:37

A Kinder, Gentler Introduction to Regular Expressions

I struggle to remember how to put together a pattern search with regular expressions, so I was excited to discover Lea Verou‘s talk from May 2012, which breaks down how to construct them for JavaScript. (What she teaches can be applied to other languages as well.)

Lea is a front-end engineer who works for the World Wide Web Consortium (W3C). She created RegExp Playground so you can experiment with your own regex patterns. She uses it in her talk below.

February 20 2012

04:54

Resources for learning about social media

I have been collecting posts, articles, tutorials and general how-to materials that relate to how journalists use social media. I started about two weeks ago, as I prepare for a workshop in Singapore.

They are curated here: Social Media and Journalists.

The collection is housed at Scoop.it, a curation site that goes a step beyond social bookmarking sites such as Delicious and Diigo, and which privileges text and tagging — rather than visuals (like Pinterest). For this particular project, I’m finding it very useful.

One example of its utility is that I can offer up a link to a subset of the complete collection by using my own tags: see all posts tagged with “Instagram.” This kind of selection is always useful in teaching and training. Unfortunately, you cannot combine tags (e.g., Instagram + howto) to narrow the search results.

I could have chosen Tumblr for this project, but I’m liking the way Scoop.it works. One of its best features is that when you “scoop” a link using the Scoop.it bookmarklet, the Scoop.it interface opens in a one-third-screen vertical overlay (shown in the first screen capture above). This allows me to scroll up and down in the source material, which makes it easy to write my annotations and choose my tags. I don’t have to flip between browser tabs.

The toolbar shown above appears at the bottom of every posted item. It’s fast and easy to edit your posts and to change or add tags. It’s also easy for others to share your posts on a variety of social networks.

A big drawback is that I can’t download or otherwise preserve my collection. If Scoop.it goes bust, I will lose all my work. There is an RSS feed, but the links go only to the Scoop.it posts; there is no link to the source material in the RSS feed. Bummer.

Scoop.it isn’t brand-new — the site launched in November 2011.

July 20 2011

14:42

How to collaborate (or crowdsource) by combining Delicious and Google Docs

RSS girl by Heather Weaver

RSS girl by HeatherWeaver on Flickr

During some training in open data I was doing recently, I ended up explaining (it’s a long story) how to pull a feed from Delicious into a Google Docs spreadsheet. I promised I would put it down online, so: here it is.

In a Google Docs spreadsheet the formula =importfeed will pull information from an RSS feed and put it into that spreadsheet. Titles, links, datestamps and other parts of the feed will each be separated into their own columns.

When combined with Delicious, this can be a useful way to collect together pages that have been bookmarked by a group of people, or any other feed that you want to analyse.

Here’s how you do it:

1. Decide on your tag, network or user

The spreadsheet will pull data from an RSS feed. Delicious provides so many of these that you are spoilt for choice. Here are the main three:

A tag

Used by various people.

Advantages: quick startup – all you need to do is tell people the tag (make sure this is unique, such as ‘unguessable2012′).

Disadvantages: others can hijack the tag – although this can be cleaned from the resulting data.

A network

Consisting of the group of people who are bookmarking:

Advantages: group cannot be infiltrated.

Disadvantages: setup time – may need to create a new account to build the network around.

A user

Created for this purpose:

Advantages: if users are not confident in using Delicious, this can be a useful workaround.

Disadvantages: longer set up time – you’ll need to create a new account, and work out an easy way for it to automatically capture bookmarks from the group. One way is to pull an RSS feed of any mentions on Twitter and use Twitterfeed to auto-tweet them with a hashtag, and then Packrati.us to auto-bookmark all tweeted links (a similar process is detailed here).

The RSS feed for each will be found at the bottom of pages, and is consistently formatted like so:

Delicious.com/tag/unguessable2012

Delicious.com/network/unguessable2012

Delicious.com/unguessable2012

2. Create your spreadsheet

In Google Docs, create a new spreadsheet and in the first cell type the following formula:

=importfeed(“

…adding your RSS feed after the quotation mark, and then this at the end:

“)

So it looks something like this:

=importfeed(“http://feeds.delicious.com/v2/rss/tag/unguessable2012?count=15″)

Now press enter and after a moment the spreadsheet should populate with data from that feed.

You’ll note, however, that at most you will have only 15 rows of data here. That’s because the RSS feed you’ve copied includes that limitation.

If you look at the RSS feed you’ll see an easy clue on how to change this…

So, try editing it so that the count=15 part of that URL reads count=20 instead. You can put a higher number – but Google Docs will limit results to 20 at a time.

3. Collecting contributions

Technically, you’re now all set up. The bigger challenge is, of course, in getting people to contribute. It helps if they can see the results – so think about publishing your spreadsheet.

You’ll also need to make sure that you check it regularly and copy into a backup spreadsheet so you don’t miss results after that top 20.

But if you find it doesn’t work it may be worth thinking of other ways of doing this – for example, with a Google Form, or using =importfeed with the RSS feed for a search on results for a Twitter hashtag containing links (Twitter’s advanced search allows you to limit results accordingly – and all search results come with an RSS feed link like this one)

Of course there are far more powerful ways of doing this which are worth exploring once you’ve understood the basic possibilities.

Doing more with =importfeed

The =importfeed formula has some other elements that we haven’t used.

Another way to do this, for example, is to paste your RSS feed URL into cell A1 and type the following anywhere else:

=importfeed(A1, ”Items Title”, FALSE, 20)

This has 4 parts in the parentheses:

  1. A1 – this points at the URL you just pasted in cell A1, and means that you only have to change what’s in A1 to change the feed being grabbed, rather than having to edit the formula itself
  2. “Items Title” – this is the part of the feed that is being grabbed. If you look in the feed you will see a part that says <item> and within that, an element called <title> – that’s it. You could change this to “Items URL” to get the <URL> part of <title> instead, for example. Or you could just put “Items” and get all 5 parts of each item (title, author, URL, date created, and summary). You can also use “feed” to get information about the feed itself, or “feed URL” or “feed title” or “feed description” to get that single piece of information.
  3. FALSE – this just says whether you want a header row or not. Setting to TRUE will add an extra row saying ‘Title’, for example.
  4. 20 – the number of results you want.

You can see an example spreadsheet with 3 sheets demonstrating different uses of this formula here.

PrintFriendly

April 12 2011

10:25

Tech Tips: Making Sense of JSON Strings – Follow the Structure

Reading through the Online Journalism blog post on Getting full addresses for data from an FOI response (using APIs), the following phrase – relating to the composition of some Google Refine code to parse a JSON string from the Google geocoding API – jumped out at me: “This took a bit of trial and error…”

Why? Two reasons… Firstly, because it demonstrates a “have a go” attitude which you absolutely need to have if you’re going to appropriate technology and turn it to your own purposes. Secondly, because it maybe (or maybe not…) hints at a missed trick or two…

So what trick’s missing?

Here’s an example of the sort of thing you get back from the Google Geocoder:

{ “status”: “OK”, “results”: [ { "types": [ "postal_code" ], “formatted_address”: “Milton Keynes, Buckinghamshire MK7 6AA, UK”, “address_components”: [ { "long_name": "MK7 6AA", "short_name": "MK7 6AA", "types": [ "postal_code" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "locality", "political" ] }, { “long_name”: “Buckinghamshire”, “short_name”: “Buckinghamshire”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “United Kingdom”, “short_name”: “GB”, “types”: [ "country", "political" ] }, { “long_name”: “MK7″, “short_name”: “MK7″, “types”: [ "postal_code_prefix", "postal_code" ] } ], “geometry”: { “location”: { “lat”: 52.0249136, “lng”: -0.7097474 }, “location_type”: “APPROXIMATE”, “viewport”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } }, “bounds”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } } } } ] }

The data represents a Javascript object (JSON = JavaScript Object Notation) and as such has a standard form, a hierarchical form.

Here’s another way of writing the same object code, only this time laid out in a way that reveals the structure of the object:

{
  "status": "OK",
  "results": [ {
    "types": [ "postal_code" ],
    "formatted_address": "Milton Keynes, Buckinghamshire MK7 6AA, UK",
    "address_components": [ {
      "long_name": "MK7 6AA",
      "short_name": "MK7 6AA",
      "types": [ "postal_code" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "locality", "political" ]
    }, {
      "long_name": "Buckinghamshire",
      "short_name": "Buckinghamshire",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "United Kingdom",
      "short_name": "GB",
      "types": [ "country", "political" ]
    }, {
      "long_name": "MK7",
      "short_name": "MK7",
      "types": [ "postal_code_prefix", "postal_code" ]
    } ],
    "geometry": {
      "location": {
        "lat": 52.0249136,
        "lng": -0.7097474
      },
      "location_type": "APPROXIMATE",
      "viewport": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      },
      "bounds": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      }
    }
  } ]
}

Making Sense of the Notation

At its simplest, the structure has the form: {“attribute”:”value”}

If we parse this object into the jsonObject, we can access the value of the attribute as jsonObject.attribute or jsonObject["attribute"]. The first style of notation is called a dot notation.

We can add more attribute:value pairs into the object by separating them with commas: a={“attr”:”val”,”attr2″:”val2″} and address them (that is, refer to them) uniquely: a.attr, for example, or a["attr2"].

Try it out for yourself… Copy and past the following into your browser address bar (where the URL goes) and hit return (i.e. “go to” that “location”):

javascript:a={"attr":"val","attr2":"val2"}; alert(a.attr);alert(a["attr2"])

(As an aside, what might you learn from this? Firstly, you can “run” javascript in the browser via the location bar. Secondly, the javascript command alert() pops up an alert box:-)

Note that the value of an attribute might be another object.

obj={ attrWithObjectValue: { “childObjAttr”:”foo” } }

Another thing we can see in the Google geocoder JSON code are square brackets. These define an array (one might also think of it as an ordered list). Items in the list are address numerically. So for example, given:

arr[ "item1", "item2", "item3" ]

we can locate “item1″ as arr[0] and “item3″ as arr[2]. (Note: the index count in the square brackets starts at 0.) Try it in the browser… (for example, javascript:list=["apples","bananas","pears"]; alert( list[1] );).

Arrays can contain objects too:

list=[ "item1", {"innerObjectAttr":"innerObjVal" } ]

Can you guess how to get to the innerObjVal? Try this in the browser location bar:

javascript: list=[ "item1", { "innerObjectAttr":"innerObjVal" } ]; alert( list[1].innerObjectAttr )

Making Life Easier

Hopefully, you’ll now have a sense that there’s structure in a JSON object, and that that (sic) structure is what we rely on if we want to cut down on the “trial an error” when parsing such things. To make life easier, we can also use “tree widgets” to display the hierarchical JSON object in a way that makes it far easier to see how to construct the dotted path that leads to the data value we want.

A tool I have appropriated for previewing JSON objects is Yahoo Pipes. Rather than necessarily using Pipes to build anything, I simply make use of it as a JSON viewer, loading JSON into the pipe from a URL via the Fetch Data block, and then previewing the result:

Another tool (and one I’ve just discovered) is an Air application called JSON-Pad. You can paste in JSON code, or pull it in from a URL, and then preview it again via a tree widget:

Clicking on one of the results in the tree widget provides a crib to the path…

Summary

Getting to grips with writing addresses into JSON objects helps if you have some idea of the structure of a JSON object. Tree viewers make the structure of an object explicit. By walking down the tree to the part of it you want, and “dotting” together* the nodes/attributes you select as you do so, you can quickly and easily construct the path you need.

* If the JSON attributes have spaces or non-alphanumeric characters in them, use the obj["attr"] notation rather than the dotted obj.attr notation…

PS Via my feeds today, though something I had bookmarked already, this Data Converter tool may be helpful in going the other way… (Disclaimer: I haven’t tried using it…)

If you know of any other related tools, please feel free to post a link to them in the comments:-)


10:25

Tech Tips: Making Sense of JSON Strings – Follow the Structure

Reading through the Online Journalism blog post on Getting full addresses for data from an FOI response (using APIs), the following phrase – relating to the composition of some Google Refine code to parse a JSON string from the Google geocoding API – jumped out at me: “This took a bit of trial and error…”

Why? Two reasons… Firstly, because it demonstrates a “have a go” attitude which you absolutely need to have if you’re going to appropriate technology and turn it to your own purposes. Secondly, because it maybe (or maybe not…) hints at a missed trick or two…

So what trick’s missing?

Here’s an example of the sort of thing you get back from the Google Geocoder:

{ “status”: “OK”, “results”: [ { "types": [ "postal_code" ], “formatted_address”: “Milton Keynes, Buckinghamshire MK7 6AA, UK”, “address_components”: [ { "long_name": "MK7 6AA", "short_name": "MK7 6AA", "types": [ "postal_code" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "locality", "political" ] }, { “long_name”: “Buckinghamshire”, “short_name”: “Buckinghamshire”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “United Kingdom”, “short_name”: “GB”, “types”: [ "country", "political" ] }, { “long_name”: “MK7″, “short_name”: “MK7″, “types”: [ "postal_code_prefix", "postal_code" ] } ], “geometry”: { “location”: { “lat”: 52.0249136, “lng”: -0.7097474 }, “location_type”: “APPROXIMATE”, “viewport”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } }, “bounds”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } } } } ] }

The data represents a Javascript object (JSON = JavaScript Object Notation) and as such has a standard form, a hierarchical form.

Here’s another way of writing the same object code, only this time laid out in a way that reveals the structure of the object:

{
  "status": "OK",
  "results": [ {
    "types": [ "postal_code" ],
    "formatted_address": "Milton Keynes, Buckinghamshire MK7 6AA, UK",
    "address_components": [ {
      "long_name": "MK7 6AA",
      "short_name": "MK7 6AA",
      "types": [ "postal_code" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "locality", "political" ]
    }, {
      "long_name": "Buckinghamshire",
      "short_name": "Buckinghamshire",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "United Kingdom",
      "short_name": "GB",
      "types": [ "country", "political" ]
    }, {
      "long_name": "MK7",
      "short_name": "MK7",
      "types": [ "postal_code_prefix", "postal_code" ]
    } ],
    "geometry": {
      "location": {
        "lat": 52.0249136,
        "lng": -0.7097474
      },
      "location_type": "APPROXIMATE",
      "viewport": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      },
      "bounds": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      }
    }
  } ]
}

Making Sense of the Notation

At its simplest, the structure has the form: {“attribute”:”value”}

If we parse this object into the jsonObject, we can access the value of the attribute as jsonObject.attribute or jsonObject["attribute"]. The first style of notation is called a dot notation.

We can add more attribute:value pairs into the object by separating them with commas: a={“attr”:”val”,”attr2″:”val2″} and address them (that is, refer to them) uniquely: a.attr, for example, or a["attr2"].

Try it out for yourself… Copy and past the following into your browser address bar (where the URL goes) and hit return (i.e. “go to” that “location”):

javascript:a={"attr":"val","attr2":"val2"}; alert(a.attr);alert(a["attr2"])

(As an aside, what might you learn from this? Firstly, you can “run” javascript in the browser via the location bar. Secondly, the javascript command alert() pops up an alert box:-)

Note that the value of an attribute might be another object.

obj={ attrWithObjectValue: { “childObjAttr”:”foo” } }

Another thing we can see in the Google geocoder JSON code are square brackets. These define an array (one might also think of it as an ordered list). Items in the list are address numerically. So for example, given:

arr[ "item1", "item2", "item3" ]

we can locate “item1″ as arr[0] and “item3″ as arr[2]. (Note: the index count in the square brackets starts at 0.) Try it in the browser… (for example, javascript:list=["apples","bananas","pears"]; alert( list[1] );).

Arrays can contain objects too:

list=[ "item1", {"innerObjectAttr":"innerObjVal" } ]

Can you guess how to get to the innerObjVal? Try this in the browser location bar:

javascript: list=[ "item1", { "innerObjectAttr":"innerObjVal" } ]; alert( list[1].innerObjectAttr )

Making Life Easier

Hopefully, you’ll now have a sense that there’s structure in a JSON object, and that that (sic) structure is what we rely on if we want to cut down on the “trial an error” when parsing such things. To make life easier, we can also use “tree widgets” to display the hierarchical JSON object in a way that makes it far easier to see how to construct the dotted path that leads to the data value we want.

A tool I have appropriated for previewing JSON objects is Yahoo Pipes. Rather than necessarily using Pipes to build anything, I simply make use of it as a JSON viewer, loading JSON into the pipe from a URL via the Fetch Data block, and then previewing the result:

Another tool (and one I’ve just discovered) is an Air application called JSON-Pad. You can paste in JSON code, or pull it in from a URL, and then preview it again via a tree widget:

Clicking on one of the results in the tree widget provides a crib to the path…

Summary

Getting to grips with writing addresses into JSON objects helps if you have some idea of the structure of a JSON object. Tree viewers make the structure of an object explicit. By walking down the tree to the part of it you want, and “dotting” together* the nodes/attributes you select as you do so, you can quickly and easily construct the path you need.

* If the JSON attributes have spaces or non-alphanumeric characters in them, use the obj["attr"] notation rather than the dotted obj.attr notation…

PS Via my feeds today, though something I had bookmarked already, this Data Converter tool may be helpful in going the other way… (Disclaimer: I haven’t tried using it…)

If you know of any other related tools, please feel free to post a link to them in the comments:-)


10:25

Tech Tips: Making Sense of JSON Strings – Follow the Structure

Reading through the Online Journalism blog post on Getting full addresses for data from an FOI response (using APIs), the following phrase – relating to the composition of some Google Refine code to parse a JSON string from the Google geocoding API – jumped out at me: “This took a bit of trial and error…”

Why? Two reasons… Firstly, because it demonstrates a “have a go” attitude which you absolutely need to have if you’re going to appropriate technology and turn it to your own purposes. Secondly, because it maybe (or maybe not…) hints at a missed trick or two…

So what trick’s missing?

Here’s an example of the sort of thing you get back from the Google Geocoder:

{ “status”: “OK”, “results”: [ { "types": [ "postal_code" ], “formatted_address”: “Milton Keynes, Buckinghamshire MK7 6AA, UK”, “address_components”: [ { "long_name": "MK7 6AA", "short_name": "MK7 6AA", "types": [ "postal_code" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "locality", "political" ] }, { “long_name”: “Buckinghamshire”, “short_name”: “Buckinghamshire”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “Milton Keynes”, “short_name”: “Milton Keynes”, “types”: [ "administrative_area_level_2", "political" ] }, { “long_name”: “United Kingdom”, “short_name”: “GB”, “types”: [ "country", "political" ] }, { “long_name”: “MK7″, “short_name”: “MK7″, “types”: [ "postal_code_prefix", "postal_code" ] } ], “geometry”: { “location”: { “lat”: 52.0249136, “lng”: -0.7097474 }, “location_type”: “APPROXIMATE”, “viewport”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } }, “bounds”: { “southwest”: { “lat”: 52.0193722, “lng”: -0.7161451 }, “northeast”: { “lat”: 52.0300728, “lng”: -0.6977000 } } } } ] }

The data represents a Javascript object (JSON = JavaScript Object Notation) and as such has a standard form, a hierarchical form.

Here’s another way of writing the same object code, only this time laid out in a way that reveals the structure of the object:

{
  "status": "OK",
  "results": [ {
    "types": [ "postal_code" ],
    "formatted_address": "Milton Keynes, Buckinghamshire MK7 6AA, UK",
    "address_components": [ {
      "long_name": "MK7 6AA",
      "short_name": "MK7 6AA",
      "types": [ "postal_code" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "locality", "political" ]
    }, {
      "long_name": "Buckinghamshire",
      "short_name": "Buckinghamshire",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "Milton Keynes",
      "short_name": "Milton Keynes",
      "types": [ "administrative_area_level_2", "political" ]
    }, {
      "long_name": "United Kingdom",
      "short_name": "GB",
      "types": [ "country", "political" ]
    }, {
      "long_name": "MK7",
      "short_name": "MK7",
      "types": [ "postal_code_prefix", "postal_code" ]
    } ],
    "geometry": {
      "location": {
        "lat": 52.0249136,
        "lng": -0.7097474
      },
      "location_type": "APPROXIMATE",
      "viewport": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      },
      "bounds": {
        "southwest": {
          "lat": 52.0193722,
          "lng": -0.7161451
        },
        "northeast": {
          "lat": 52.0300728,
          "lng": -0.6977000
        }
      }
    }
  } ]
}

Making Sense of the Notation

At its simplest, the structure has the form: {“attribute”:”value”}

If we parse this object into the jsonObject, we can access the value of the attribute as jsonObject.attribute or jsonObject["attribute"]. The first style of notation is called a dot notation.

We can add more attribute:value pairs into the object by separating them with commas: a={“attr”:”val”,”attr2″:”val2″} and address them (that is, refer to them) uniquely: a.attr, for example, or a["attr2"].

Try it out for yourself… Copy and past the following into your browser address bar (where the URL goes) and hit return (i.e. “go to” that “location”):

javascript:a={"attr":"val","attr2":"val2"}; alert(a.attr);alert(a["attr2"])

(As an aside, what might you learn from this? Firstly, you can “run” javascript in the browser via the location bar. Secondly, the javascript command alert() pops up an alert box:-)

Note that the value of an attribute might be another object.

obj={ attrWithObjectValue: { “childObjAttr”:”foo” } }

Another thing we can see in the Google geocoder JSON code are square brackets. These define an array (one might also think of it as an ordered list). Items in the list are address numerically. So for example, given:

arr[ "item1", "item2", "item3" ]

we can locate “item1″ as arr[0] and “item3″ as arr[2]. (Note: the index count in the square brackets starts at 0.) Try it in the browser… (for example, javascript:list=["apples","bananas","pears"]; alert( list[1] );).

Arrays can contain objects too:

list=[ "item1", {"innerObjectAttr":"innerObjVal" } ]

Can you guess how to get to the innerObjVal? Try this in the browser location bar:

javascript: list=[ "item1", { "innerObjectAttr":"innerObjVal" } ]; alert( list[1].innerObjectAttr )

Making Life Easier

Hopefully, you’ll now have a sense that there’s structure in a JSON object, and that that (sic) structure is what we rely on if we want to cut down on the “trial an error” when parsing such things. To make life easier, we can also use “tree widgets” to display the hierarchical JSON object in a way that makes it far easier to see how to construct the dotted path that leads to the data value we want.

A tool I have appropriated for previewing JSON objects is Yahoo Pipes. Rather than necessarily using Pipes to build anything, I simply make use of it as a JSON viewer, loading JSON into the pipe from a URL via the Fetch Data block, and then previewing the result:

Another tool (and one I’ve just discovered) is an Air application called JSON-Pad. You can paste in JSON code, or pull it in from a URL, and then preview it again via a tree widget:

Clicking on one of the results in the tree widget provides a crib to the path…

Summary

Getting to grips with writing addresses into JSON objects helps if you have some idea of the structure of a JSON object. Tree viewers make the structure of an object explicit. By walking down the tree to the part of it you want, and “dotting” together* the nodes/attributes you select as you do so, you can quickly and easily construct the path you need.

* If the JSON attributes have spaces or non-alphanumeric characters in them, use the obj["attr"] notation rather than the dotted obj.attr notation…

PS Via my feeds today, though something I had bookmarked already, this Data Converter tool may be helpful in going the other way… (Disclaimer: I haven’t tried using it…)

If you know of any other related tools, please feel free to post a link to them in the comments:-)


February 07 2011

16:56

MediaStorm Guide to Crossfades

In my last post, 10 More Ways to Improve Your Multimedia Right Now, I wrote:

Delete all dissolves between images…The eye sees cuts. When we look from one object to another, we see a blink. We don’t see one object then dissolve to another.

A reader responded with a comparison: “Our eyes don’t see shallow depth-of-field [either], but that doesn’t mean we shouldn’t use shallow DOF when appropriate.”

I agree.

So before I discuss when it’s appropriate to dissolve between pictures, let’s quickly revisit why doing so usually doesn’t work.

The problem is that crossfades create an unexpected middle image. In most cases, this intermediate picture, a combination of two hopefully strong ones, is both messy and confusing. There’s no particular meaning to be gleaned from this superimposition. Now, repeat this between every image over the course of a three- to five-minute project, and it’s not hard to see how exhausting it can become. In short, it’s a distraction. And anything that distracts from your story needs to be removed.

I’m not steadfast about most rules, but this one I think is pretty solid: the number one way to improve your multimedia right now is to not make crossfades your default transition. Use a simple cut as your default instead.

But back to the question, when do dissolves actually work?

I think the answer is when there’s some commonality between the two images. That is, some portion of the images stays the same so that when you do dissolve, only the most important part of the frame changes. The rest stays the same.

At the 9:09 mark of The Marlboro Marine by Luis Sinco, James Blake Miller is standing at a window while his wife, Jessica, sits on a mattress on the floor. As Miller describes the challenges he faces, he says of his wife, “And before I put her through that, I’d rather be without her.” The image then dissolves to the same scene, Miller still at the window but now he’s alone.

Producer Chad Stevens successfully uses a dissolve here to great effect. It works because the majority of the image stays the same; the dissolve highlights just part of the frame, Miller’s wife as she literally disappears. Combine the pictures with the dialogue, “I’d rather be without her,” and you have a seamless match between technique and story. It’s the kind of harmony we should all strive towards.

A second example can also be found in The Marlboro Marine at 9:30. Here a dissolve is used to emulate a rack focus. Miller is at the back of the frame while the motorcycle he’s working on is in the front, out of focus. A crossfade changes the viewers focus, shifting Miller out of focus to detail the bike. Again, the frame stays largely the same. All that changes is the focal point of the image.

So in summary, to effectively use dissolves between images, most of the image should remain constant and the crossfade should be used to show a change in time or to shift where the viewer is meant to focus. But, like any technique that draws attention to itself, use it sparingly. Once or twice per project is usually more than enough. Ideally, you want dissolves to be seen as a surprise, not a given.

Finally, if you can think of other effective ways to use dissolves, by all means, add your thoughts to the comments below.

November 30 2010

16:15

MediaStorm’s 10 More Ways to Improve Your Multimedia Right Now

As a followup to a previous post, here are ten more ways to improve your work right now, no matter how challenging your original assets may be.

Make edits with purpose. Always ask why you are making an edit at a particular place. Is the cut motivated by action? A musical beat? A pause in narration? If you don’t have a reason, you need to find a new location for your edit. Every edit must be motivated.

When editing your visuals, don’t cut in the middle of a word. Doing so is confusing. Edit between words, or even better, edit according to written grammar: at a comma, a period, or to emphasize a word. Cutting after words like because and however is also effective.

Edit rhythmically. Make the first cut at the beginning of a spoken phrase. Time the first phrase so it ends right before a musical beat. Cut to another a second image on the musical beat. Pause a few frames before completing the audio under this second. It’s easier to understand once you see the rhythm in action. Check out the “Town Bar” section of Driftless at the 1:37 mark.

Limit the number of times your interview subject is on-screen. There are three main reasons to show your subject on-screen: 1) To introduce someone so the audience knows who is talking. 2) When the subject is expressing emotion that you want your audience to see. 3) As filler, i.e., when you have nothing else to use as coverage. Avoid situation three whenever possible. For an example of an emotional response, see the end of Kingsley’s Crossing.

Don’t start your project with text. The first 10 seconds of a project are crucial. It’s where you audience decides whether they trust you enough to stick around. Starting with text says that your work can’t sustain itself without first reading some background information. That’s not dramatic. There’s nothing wrong with using text, just try to avoid it first thing.

Make sure your text slides are long enough to comprehend. You should be able to read them to yourself at least twice. If not, lengthen them.

If you need to use explanatory text, don’t clump it together in a paragraph. Paragraphs are for print. Show one sentence at a time. If you need a second one, wait until you think the viewer has finished the first to bring the second on screen.

Choose your fonts wisely. Fonts are like attire. Pick one that best represents the occasion. Of the thousands of fonts available to you, which one best represents the spirit and mood of your work? Use that one.

Always lead with your strongest images. You may instinctively want to save your best work for the end. But if you don’t grab the viewers’ attention quickly, they won’t make it to the end. See the beginning of Marcus Bleasdale’s Rape of a Nation for an example.

Delete all dissolves between images. I’ve mentioned this one before but it’s worth repeating. The eye sees cuts. When we look from one object to another, we see a blink. We don’t see one object then dissolve to another. Remove all of your image dissolves and your work will improve immediately. For more on the eye and its relationship to editing, see Walter Murch’s excellent book In the Blink of an Eye.

Please add your own tips and tricks in the comments below.

October 18 2010

18:18

Best advice for Soundslides

I was asking around recently, among my friends at other j-schools who teach photojournalism. Yes, they are still teaching Soundslides. The No. 1 reason is almost unanimous: It’s a great transition from making stills to making video. I think it also helps — a lot — with teaching storytelling.

Right now I’m in the midst of a four-week module where I teach green young journalism students to tell a story with Soundslides. Fortunately, they’ve just finished four weeks of gathering and editing audio. Unfortunately, most of them have no experience with photojournalism.

Gathering Audio. Part 2: A Practical Guide. Brian Storm and Jim Seida wrote this guide years ago, and I think it’s still the best. I was just re-reading it earlier today, and man, it rocks. It’s 4,000 words, or about 10 pages single-spaced, and I would bet most of my students don’t take the time to read it — even though I assign it every semester. What a pity. It’s like gold.

So I’m blogging it here in case you’ve never read it. Or maybe you read it a long time ago and forgot how great it is.

Which should I work on first, pictures or sound?

That depends. If there’s sound that I think might be gone in a few minutes, I’ll probably break out my MiniDisc and start recording. If the light is perfect but fading, I’ll most likely make pictures first.

There’s no right way to do it, and there’s always a tradeoff. You have to accept the fact that when you are recording, you’ll miss some great images and when you are shooting you’ll miss some wonderful sound. I’ve tried doing both at once, it doesn’t work very well. Getting good sound takes just as much skill, energy and focus as getting good pictures; it’s tough to do both things at the same time.

– Meredith Birkett, Special Projects Multimedia Producer, MSNBC.com

That’s just a taste. Ha, we don’t use MiniDiscs any more (thank God!), but all the advice still fits. There’s lots more, just as good as that bit.

October 15 2010

13:54

Storytelling 101 with ‘The Annoying Orange’

Let’s look at storytelling again. I always use this diagram to show students how to structure a very short video story or audio slideshow (120 seconds):

Recently a student showed me this video, and I noticed that at about 1 min. 30 sec., it illustrates the story arc perfectly!

The opening has elements that grab our attention (like any good opening): While we see and read the title (“The Annoying Orange”) at the 3-second mark, we hear a voice saying, “Hey, Apple!” This raises as many as three questions: Who is speaking? What is “Apple”? Why is this story called “The Annoying Orange”?

At 6 seconds, two of our questions are answered when we see the orange; 3 seconds later, we see the apple. Take note of how fast our questions are answered.

At 20 seconds, the orange makes a lame joke, answering another question: “What does he want?”

The orange continues his taunts, each one lasting only a few seconds. The apple sounds more and more annoyed.We are traveling in a straight, unbroken line up to the climax.

At about 50 seconds, we get a somewhat different visual from the orange that works to stave off boredom for the viewer.

Around 1:06, tension is introduced as the orange tries to keep quiet for 3 seconds, at the apple’s request.

At 1:12, the climax begins! (Cut to a great reaction shot at 1:14, and another one at 1:17.)

At 1:24, the resolution. At 1:28, the end.

July 15 2010

15:34

MediaStorm’s Guide to Using Apple’s ProRes 422 Codec

One of the questions I’m asked most frequently about Final Cut is, “When do I use Apple’s ProRes 422 codec?”

To tackle this question it’s first necessary to understand a few things about codecs.

Shooting video is a very intensive digital capture process. It requires cameras to capture lots of information in a short amount of time. To handle so much raw data, most cameras need to compress what they capture. A codec is essentially a compression scheme, a way to encapsulate so much material into a containable format.

Standard DV footage, for instance, uses a compression scheme referred to as the DV codec. Similarly, HD footage — 1080i60, 1080p, etc. — uses the HDV codec.

Codec takes its name from “encoder” and “decoder” since your computer must now decode the encoded file during playback.

When you create a new sequence in Final Cut, you are building what will become a new movie. Thus, you need to define what kind of codec you’ll be using when it’s played back. This is where you need to decide if ProRes 422 is the proper choice.

The answer is actually quite simple.

If you will be using only one kind of video in your project, HDV 1080p for example, then you should use 1080p’s native codec, HDV, for your sequence. If you’re using only DV footage, set your sequence codec to DV.

On the other hand, if your project includes more than one format — both DV and HDV material for example — then you’ll need to set your FCP sequence codec to ProRes 422.

The ProRes 422 codec is so powerful that it can contain two or more other codecs within it, without requiring extra rendering. It’s a truly remarkable ability.

For a technical description of how to set up your sequence’s codec, see Tips from the MediaStorm Final Cut Pro workflow.

In Final Cut 7, the latest version, Apple introduced three new flavors of 422. About 90 percent of the time, you’ll use the standard ProRes 422. But if you’d like more information on the other options, see Final Cut Pro 7. Expanded ProRes Family.

Finally, if you’re unsure of your file’s codec, open it in QuickTime, then use the shortcut key command-I to open the application’s Movie Inspector window.

There you’ll see all kinds of metadata including the clip’s length and file size as well as the codec it uses.

April 24 2010

21:24

New Flash journalism tutorials

I would call this past 12 months “the year I re-learned ActionScript.” After a semester of teaching Adobe Flash CS4 and ActionScript 3.0 to journalism/design students, I’ve got a better handle on both. I’m not too worried about CS5 changing too much — CS4 was the radical, once-in-a-decade (I hope) reworking of the foundations of the Flash application.

So as to share the wealth, I organized many of the tutorial files I created for my students and put them online here: How to Do Stuff in Flash CS4 (AS3).

This is not a comprehensive, how-to-do-everything-in-Flash set of lessons. The students also had a textbook. For further details, see Updating Flash Journalism (Dec. 6, 2009) and Updating Flash Journalism (Part 2).

The tutorials and Flash files are free to download and use for any non-commercial purpose.

April 08 2010

17:36

MediaStorm’s Guide to Backing Up

There are two types of people, the saying goes, those whose hard drives have already failed and those whose will.

That’s why it’s important to have redundancy. For every drive you use, you should always have a backup.

At MediaStorm, we keep our system software–the OS and all applications including Final Cut Pro–on a drive named A. The media for our projects reside on a B drive. We have a duplicate of each that is backed up religiously.

A gets backed up to A_backup; B to B_backup.

For that task we use SuperDuper! from shirt-pocket.com. The application copies the entirety of the drive to it’s respective clone. In other words, you are mirroring the whole drive, not just backing up specific folders.

SuperDuper’s! one window interface is quite simple.

Simply choose the drive you’d like to back up from the Copy pulldown menu and the backup destination from the to: menu.

Choose Backup – all files from the using pulldown menu.

SuperDuper!

At the bottom of the window, selects the Options… button.

SuperDuper! 2

Choose Smart Update from the During Copy menu. This option is the fastest. According to the SuperDuper! manual, smart updates will update any files on your clone drive, new or old, that do not match the primary drive. It will also remove any files on your clone that no longer exist on the original.

Finally, at the bottom of the window, SuperDuper! offers several options once the backup is complete. Shut Down Computer is a useful choice.

Since we generally run the application at the end of the day, this option allows us to start the backup right before we leave. Just make sure you are logged out of all other applications first.

Backing up is essential. The best way to make sure it’s part of your daily workflow is to find a simple solution like SuperDuper! Until you’ve made using it a habit, leave yourself a Stickie note so it’s the last thing you see before shutting down.

Drives are going to fail on you, it’s just a matter of time. The question is, when that happens will you have a backup ready?

January 21 2010

18:14

Mastering Multimedia useful tips roundup


Many of may old posts that deal with tips about how to do video storytelling and audio slideshows get linked on a lot of blogs used by college professors who teach digital media classes. Most of these posts are buried amongst my pontifications about the changes facing the newspaper industry. So for anyone interested,  here is a roundup of my best multimedia suggestions and useful tip posts in one place…

How to make your audio slideshows better

Great audio starts in the field

How best to approach a video story

Sequencing: The foundation of video storytelling

How to make your video editing easier

Get creative with your video camera

Opening your video: How not to lose viewers

Random Final Cut tip: Lower thirds titles

What we can learn from TV new shooters

January 20 2010

16:01

Updating Flash Journalism (Part 2)

The other day I received an e-mail from someone with a programming background who’s interested in learning how to build journalism packages in Flash. He asked how to get started and whether I was planning to release a new edition of my 2005 book Flash Journalism: How to Create Multimedia News Packages.

First I directed him to my December 2009 post about why I will not be updating my book.

I am recommending Adobe Flash CS4 Professional Classroom in a Book. It’s not directed specifically at journalists or news graphics reporters, but it’s easy to follow for the most part.

Then I gave him this outline of what he needs to learn:

  1. Button scripting (for navigation through the package): Adobe Flash CS4 Professional Classroom in a Book, Lesson 6; see also AS3 Buttons Tutorial
  2. Loading external content dynamically: Adobe Flash CS4 Professional Classroom in a Book, Lesson 9
  3. How to optimize images in Flash (Bitmap Properties):  Imported Bitmaps
  4. How to load and control external MP3s: Using Sound in ActionScript 3
  5. How to load and control video: Adobe Flash CS4 Professional Classroom in a Book, Lesson 7 (starting on page 252)
  6. ActionScript 3 and XML loading/controls (XML works awesomely well with AS3): I have built a tutorial for this that is meant to be used in conjunction with the files and the exercise in Adobe Flash CS4 Professional Classroom in a Book, Lesson 8 (download the files; 234 KB). Please note that the exercise will not make sense without the book!

Now, after you’ve got all that under your belt, you will need to spend some time learning how to use the Bandwidth Profiler (Adobe Flash CS4 Professional Classroom in a Book, Lesson 10) to make sure no one can accuse you of building heavy (overly large) Flash files. Heavy Flash files are NOT an indicator that Flash is bad; they simply show that the person who built the files didn’t know how to do it right!

If someone tells you that Flash graphics do not show up in Google or Yahoo! searches — that is incorrect.

If someone tells you that SWF is a proprietary file format, or that SWFs can be created only with Adobe software applications, that is also incorrect.

You should also learn how to use SWFObject to embed your Flash files (SWFs) in regular Web pages.

December 06 2009

21:44

Updating Flash Journalism

This post is for anyone who teaches Adobe Flash to journalism students. It might also be useful for journalists (especially graphics reporters) who are learning Flash right now, or who might be intending to learn Flash over the holidays or sometime soon.

In 2004 I wrote a book titled Flash Journalism: How to Create Multimedia News Packages. It was published in early 2005 by Focal Press/Elsevier. It sold pretty well. It was well received among working journalists.

The book is now outdated. If you’re using Flash CS4, you cannot use the book. At all. It’s okay to use it with CS3, but only if you do everything in ActionScript 2.0. But I do not recommend that. You should be using CS4 and ActionScript 3.0.

I’m not going to update the book. There are a bunch of reasons, but to make a long explanation quite short, it takes far too many hours to write such a book, which includes creating and testing all the example files to be used in Flash. It’s a matter of time. I just don’t have that kind of time available.

Losing my book puts me in a tough spot as a journalism educator. I teach an advanced Web course each spring, and 10 of the 15 weeks are spent on Flash. When people compliment me on the clarity of my Flash book, I tell them that every lesson in the book was battle-tested on real, live journalism students. That’s a fact.

So now I’m left without a book. Well, I gave that a lot of thought last spring, when I knew I was teaching with my own book for the last time. Our computer labs did not get CS4 until summer 2009, so I was able to use my book last spring. And yes, I taught those students ActionScript 2.0 (with a wee bit of AS3 thrown in here and there). Quite frankly, I was  still learning AS3 last spring (you never really stop learning ActionScript). And I had no access at all to CS4 until late in the spring semester.

A New Book and a New Syllabus

What to do? Well, first I had to find a book. The good news is, I’m pretty happy with the book I’m requiring my students to use this spring: Adobe Flash CS4 Professional Classroom in a Book. It’s far from perfect, but it’s a damn sight better than the Flash books that were out there in 2003.

Second, I have to reconfigure two-thirds of my syllabus, because I can’t teach the 10 weeks of Flash the same way I used to — not only am I using a different book, but Flash has changed so much in CS4, I have to adjust the way I teach it. So that’s what I’m in the middle of right now. Getting ready for spring and my Flash class.

And why should you care? Because I’m going to share, that’s why.

The syllabus is not finished yet, but I can tell you about the first four weeks of the 10. And I’ve been busy making some supplemental materials for my students, so I’ll link to those here.

One really good thing about the CS4 Adobe Classroom in a Book (hereafter ACiaB) is that lessons 1 and 2 provide a very solid foundation in both the Flash CS4 interface and tools and also how to draw, import image files, and create text. So I’m off the hook for that stuff (hoorah!). There is nothing more boring than pointing at the projection screen in front of the room and saying, “Click this little widget here.” But if you don’t have a good book, that’s what you have to do.

Those two lessons in the book should take the students about two hours to complete (maybe three hours for some students). We will do them both in the first lab.

In the lecture before that first lab, I’ll show them some recent examples of how journalists use Flash — mostly from The New York Times — so they will know what Flash is good for.

After Drawing, Animation

Here’s where I hit a snag in ACiaB: Lesson 3 was tedious and unnecessarily long and complicated. It teaches some essential stuff, but this is where my Flash experience comes into play — it’s stuff we need to get past quickly at this stage so that we can get on to animation, and the students can feel successful and competent. They need to be animating and making stuff move in the second week. But the book’s lesson 3 does not let them animate anything.

Fine, you might say — skip to lesson 4 in the book. Yes, that is the first animation lesson. But even there, the author has loaded up too much complexity for the average journalism student (typically not an artist, and not trained in complex visual compositions).

So what I need to do is give the students a more cut-to-the-chase lesson in symbols and animation. I need to make the basic animation techniques clear and simple before I let them loose on lesson 4 in the book (which they will spend their second lab working on).

So I had to make some tutorials. I made three (all short), and there will be one more (maybe next week). They are all linked here, under the heading Animation Basics.

And yes, we will skip lesson 3 in the book altogether. We’re not even looking at it.

As for lesson 4, it’s a pretty long one. I’m not sure how far most students will be able to get in two hours during our lab, but I’ll be finding out sometime in February. A good thing about this text is that it includes an extensive section (in lesson 4) about how to use the new Motion Editor, which is one of the really huge changes in Flash CS4 (different from all previous versions).

We’re also going to skip lesson 5, which goes into deep detail about the new Bone tool in CS4. This is no doubt great for professional animators, but it’s just cognitive overload for most journalism students.

Buttons and ActionScript 3.0

The chapter of ACiaB that totally sold me was lesson 6, which not only provides one of the most practical Flash lessons I have ever seen anywhere (with files on the included CD, I might add) but also supplies a clear and practical guide to using ActionScript 3.0.

The scripting in Flash drives a lot of people away (some of them probably screaming as they run). If you ever learned to write any kind of programming language in a class in school, you would not find ActionScript difficult, I’m sure. But most journalism students have never learned even one single line of computer programming. What’s more, a fair percentage of them have math phobia. (I can empathize with them.)

Buttons in Flash must be scripted, though, and buttons are the heart of what makes an interactive journalism package work. No scripting? No buttons. No buttons? No interactivity.

So lesson 6 (which is likely to take two to three hours to finish, but luckily we do have a three-hour lab period) is worth the price of the book, for me. It’s a really good introduction to the most useful way to work with buttons in Flash CS4. I built a small supplement to the book for basics of button scripting. It’s not likely to make sense on its own, but combined with the book, I think it will make the material very clear.

Is this all the ActionScript 3.0 a Flash-savvy journalist needs to know? Hardly. And in fact there’s some stuff coming later (I’ll save that for another blog post) where ACiaB falls woefully short (in particular when it comes to audio control).

So by the start of week 4, the students will have animation and button scripting under their belts. Then they will be ready to begin building interactive Flash packages.

What’s Next?

This is what remains to be covered:

  1. Movie Clip Symbols
  2. Video in Flash
  3. Packages with Loaded SWFs
  4. Sound Controls (ActionScript and external files)
  5. Loaded Data (XML and ActionScript)

The ACiaB text is pretty good for video (including use of the Flash Encoder) and for loading external SWFs, which allow us to make our packages very modular and light. The book doesn’t include anything about XML, but I’m working on a tutorial for that now, and I’m going to adapt the book’s lesson 8 to an XML version — which is how any news organization would handle a project similar to the one used in lesson 8. Nowadays I think journalism students really need some exposure to XML, and this should be a pretty neat way to get that done.

So stay tuned, and I’ll let you know how the remainder of the Flash instruction will be divided up.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl