Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

October 22 2010


Help Me Investigate – anatomy of an investigation

Earlier this year I and Andy Brightwell conducted some research into one of the successful investigations on my crowdsourcing platform Help Me Investigate. I wanted to know what had made the investigation successful – and how (or if) we might replicate those conditions for other investigations.

I presented the findings (presentation embedded above) at the Journalism’s Next Top Model conference in June. This post sums up those findings.

The investigation in question was ‘What do you know about The London Weekly?‘ – an investigation into a free newspaper that was (they claimed – part of the investigation was to establish if this was a hoax) about to launch in London.

The people behind the paper had made a number of claims about planned circulation, staffing and investment that most of the media reported uncritically. Martin Stabe, James Ball and Judith Townend, however, wanted to dig deeper. So, after an exchange on Twitter, Judith logged onto Help Me Investigate and started an investigation.

A month later members of the investigation had unearthed a wealth of detail about the people behind The London Weekly and the facts behind their claims. Some of the information was reported in MediaWeek and The Media Guardian podcast Media Talk; some formed the basis for posts on James Ball’s blog, Journalism.co.uk and the Online Journalism Blog. Some has, for legal reasons, remained unpublished.

A note on methodology

Andrew conducted a number of semi-structured interviews with contributors to the investigation. The sample was randomly selected but representative of the mix of contributors, who were categorised as either ‘alpha’ contributors (over 6 contributions), ‘active’ (2-6 contributions) and ‘lurkers’ (whose only contribution was to join the investigation). These interviews formed the qualitative basis for the research.

Complementing this data was quantitative information about users of the site as a whole. This was taken from two user surveys – one when the site was 3 months’ old and another at 12 months – and analysis of analytics taken from the investigation (such as numbers and types of actions, frequency, etc.)

What are the characteristics of a crowdsourced investigation?

One of the first things I wanted to analyse was whether the investigation data matched up to patterns observed elsewhere in crowdsourcing and online activity. An analysis of the number of actions by each user, for example, showed a clear ‘power law’ distribution, where a minority of users accounted for the majority of activity.

This power law, however, did not translate into a breakdown approaching the 90-9-1 ‘law of participation inequality‘ observed by Jakob Nielsen. Instead, the balance between those who made a couple of contributions (normally the 9% of the 90-9-1 split) and those who made none (the 90%) was roughly equal. This may have been because the design of the site meant it was not possible to ‘lurk’ without being a member of the site already, or being invited and signing up.

Adding in data on those looking at the investigation page who were not members may have shed further light on this.

What made the crowdsourcing successful?

Clearly, it is worth making a distinction between what made the investigation successful as a series of outcomes, and what made crowdsourcing successful as a method.

What made the community gather, and continue to return? One hypothesis was that the nature of the investigation provided a natural cue to interested parties – The London Weekly was published on Fridays and Saturdays and there was a build up of expectation to see if a new issue would indeed appear.

I was curious to see if the investigation had any ‘rhythm’. Would there be peaks of interest correlating to the expected publication?

The data threw up something else entirely. There was indeed a rhythm but it was Wednesdays that were the most popular day for people contributing to the investigation.

Why? Well, it turned out that one of the investigation’s ‘alpha’ contributors – James Ball – set himself a task to blog about the investigation every week. His blog posts appeared on a Wednesday.

That this turned out to be a significant factor in driving activity tells us one important lesson: talking publicly and regularly about the investigation’s progress is key.

This data was backed up from the interviews. One respondent mentioned the “weekly cue” explicitly.

More broadly, it seems that the site helped keep track of a number of discussions taking place around the web. Having been born from a discussion on Twitter, further conversations on Twitter resulted in further people signing up, along with comments threads and other online discussion. This fit the way the site was designed culturally – to be part of a network rather than asking people to do everything on-site.

But the planned technical connectivity of the site with the rest of the web (being able to pull related tweets or bookmarks, for example) had been dropped during development as we focused on core functionality. This was not a bad thing, I should emphasise, as it prevented us becoming distracted with ‘bells and whistles’ and allowed us to iterate in reaction to user activity rather than our own assumptions of what users would want. This research shows that user activity and informs future development accordingly.

The presence of ‘alpha’ users like James and Judith was crucial in driving activity on the site – a pattern observed in other successful investigations. They picked up the threads contributed by others and not only wove them together into a coherent narrative that allowed others to enter more easily, but also set the new challenges that provided ways for people to contribute. The fact that they brought with them a strong social network presence is probably also a factor – but one that needs further research.

The site has always been designed to emphasise the role of the user in driving investigations. The agenda is not owned by a central publisher, but by the person posing the question – and therefore the responsibility is theirs as well. In this sense it draws on Jenkins’ argument that “Consumers will be more powerful within convergence culture – but only if they recognise and use that power.” This cultural hurdle may be the biggest one that the site has to address.

Indeed, the site is also designed to offer “Failure for free”, allowing users to learn what works and what doesn’t, and begin to take on that responsibility where required.

The investigation also suited crowdsourcing well, as it could be broken down into separate parts and paths – most of which could be completed online: “Where does this claim come from?” “Can you find out about this person?” “What can you discover about this company?”. One person, for example, used Google Streetview to establish that the registered address of the company was a postbox.

Other investigations that are less easily broken down may be less suitable for crowdsourcing – or require more effort to ensure success.

A regular supply of updates provided the investigation with momentum. The accumulation of discoveries provided valuable feedback to users, who then returned for more. In his book on Wikipedia, Andrew Lih (2009 p82) notes a similar pattern – ‘stigmergy‘ – that is observed in the natural world: “The situation in which the product of previous work, rather than direct communication [induces and directs] additional labour”. An investigation without these ‘small pieces, loosely joined’ might not suit crowdsourcing so well.

One problem, however, was that those paths led to a range of potential avenues of enquiry. In the end, although the core questions were answered (was the publication a hoax and what were the bases for their claims) the investigation raised many more questions.

These remained largely unanswered once the majority of users felt that their questions had been answered. Like any investigation, there came a point at which those involved had to make a judgement whether they wished to invest any more time in it.

Finally, the investigation benefited from a diverse group of contributors who contributed specialist knowledge or access. Some physically visited stations where the newspaper was claiming distribution to see how many copies were being handed out. Others used advanced search techniques to track down details on the people involved and the claims being made, or to make contact with people who had had previous experiences with those behind the newspaper.

The visibility of the investigation online led to more than one ‘whistleblower’ approach providing inside information.

What can be done to make it better?

Looking at the reasons that users of the site as a whole gave for not contributing to an investigation, the majority attributed this to ‘not having enough time’. Although at least one interviewee, in contrast, highlighted the simplicity and ease of contributing, it needs to be as easy and simple as possible for users to contribute in order to lower the perception of effort and time needed.

Notably, the second biggest reason for not contributing was a ‘lack of personal connection with an investigation’, demonstrating the importance of the individual and social dimension of crowdsourcing. Likewise, a ‘personal interest in the issue’ was the single largest factor in someone contributing. A ‘Why should I contribute?’ feature on each investigation may be worth considering.

Others mentioned the social dimension of crowdsourcing – the “sense of being involved in something together” – what Jenkins (2006) would refer to as “consumption as a networked practice”.

This motivation is also identified by Yochai Benkler in his work on networks. Looking at non-financial reasons why people contribute their time to online projects, he refers to “socio-psychological reward”. He also identifies the importance of “hedonic personal gratification”. In other words, fun. (Interestingly, these match two of the three traditional reasons for consuming news: because it is socially valuable, and because it is entertaining. The third – because it is financially valuable – neatly matches the third reason for working).

While it is easy to talk about “Failure for free”, more could be done to identify and support failing investigations. We are currently developing a monthly update feature that would remind users of recent activity and – more importantly – the lack of activity. The investigators in a group might be asked whether they wish to terminate the investigation in those cases, emphasising their role in its progress and helping ‘clean up’ the investigations listed on the first page of the site.

That said, there is also a danger is interfering too much in reducing failure. This is a natural instinct, and I have to continually remind myself that I started the project with an expectation of 95-99% of investigations ‘failing’ through a lack of motivation on the part of the instigator. That was part of the design. It was the 1-5% of questions that gained traction that would be the focus of the site (this is how Meetup works, for example – most groups ‘fail’ but there is no way to predict which ones. As it happens, the ‘success’ rate of investigations has been much higher than expected). One analogy is a news conference where members throw out ideas – only a few are chosen for investment of time and energy, the rest ‘fail’.

In the end, it is the management of that tension between interfering to ensure everything succeeds – and so removing the incentive for users to be self-motivated – and not interfering at all – leaving users feeling unsupported and unmotivated – that is likely to be the key to a successful crowdsourcing project. More than a year into the project, this is still a skill that I am learning.

June 11 2010


More from #JNTM: Flawed thinking behind government local TV plans

Following on from the previous post, another government policy up for criticism at this week’s Journalism’s Next Top Model conference was the much-mooted local TV plans.

This was a recurring theme of Culture Secretary Jeremy Hunt’s speeches while in opposition, and this week he announced that Nicholas Shott, Head of UK Investment Banking at Lazard, will “look at the potential for commercially viable local television stations within the local media landscape right across the nations and regions of the UK.”

Roger Parry - credited with much of the thinking behind Hunt’s proposals – backed the plan, seeing it as being more about local multimedia than local TV. He expected around 80 local TV stations to be made available on Freeview with a consumer-focused mix of programming (gardening, DIY, etc.) and sponsorship rather than spot advertising.

But Clare Enders (Enders Analysis) and Will Perrin (Talk About Local) were hugely sceptical of the commercial basis for a local TV market in the UK.

Enders pointed out that while local newspaper advertising was worth £2.6bn this year, £1bn of that came from classifieds – a form that doesn’t translate to TV (ITV’s previous experiments with ‘video classifieds’ was, she said, a “disaster”).

She also highlighted the vast differences between the US local TV market – where national networks support state affiliates and states have their own “separateness” – and that in the UK where, she predicted, impending budget cuts “will kill some local economies – not inspire ad growth. It will get worse.”

“[Local TV] hasn’t panned out and my goodness has it been looked at in the past couple of decades.”


Media economist: Government ‘doesn’t understand economics’ over relaxing ownership rules

On Wednesday I spoke at the thoroughly enjoyable Journalism’s Next Top Model conference at Westminster University. Highlight of the day was keynote speaker Robert Picard, a media economist able to separate publishers’ sense of entitlement from the hard realities of economics and business (mis)management.

Journalism will survive, he said, because there will always be a demand for it. But most print publishers will die because over the past few decades they quite simply haven’t managed their accounts responsibly. While a typical business should have a debt-to-equity ratio of around 1:1, some publishers have racked up ratios ranging from 6:1 to 66:1.

“If you haven’t managed your balance sheet you get in trouble in a recession. Do I feel bad for them? No. They made stupid mistakes.”

One particular mistake highlighted by Picard was the switch in the 1990s from making acquisitions with stock to making acquisitions with debt.

“All the newspapers were making profits when they went bankrupt,” he pointed out. It was their handling of debt that killed them.

I asked Robert about the government’s plans to relax (and consider removing) local media ownership rules – and whether that would indeed create the environment for entrepreneurialism they want to encourage. His response was simple: “You don’t encourage competition by relaxing ownership rules.

“They don’t understand economics,” if they thought that would happen, he continued. “We need people to start more media organisations, not merge into fewer organisations.”

Picard seemed to feel that the Dutch government’s moves to provide funds to help news organisations restructure, or to re-skill journalists, were more intelligent responses.

June 09 2010


#JNTM: Professor Robert Picard on why newspapers deserve to die

“Newspapers deserve to die,” Professor Robert Picard told delegates at the University of Wesminster / British Journalism Review Journalism’s Next Top Model conference this morning.

But that doesn’t mean he wants to see journalism die: it’s time to change the products and the platforms, he said. The future of journalism is dependent on journalists and other distribution platforms; not newspapers.

Picard, Hamrin Professor of Media Economics and director of the Media Management and Transformation Centre, at Jonkoping University in Sweden and fellow at the Reuters Institute in Oxford, claimed that print distribution is an expensive and inefficient way to spread news. “I think we’ll have paper for a while,” he said. For 20 years even, he guessed, but we’ll see more migration to screen.

He’s not at all nostalgic about news organisations’ bureaux spread out over the world, and says it’s time for newspapers to pool resources and become more efficient. As newspapers grew in the second half of the 20th century, they developed complex systems and bureaucracy, which has led to inefficiency, he said.

“You get very high overhead costs to support the corporations along the way, one of the big problems with success,” he said.

He encouraged news organisations to consider:

  • Smaller and more agile operations
  • A more entrepreneurial approach
  • More innovation in products and process
  • Alliances, networking and cooperation
  • Multiple sources of financial funding
  • Rethinking of entire business model of media and how it creates value for customers and itself

Something is wrong with the product, he said, when 40 per cent of public claim they don’t want to read the newspaper they used to read (source of stat not cited).

“I’ve been saying for 10 years – why in the world are newspaper printing stock tables?” It’s time to kill these, along with the television guides, he said, as consumers find with other ways of sourcing up-to-date information.

Stop simply reporting news and provide value to the consumer, he said. Consumer can get top ten headlines from internet services, so newspaper organisations have to provide something different than the “flow of information”.

Answering a question about the realities for newspapers, he speculated that while the Guardian is North America’s biggest news site (that it attracts the highest number of unique users in the region is a little known fact, he said), the newspaper itself (not the org, necessarily) is likely to die – along with the Independent. Newspapers don’t interest Picard at all – but saving journalism does.

Professor Picard recently sat on a panel between Arianna Huffington and Rupert Murdoch, who don’t like each other very much. Murdoch is saying we’ve got to save the business; Huffington is saying we have to destroy the business. Some place between Huffington and Murdoch’s realities is where we are, he said.

Similar Posts:

June 08 2010


#JNTM: Journalism’s Next Top Model event at University of Westminster

The University of Westminster and British Journalism Review is hosting a two day event (8/9 June) discussing Journalism’s Next Top Model – the industry discussion takes place tomorrow (Wednesday). Wifi access is limited, but the Cover It Live blog below should pick up some of the tweets coming out of the event. Westminster students from the event are due to update their blog at this link. The University plans to live stream some of the event at this link.

Similar Posts:

June 04 2010


University of Westminster gets ready to discuss ‘Journalism’s Next Top Model’

The University of Westminster has chosen a more upbeat theme for its 2010 journalism event. Whereas we discussed ‘journalism in crisis’ this time last year, delegates from around the world will gather next week to find ‘Journalism’s Next Top Model’, a conference held by Westminster’s department of journalism in association with the British Journalism Review.

News media all over the world are failing as the traditional revenue from advertising bleeds to the web. Free news websites are discouraging the young from buying newspapers. So who will pay for what sort of journalism in the future?

The academic-oriented discussions take place on Tuesday 8 June; including presentations by Francois Nel from the University of Central Lancashire; Rasmus Kleis Nielsen and David Levy from the Reuters Institute; and Professor Steven Barnett from University of Westminster, who will be joined by speakers from all over the world – Sweden, Turkey, Norway and Macedonia, for example.

On the Wednesday, industry panellists will include: Jane Moore (the Sun); Peter Oborne (the Mail/ Channel 4); Roger Parry (former chair, Johnston Press); William Perrin (activist and blogger); and Claire Enders (Enders Analysis).

The conference concludes with the presentation of the British Journalism Review’s Outstanding Contribution to Broadcast Journalism prize; and a memorial lecture by Boris Johnson.

The conference will take place 8 June 2010 – 9 June 2010 at 309 Regent Street, London W1. Registration is via this form (download at this link) and we’re told applications will still be accepted on Monday.

Similar Posts:

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!