Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 02 2012

14:41

Hate transcribing audio? Crowdsource it instead

For journalists who don't mind getting their hands dirty, Amazon's Mechanical Turk service can be a cost-effective way to avoid one of the least thrilling parts of the reporting process: transcribing. Read More »

July 26 2012

11:54

Analyzing documents with the help of the crowd

A Duke computer scientist and his graduate students are hoping their FirstPass project will help journalists analyze massive dumps of public records by harnessing the power of the crowd with Amazon's Mechanical Turk. Read More »

March 24 2011

16:00

The power of brand to inspire bias: How do perceptions of Al Jazeera English change once the logo’s gone?

William Youmans and Katie Brown are Ph.D. candidates in communication studies at the University of Michigan who just published an interesting paper in the journal Arab Media & Society about how audience bias against Al Jazeera is pushing the network to seek nontraditional methods of distribution. You can read the entire academic paper, but they’ve written a summary for the Lab below.

The diminished capacity of American TV news networks to cover international news became sharply evident during the recent uprisings in the Middle East, most notably Egypt. Into that void stepped Al Jazeera English (AJE). With headquarters in Qatar and staff already stationed in Egypt, the global news media outlet quickly mobilized an on-the-ground newsgathering presence.

But most Americans couldn’t just turn on their televisions to watch AJE’s coverage. The network is largely absent from cable and the main satellite providers’ offerings despite being available in 250 million homes globally. As Ph.D candidates in communication studies at the University of Michigan, we were interested in the role that Americans’ perception of the channel might have in its difficulties getting cable carriage — and how online distribution might serve as a fruitful workaround. That led us to an experimental study that looked at how Al Jazeera branding might influence public perception of a piece of journalism.

The Egypt effect

For years, some in the Bush administration and the American media spoke of the Arabic Al Jazeera channel (AJ) as a spreader for enemy propaganda in Afghanistan and Iraq. This association proved robust in American political discourse. It was one reason AJE had such a tough time getting into the American market when it launched in late 2006. Even today, only cable systems in Washington, D.C., Burlington, Vermont, and Toledo, Ohio currently carry the channel in its entirety.

AJE’s coverage of Egypt was something of a turning point for the network’s image in the United States. Visits to AJE’s website increased 25 times, with more than half of the traffic coming from the U.S. The D.C. area was one of the leaders in Google searches for “Al Jazeera English” at the time. The press not only turned to AJE for information and footage, but lauded its work; ABC News’ Sam Donaldson thanked the network on air. Secretary of State Hillary Clinton called AJE “real news” and juxtaposed it with the talking head-dominated American channels.

As public discourse about AJE changed, many began to question its lack of availability on television sets. Then-New York Times columnist Frank Rich made a tongue-in-cheek analogy: during the Egypt story, “news-starved Americans” tracking down AJE online were like “Iron Curtain citizens clandestinely trying to pull in the jammed Voice of America signal in the 1950s.”

But despite the accolades and calls for carriage, cable companies appeared to let AJE’s “moment” pass, at least for now. In late February, AJE met with the nation’s two largest cable operators, Comcast and Time-Warner. No deal has been announced in the month since (although carriage deals often take longer to materialize).

It is likely the operators are holding out for evidence that attention on AJE sustains or increases. The question of cable carriage is not just a function of policymakers, the press, and cable company preferences. Public demand is an important part of the equation. Are Americans generally open-minded towards AJE after the Egypt coverage?

We conducted an experimental study (pdf) on how potential viewer attitudes toward AJE change with exposure to the channel’s news content. Carried out online in late February to early March, our study involved 177 American participants, drawn from Amazon’s Mechanical Turk pool.

The participants were randomly assigned to three groups. Two of them watched an AJE-produced news clip about the Taliban’s position towards peace talks, which included minimal reference to America. The first group watched the original clip with AJE’s branding:

The second group saw the same news piece re-edited to carry CNN International’s (CNNI) logo.

The third group, the control, viewed no clip. We then asked participants in each group to rate, in general, how biased they thought AJE and CNNI were.

Watching the AJE clip — branded as AJE — did not seem to have an impact on perceptions of bias; bias ratings were equal between those in the AJE-clip-watching group and the control group.

But in the group that had just watched the clip with fake CNNI branding, participants rated CNNI as less biased than those in the control group.

This suggests that many Americans may be unwilling to change their perceptions of AJE — despite the fact that the same clip, when attributed to CNNI, boosted their impressions of the American network.

We also asked all the participants about views towards cable carriage: Should AJE be on cable systems? The responses were distributed in a bell-curve, with no significant differences between conditions. The largest group, about 40 percent, was indifferent. Roughly 25 percent said they prefer carriage but would not take action to promote it. Slightly fewer, about 20 percent, said they would merely prefer it’s not on air, but would do nothing about it either way. While 5 percent said they would contact cable companies to request AJE, 7 percent said they would actively oppose AJE’s carriage. (No one said they would take action opposing CNNI’s availability.)

This finding of an oppositional minority is echoed by actual action, ranging from national petitions to protests against a Pacifica radio station in Houston and a campaign against a small college cable system’s airing of AJE programming in Daytona Beach. In Vermont, some members of the public and Burlington city officials protested the presence of AJE on the municipally-run telecom, sparking a local debate. AJE remained a part of the lineup. Former NBC executive Jeff Zucker suggested that one cause of AJE’s cable troubles is the fear advocacy groups and high-profile media figures “would go after some of those distributors if they were to put Al-Jazeera on.”

But even absent public opposition, there would still be doubts about the commercial feasibility of another news network. Cable companies can point to declining news audiences and the supposed lack of American public interest in international news, arguing that the TV news market has reached a saturation point. These, along with the fear of backlash, only creates further reluctance in an already risk averse industry. The preferences of those in favor of AJE’s availability, around one-third of our respondents, are overridden by this outcome. The power of cable as a gatekeeper prevents AJE from participating in the open competition of ideas so important to American free press values.

Circumventing cable

AJE’s best chance for getting around cable gatekeepers is by continuing to develop new, mostly online, distribution channels. Survey research from Pew suggests that while TV news viewing since 1996 has been relatively stable, online news consumption since 2006 has been on the rise.

The lack of cable carriage may force AJE to look ahead of the curve if it is to build an American audience. AJE’s online news gathering, presentation, and distribution are still developing, but have shown major improvements in the past year especially.

AJE’s provision of video clips and online livestreaming via its website and YouTube, where it is currently the third most watched news and politics channel, enhanced its accessibility tremendously. Google, to the extent it is increasingly becoming a media company, has been hospitable to AJE.

AJE has arranged a deal for carriage through Roku, the Internet-based set-top video delivery company — although how much of a substitute such Internet-based TV systems will be for cable is still an open question. And AJE continued to roll out smartphone distribution by adding an Android app to its iPhone, Nokia, and Samsung lineup.

During the Egypt story, the network’s website coverage and online videos were heavily redistributed via social media such as Twitter and Facebook and led many to AJE’s website. At times, as many as 70 percent of its website visitors linked in from social networking platforms and sites.

News flows online are diffuse and remain relatively free of large gatekeepers. Small vocal groups are less able to deny access to news and information they oppose through protests and threats of boycott. Questions of middle-man profitability and channel capacity constraints do not constrain online distribution. One unintentional advantage of its exclusion from cable and American satellite is that AJE will be better placed as news consumption routines increasingly depend on the Internet — assuming new, powerful gatekeepers do not arise to block others’ access to information.

November 03 2010

14:00

It’s people! Meet Soylent, the crowdsourced copy editor

The phrase “on-demand human computation” has a sinister tinge to it, if only because the idea of sucking the brain power out of a group of people is generally frowned upon. And yet, if you call it “crowdsourcing” everything sounds so much friendlier!

But calling Soylent “crowdsourced copy-editing” isn’t quite fair, since the system performs the type of jobs that are somewhere in the gray area between man and machine. More than a spell check, not quite the nightside copy editor versed in AP style, Soylent really is on-demand computation. It’s what all word processors need, the “Can you take a look at this?” button with a small workforce of people at your disposal.

Soylent is an add-in for Microsoft Word that uses Mechanical Turk as a distributed copy-editing system to perform tasks like proofreading and text-shortening, as well as a type of specialized edits its developers call “The Human Macro.” Currently in closed beta, Soylent was created by compsci students at MIT, Berkeley, and University of Michigan.

For those unfamiliar, Mechanical Turk is an Amazon service that makes it easier for small tasks (and the money to pay for them) to be distributed among a group of humans called Turkers. While savvy writers could already use MTurk to edit their work, the team at Soylent believes their system can produce better and more efficient results than would a writer working alone.

“The idea of Soylent is, what if we could embed human knowledge in the word processor?” MIT’s Michael Bernstein, the lead researcher on Soylent, told me.

That sounds technical, but as Bernstein explains, we all call on friends for help when writing. Research paper, essay, email, story, or blog post — most people rely on a second pair of eyeballs for help at least some of the time. And one thing Mechanical Turk has to offer is a lot of eyeballs.

Soylent’s three current features are called Shortn, Crowdproof, and the Human Macro:

Shortn: Ever write 1,700 words and blow right past your 1,200 word count? Shortn lets writers submit passages of text to MTurk for trimming. They can determine how much they want to cut with a handy slider tool.

Crowdproof: A superpowered, sophisticated spell, grammar and style check that provides suggestions as well as explanations why your choices are wrong.

The Human Macro: For more complicated changes — something like “change all verbs to past tense” — the Human Macro is, as Bernstein says, programming-as-craigslist-ad. The writer describes the changes she wants (capitalization of proper names, altering verb tense, annotating references with Creative Commons photos) in a request form, which humans then act on.

Bernstein argues that Soylent’s cold, detached eye is just want some writing needs. “It’s really hard to kill your own babies in your writing,” Bernstein said. “To be honest, another motivation for me is that it’s very time consuming to go and snip words and cut things from paragraphs an hour before deadline.”

But to writers already nervous about those babies being disappeared on the copy desk, handing over their copy to the faceless masses might not sound like a solution. In their research, Bernstein and his colleagues identified “lazy” and “overeager” individual Turkers, with the lazy ones doing the minimal amount of work and the overeager making wholesale changes. Bernstein said the distributed editing process behind Soylent eliminates this problem because no one Turker is working with whole passages of a document; the work is split among many.

Some in news circles are already experimenting with Mechanical Turk; ProPublica used it to identify companies getting stimulus dollars for the Recovery Tracker project. (Here at the Lab, we use it for the long transcripts we sometimes run of video or audio interviews.) MTurk could be used for any number of tasks that call for on-demand labor. But what makes Soylent different from using MTurk directly is a programming pattern Bernstein and his colleagues created called Find-Fix-Verify, which disseminates tasks across a large group of workers. The only thing required of writers is an Amazon account to pay Turkers; Soylent sets the payment rates.

Instead of one Turker reading over an entire page or paragraph, Soylent asks a group of workers to find areas that need fixing and make corrections. Those fixes are then then filtered by other Turkers for inaccuracies, which produces a set of recommendations or an edited graph to a writer. Depending on the job and the document, it usually took Soylent around 40 minutes to complete a task.

To news traditionalists, Soylent may sound like the latest turn toward outsourcing in journalism that has sent copy editing jobs to places in India. It could also be akin to the automated journalism being tested by some companies or the Huffington Post’s real-time headline testing. And some day it may be. But Soylent is far from ready for the mainstream, thanks to the processing time and payment methods. Bernstein says they’re working towards having real-time edits and managing payment through Soylent, as well adapting the program to work on photo editing. Instead of outsourcing, think of Soylent as microsourcing.

And about that name: It comes from exactly what you’re thinking. Bernstein said they were looking for something familiar but also true to the idea of what they created. Soylent, is made of people. It is indeed, people.

“The original name was Homunculus,” Bernstein said. “It didn’t have the same ring to it.”

January 13 2010

17:35
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl