Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 29 2013

16:51

What’s New in Digital Scholarship: Teen sharing on Facebook, how Al Jazeera uses metrics, and the tie between better cellphone coverage and violence

library-shelves-of-academic-journals-cc

Editor’s note: There’s a lot of interesting academic research going on in digital media — but who has time to sift through all those journals and papers?

Our friends at Journalist’s Resource, that’s who. JR is a project of the Shorenstein Center on the Press, Politics and Public Policy at the Harvard Kennedy School, and they spend their time examining the new academic literature in media, social science, and other fields, summarizing the high points and giving you a point of entry. Roughly once a month, JR managing editor John Wihbey will sum up for us what’s new and fresh.

This month’s edition of What’s New In Digital Scholarship is an abbreviated installment — we’re just posting our curated list of interesting new papers and their abstracts. We’ll provide a fuller analysis at the half-year mark, in our June edition. Until then, happy geeking out!

“Mapping the global Twitter heartbeat: The geography of Twitter.” Study from the University of Illinois Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign, published in First Monday. By Kalev Leetaru, Shaowen Wang, Guofeng Cao, Anand Padmanabhan, and Eric Shook.

Summary: “In just under seven years, Twitter has grown to count nearly three percent of the entire global population among its active users who have sent more than 170 billion 140-character messages. Today the service plays such a significant role in American culture that the Library of Congress has assembled a permanent archive of the site back to its first tweet, updated daily. With its open API, Twitter has become one of the most popular data sources for social research, yet the majority of the literature has focused on it as a text or network graph source, with only limited efforts to date focusing exclusively on the geography of Twitter, assessing the various sources of geographic information on the service and their accuracy. More than three percent of all tweets are found to have native location information available, while a naive geocoder based on a simple major cities gazetteer and relying on the user — provided Location and Profile fields is able to geolocate more than a third of all tweets with high accuracy when measured against the GPS-based baseline. Geographic proximity is found to play a minimal role both in who users communicate with and what they communicate about, providing evidence that social media is shifting the communicative landscape.

“Predicting Dissemination of News Content in Social Media: A Focus on Reception, Friending, and Partisanship.” Study from Ohio State, published in Journalism & Mass Communication Quarterly. By Brian E. Weeks and R. Lance Holbert.

Summary: “Social media are an emerging news source, but questions remain regarding how citizens engage news content in this environment. This study focuses on social media news reception and friending a journalist/news organization as predictors of social media news dissemination. Secondary analysis of 2010 Pew data (N = 1,264) reveals reception and friending to be positive predictors of dissemination, and a reception-by-friending interaction is also evident. Partisanship moderates these relationships such that reception is a stronger predictor of dissemination among partisans, while the friending-dissemination link is evident for nonpartisans only. These results provide novel insights into citizens’ social media news experiences.”

“Al Jazeera English Online: Understanding Web metrics and news production when a quantified audience is not a commodified audience.” Study from George Washington University, published in Digital Journalism. By Nikki Usher.

Summary: “Al Jazeera English is the Arab world’s largest purveyor of English language news to an international audience. This article provides an in-depth examination of how its website employs Web metrics for tracking and understanding audience behavior. The Al Jazeera Network remains sheltered from the general economic concerns around the news industry, providing a unique setting in which to understand how these tools influence newsroom production and knowledge creation. Through interviews and observations, findings reveal that the news organization’s institutional culture plays a tremendous role in shaping how journalists use and understand metrics. The findings are interpreted through an analysis of news norms studies of the social construction of technology.”

“Teens, Social Media and Privacy.” Report from the Pew Internet & American Life Project and Harvard’s Berkman Center for Internet & Society. By Mary Madden, Amanda Lenhart, Sandra Cortesi, Urs Gasser, Maeve Duggan, and Aaron Smith.

Summary: “Teens are sharing more information about themselves on social media sites than they have in the past, but they are also taking a variety of technical and non-technical steps to manage the privacy of that information. Despite taking these privacy-protective actions, teen social media users do not express a high level of concern about third-parties (such as businesses or advertisers) accessing their data; just 9% say they are ‘very’ concerned. Key findings include: Teens are sharing more information about themselves on their social media profiles than they did when we last surveyed in 2006: 91% post a photo of themselves, up from 79% in 2006; 71% post their school name, up from 49%; 71% post the city or town where they live, up from 61%; 53% post their email address, up from 29%; 20% post their cell phone number, up from 2%. 60% of teen Facebook users set their Facebook profiles to private (friends only), and most report high levels of confidence in their ability to manage their settings: 56% of teen Facebook users say it’s ‘not difficult at all’ to manage the privacy controls on their Facebook profile; 33% Facebook-using teens say it’s ‘not too difficult’; 8% of teen Facebook users say that managing their privacy controls is ‘somewhat difficult,’ while less than 1% describe the process as ‘very difficult.’”

“Historicizing New Media: A Content Analysis of Twitter.” Study from Cornell, Stoneybrook University, and AT&T Labs Research, published in the Journal of Communication. By Lee Humphreys, Phillipa Gill, Balachander Krishnamurthy, and Elizabeth Newbury.

Summary: “This paper seeks to historicize Twitter within a longer historical framework of diaries to better understand Twitter and broader communication practices and patterns. Based on a review of historical literature regarding 18th and 19th century diaries, we created a content analysis coding scheme to analyze a random sample of publicly available Twitter messages according to themes in the diaries. Findings suggest commentary and accounting styles are the most popular narrative styles on Twitter. Despite important differences between the historical diaries and Twitter, this analysis reveals long-standing social needs to account, reflect, communicate, and share with others using media of the times.” (See also.)

“Page flipping vs. clicking: The impact of naturally mapped interaction technique on user learning and attitudes.” Study from Penn State and Ohio State, published in Computers in Human Behavior. By Jeeyun Oh, Harold R. Robinson, and Ji Young Lee.

Summary: “Newer interaction techniques enable users to explore interfaces in a more natural and intuitive way. However, we do not yet have a scientific understanding of their contribution to user experience and theoretical mechanisms underlying the impact. This study examines how a naturally mapped interface, page-flipping interface, can influence user learning and attitudes. An online experiment with two conditions (page flipping vs. clicking) tests the impact of this naturally mapped interaction technique on user learning and attitudes. The result shows that the page-flipping feature creates more positive evaluations of the website in terms of usability and engagement, as well as greater behavioral intention towards the website by evoking greater perception of natural mapping and greater feeling of presence. In terms of learning outcomes, however, participants who flip through the online magazine show less recall and recognition memory, unless they perceive page flipping as more natural and intuitive to interact with. Participants perceive the same content as more credible when they flip through the content, but only if they appreciate the coolness of the medium. Theoretical and practical implications will be discussed.”

“Influence of Social Media Use on Discussion Network Heterogeneity and Civic Engagement: The Moderating Role of Personality Traits.” Study from the University of Alabama, Tuscaloosa, and the University of Texas at Austin, published in the Journal of Communication. By Yonghwan Kim, Shih-Hsien Hsu, and Homero Gil de Zuniga.

Summary: “Using original national survey data, we examine how social media use affects individuals’ discussion network heterogeneity and their level of civic engagement. We also investigate the moderating role of personality traits (i.e., extraversion and openness to experiences) in this association. Results support the notion that use of social media contributes to heterogeneity of discussion networks and activities in civic life. More importantly, personality traits such as extraversion and openness to experiences were found to moderate the influence of social media on discussion network heterogeneity and civic participation, indicating that the contributing role of social media in increasing network heterogeneity and civic engagement is greater for introverted and less open individuals.”

“Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds.” Study from Sammy Ofer School of Communications, Interdisciplinary Center Herzliya (Israel), published in Computers in Human Behavior. By Béatrice S. Hasler, Peleg Tuchman, and Doron Friedman.

Summary: “We conducted an experiment to evaluate the use of embodied survey bots (i.e., software-controlled avatars) as a novel method for automated data collection in 3D virtual worlds. A bot and a human-controlled avatar carried out a survey interview within the virtual world, Second Life, asking participants about their religion. In addition to interviewer agency (bot vs. human), we tested participants’ virtual age, that is, the time passed since the person behind the avatar joined Second Life, as a predictor for response rate and quality. The human interviewer achieved a higher response rate than the bot. Participants with younger avatars were more willing to disclose information about their real life than those with older avatars. Surprisingly, the human interviewer received more negative responses than the bot. Affective reactions of older avatars were also more negative than those of younger avatars. The findings provide support for the utility of bots as virtual research assistants but raise ethical questions that need to be considered carefully.”

“Technology and Collective Action: The Effect of Cell Phone Coverage on Political Violence in Africa.” Study from Duke and German Institute of Global and Area Studies (GIGA), published in the American Political Science Review. By Jan H. Pierskalla and Florian M. Hollenbach.

Summary: “The spread of cell phone technology across Africa has transforming effects on the economic and political sphere of the continent. In this paper, we investigate the impact of cell phone technology on violent collective action. We contend that the availability of cell phones as a communication technology allows political groups to overcome collective action problems more easily and improve in-group cooperation, and coordination. Utilizing novel, spatially disaggregated data on cell phone coverage and the location of organized violent events in Africa, we are able to show that the availability of cell phone coverage significantly and substantially increases the probability of violent conflict. Our findings hold across numerous different model specifications and robustness checks, including cross-sectional models, instrumental variable techniques, and panel data methods.”

Photo by Anna Creech used under a Creative Commons license.

August 21 2012

16:17

August 17 2012

16:07

Metrics, metrics everywhere: How do we measure the impact of journalism?

If democracy would be poorer without journalism, then journalism must have some effect. Can we measure those effects in some way? While most news organizations already watch the numbers that translate into money (such as audience size and pageviews), the profession is just beginning to consider metrics for the real value of its work.

That’s why the recent announcement of a Knight-Mozilla Fellowship at The New York Times on “finding the right metric for news” is an exciting moment. A major newsroom is publicly asking the question: How do we measure the impact of our work? Not the economic value, but the democratic value. The Times’ Aaron Pilhofer writes:

The metrics newsrooms have traditionally used tended to be fairly imprecise: Did a law change? Did the bad guy go to jail? Were dangers revealed? Were lives saved? Or least significant of all, did it win an award?

But the math changes in the digital environment. We are awash in metrics, and we have the ability to engage with readers at scale in ways that would have been impossible (or impossibly expensive) in an analog world.

The problem now is figuring out which data to pay attention to and which to ignore.

Evaluating the impact of journalism is a maddeningly difficult task. To begin with, there’s no single definition of what journalism is. It’s also very hard to track what happens to a story once it is released into the wild, and even harder to know for sure if any particular change was really caused by that story. It may not even be possible to find a quantifiable something to count, because each story might be its own special case. But it’s almost certainly possible to do better than nothing.

The idea of tracking the effects of journalism is old, beginning in discussions of the newly professionalized press in the early 20th century and flowering in the “agenda-setting” research of the 1970s. What is new is the possibility of cheap, widespread, data-driven analysis down to the level of the individual user and story, and the idea of using this data for managing a newsroom. The challenge, as Pilhofer put it so well, is figuring out which data, and how a newsroom could use that data in a meaningful way.

What are we trying to measure and why?

Metrics are powerful tools for insight and decision-making. But they are not ends in themselves because they will never exactly represent what is important. That’s why the first step in choosing metrics is to articulate what you want to measure, regardless of whether or not there’s an easy way to measure it. Choosing metrics poorly, or misunderstanding their limitations, can make things worse. Metrics are just proxies for our real goals — sometimes quite poor proxies.

An analytics product such as Chartbeat produces reams of data: pageviews, unique users, and more. News organizations reliant on advertising or user subscriptions must pay attention to these numbers because they’re tied to revenue — but it’s less clear how they might be relevant editorially.

Consider pageviews. That single number is a combination of many causes and effects: promotional success, headline clickability, viral spread, audience demand for the information, and finally, the number of people who might be slightly better informed after viewing a story. Each of these components might be used to make better editorial choices — such as increasing promotion of an important story, choosing what to report on next, or evaluating whether a story really changed anything. But it can be hard to disentangle the factors. The number of times a story is viewed is a complex, mixed signal.

It’s also possible to try to get at impact through “engagement” metrics, perhaps derived from social media data such as the number of times a story is shared. Josh Stearns has a good summary of recent reports on measuring engagement. But though it’s certainly related, engagement isn’t the same as impact. Again, the question comes down to: Why would we want to see this number increase? What would it say about the ultimate effects of your journalism on the world?

As a profession, journalism rarely considers its impact directly. There’s a good recent exception: a series of public media “impact summits” held in 2010, which identified five key needs for journalistic impact measurement. The last of these needs nails the problem with almost all existing analytics tools:

While many Summit attendees are using commercial tools and services to track reach, engagement and relevance, the usefulness of these tools in this arena is limited by their focus on delivering audiences to advertisers. Public interest media makers want to know how users are applying news and information in their personal and civic lives, not just whether they’re purchasing something as a result of exposure to a product.

Or as Ethan Zuckerman puts it in his own smart post on metrics and civic impact, ”measuring how many people read a story is something any web administrator should be able to do. Audience doesn’t necessarily equal impact.” Not only that, but it might not always be the case that a larger audience is better. For some stories, getting them in front of particular people at particular times might be more important.

Measuring audience knowledge

Pre-Internet, there was usually no way to know what happened to a story after it was published, and the question seems to have been mostly ignored for a very long time. Asking about impact gets us to the idea that the journalistic task might not be complete until a story changes something in the thoughts or actions of the user.

If journalism is supposed to inform, then one simple impact metric would ask: Does the audience know the things that are in this story? This is an answerable question. A survey during the 2010 U.S. mid-term elections showed that a large fraction of voters were misinformed about basic issues, such as expert consensus on climate change or the predicted costs of the recently passed healthcare bill. Though coverage of the study focused on the fact that Fox News viewers scored worse than others, that missed the point: No news source came out particularly well.

In one of the most limited, narrow senses of what journalism is supposed to do — inform voters about key election issues — American journalism failed in 2010. Or perhaps it actually did better than in 2008 — without comparable metrics, we’ll never know.

While newsrooms typically see themselves in the business of story creation, an organization committed to informing, not just publishing, would have to operate somewhat differently. Having an audience means having the ability to direct attention, and an editor might choose to continue to direct attention to something important even it’s “old news”; if someone doesn’t know it, it’s still new news to them. Journalists will also have to understand how and when people change their beliefs, because information doesn’t necessarily change minds.

I’m not arguing that every news organization should get into the business of monitoring the state of public knowledge. This is only one of many possible ways to define impact; it might only make sense for certain stories, and to do it routinely we’d need good and cheap substitutes for large public surveys. But I find it instructive to work through what would be required. The point is to define journalistic success based on what the user does, not the publisher.

Other fields have impact metrics too

Measuring impact is hard. The ultimate effects on belief and action will mostly be invisible to the newsroom, and so tangled in the web of society that it will be impossible to say for sure that it was journalism that caused any particular effect. But neither is the situation hopeless, because we really can learn things from the numbers we can get. Several other fields have been grappling with the tricky problems of diverse, indirect, not-necessarily-quantifiable impact for quite some time.

Academics wish to know the effect of their publications, just as journalists do, and the academic publishing field has long had metrics such citation count and journal impact factor. But the Internet has upset the traditional scheme of things, leading to attempts to formulate wider ranging, web-inclusive measures of impact such as Altmetrics or the article-level metrics of the Public Library of Science. Both combine a variety of data, including social media.

Social science researchers are interested not only in the academic influence of their work, but its effects on policy and practice. They face many of the same difficulties as journalists do in evaluating their work: unobservable effects, long timelines, complicated causality. Helpfully, lots of smart people have been working on the problem of understanding when social research changes social reality. Recent work includes the payback framework which looks at benefits from every stage in the lifecycle of research, from intangibles such as increasing the human store of knowledge, to concrete changes in what users do after they’ve been informed.

NGOs and philanthropic organizations of all types also use effectiveness metrics, from soup kitchens to international aid. A research project at Stanford University is looking at the use and diversity of metrics in this sector. We are also seeing new types of ventures designed to produce both social change and financial return, such as social impact bonds. The payout on a social impact bond is contractually tied to an impact metric, sometimes measured as a “social return on investment.”

Data beyond numbers

Counting the countable because the countable can be easily counted renders impact illegitimate.

- John Brewer, “The impact of impact

Numbers are helpful because they allow standard comparisons and comparative experiments. (Did writing that explainer increase the demand for the spot stories? Did investigating how the zoning issue is tied to developer profits spark a social media conversation?) Numbers can be also compared at different times, which gives us a way to tell if we’re doing better or worse than before, and by how much. Dividing impact by cost gives measures of efficiency, which can lead to better use of journalistic resources.

But not everything can be counted. Some events are just too rare to provide reliable comparisons — how many times last month did your newsroom get a corrupt official fired? Some effects are maddeningly hard to pin down, such as “increased awareness” or “political pressure.” And very often, attributing cause is hopeless. Did a company change its tune because of an informed and vocal public, or did an internal report influence key decision makers?

Fortunately, not all data is numbers. Do you think that story contributed to better legislation? Write a note explaining why! Did you get a flood of positive comments on a particular article? Save them! Not every effect needs to be expressed in numbers, and a variety of fields are coming to the conclusion that narrative descriptions are equally valuable. This is still data, but it’s qualitative (stories) instead of quantitative (numbers). It includes comments, reactions, repercussions, later developments on the story, unique events, related interviews, and many other things that are potentially significant but not easily categorizable. The important thing is to collect this information reliably and systematically, or you won’t be able to make comparisons in the future. (My fellow geeks may here be interested in the various flavors of qualitative data analysis.)

Qualitative data is particularly important when you’re not quite sure what you should be looking for. With the right kind, you can start to look for the patterns that might tell you what you should be counting,

Metrics for better journalism

Can the use of metrics make journalism better? If we can find metrics that show us when “better” happens, then yes, almost by definition. But in truth we know almost nothing about how to do this.

The first challenge may be a shift in thinking, as measuring the effect of journalism is a radical idea. The dominant professional ethos has often been uncomfortable with the idea of having any effect at all, fearing “advocacy” or “activism.” While it’s sometimes relevant to ask about the political choices in an act of journalism, the idea of complete neutrality is a blatant contradiction if journalism is important to democracy. Then there is the assumption, long invisible, that news organizations have done their job when a story is published. That stops far short of the user, and confuses output with effect.

The practical challenges are equally daunting. Some data, like web analytics, is easy to collect but doesn’t necessarily coincide with what a news organization ultimately values. And some things can’t really be counted. But they can still be considered. Ideally, a newsroom would have an integrated database connecting each story to both quantitative and qualitative indicators of impact: notes on what happened after the story was published, plus automatically collected analytics, comments, inbound links, social media discussion, and other reactions. With that sort of extensive data set, we stand a chance of figuring out not only what the journalism did, but how best to evaluate it in the future. But nothing so elaborate is necessary to get started. Every newsroom has some sort of content analytics, and qualitative effects can be tracked with nothing more than notes in a spreadsheet.

Most importantly, we need to keep asking: Why are we doing this? Sometimes, as I pass someone on the street, I ask myself if the work I am doing will ever have any effect on their life — and if so, what? It’s impossible to evaluate impact if you don’t know what you want to accomplish.

August 01 2011

14:00

Newsbeat, Chartbeat’s news-focused analytics tool, places its bets on the entrepreneurial side of news orgs

Late last week, Chartbeat released a new product: Newsbeat, a tool that takes the real-time analytics it already offers and tailors them even more directly to the needs of news orgs. Chartbeat is already famously addictive, and Newsbeat will likely up the addiction ante: It includes social sharing information — including detailed info about who has been sharing stories on Twitter — and, intriguingly, notifications when stories’ traffic patterns deviate significantly from their expected path. (For more on how it works, Poynter has a good overview, and GigaOm’s Mathew Ingram followed up with a nice discussion of the decision-making implications of the tool.)

What most stood out to me, though, both when I chatted with Tony Haile, Chartbeat’s general manager, and when I poked around Newsbeat, is what the tool suggests about the inner workings of an increasingly online-oriented newsroom. Chartbeat, the parent product, offers an analytic overview of an entire site — say, Niemanlab.org — and provides a single-moment snapshot of top-performing stories site-wide. Newsbeat, on the other hand, can essentially break down the news site into its constituent elements via a permissioning system that provides personalized dashboards for individual reporters and editors. Newsbeat allows those individual journalists to see, Haile notes, “This is how my story’s doing right now. This is how my people are doing right now.”

On the one hand, that’s a fairly minor thing, an increasingly familiar shift in perspective from organization to person. Still, though, it’s worth noting the distinction Newsbeat is making between news org and news brand. Newsbeat emphasizes the individual entities that work together, sometimes in sync and sometimes not so much, under the auspices of a particular journalistic brand. So, per Newsbeat, The New York Times is The New York Times, yes…but it’s also, and to some extent more so, the NYT Business section and the NYT Politics page and infographics and and blogs and Chris Chivers and David Carr and Maureen Dowd. It’s a noisy, newsy amalgam, coherent but not constrained, its components working collectively — but not, necessarily, concertedly.

That could be a bad thing: Systems that lack order tend to beget all the familiar problems — redundancy, wasted resources, friction both interpersonal and otherwise — that disorder tends to produce. For news orgs, though, a little bit of controlled chaos can be, actually, quite valuable. And that’s because, in the corporate context, the flip side of fragmentation is often entrepreneurialism: Empower individuals within the organization — to be creative and decisive and, in general, expert — and the organization overall will be the better for it. Analytics, real-time and otherwise, serve among other things as data points for editorial decision-making; the message implicit in Newsbeat’s design is that, within a given news org, several people (often, many, many, many people) will be responsible for a brand’s moment-by-moment output.

Which is both obvious and important. News has always been a group effort; until recently, though, it’s also been a highly controlled group effort, with an organization’s final product — a paper, a mag, a broadcast — determined by a few key players within the organization. News outlets haven’t just been gatekeepers, as the cliché goes; they’ve also had gatekeepers, individuals who have had the ultimate responsibility over the news product before it ships.

Increasingly, though, that’s not the case. Increasingly, the gates of production are swinging open to journalists throughout, if not fully across, the newsroom. That’s a good thing. It’s also a big thing. And Newsbeat is reflecting it. With its newest tool, Chartbeat is self-consciously trying to help organize “the newsroom of the future,” Haile told me — and that newsroom is one that will be dynamic and responsive and, more than it’s ever been before, collaborative.

July 20 2011

19:43

Downloads? Engagement? - Tablet advertising: which metrics matter?

emedia vitals :: Media companies and media buyers will have to think in new ways about measuring the effectiveness of tablet advertising. Metrics are clearly moving to be more about conversions and engagement (e.g. purchases made and time spent), but understanding and standards are still evolving. 

[Glenn Hansen, BPA Worldwide:] If you talk to the media owner, they want to live in the traditional space of subscriptions and copies and add up as much as they can. If you talk to the buy side, ... (t)hey want to know what actions have been taken and what content has been shared.

The current state  of tablet advertising - continue to read Ellie Behling, emediavitals.com

June 15 2011

16:00

Does a new report mean doom and gloom for local online news? Maybe, but here are a few balancing factors

Matthew Hindman’s new paper showing miserably low levels of local online news consumption is a terrific addition to research on how journalism gets produced and consumed online. He found, using panel data from comScore, that local news sites received, on average, only about three pageviews per person per week in their local markets.

And that’s in total, adding up all local news sites — individual sites fared even worse. The largest local news site in a typical market reached only about 17.8 percent of local web users in a given month, and it drew only about five minutes of the typical web user’s attention during that month.

Nikki Usher summarized the report’s findings for us in a separate post. But while Hindman’s research is a welcome reminder of local online news’ limitations and failings, I think there are a number of factors that complicate his findings a bit. Here are four reasons why I think the doom and gloom that I expect to circle around this report might not be spot on.

comScore’s dataset isn’t perfect

Among the various traffic-measurement firms, comScore has a very solid reputation. But it is also subject to some of the criticisms that have historically faced Nielsen’s TV ratings, most notably that their sample may not be a representative one. For example, comScore panel data doesn’t measure mobile traffic. And it likely undercounts web traffic from people at work, which Pablo Boczkowski and others have shown to be where a disproportionate amount of online news consumption occurs. (Hindman, to his credit, highlights these problems with comScore’s dataset.)

And, frankly, I wouldn’t be surprised if the kind of person willing to install comScore’s traffic-recording tool on their computers isn’t perfectly representative of the web-using public. The biggest news nerds might be underrepresented.

But the reality is these are quibbles, and Hindman’s larger point remains. Even if comScore is undercounting by a factor of three or four, we’re still talking about a small-numbers showing for local online media.

Reach does not mean impact

If raw readership totals equaled impact — on political discussion, on democracy, on the culture — then USA Today would be more important than The New York Times and Reader’s Digest would be more important than The New Yorker. Reaching the “right” people — and by that I mean the people who have disproportionate influence in political discussion, democracy, or culture — can make an outlet’s reach more potent than traffic numbers would suggest.

So for sites like MinnPost or Voice of San Diego, which write extensively about politics and local government, it’s possible to be both a must-read in the corridors of City Hall or the statehouse and still reach an audience that’s disproportionately influential.

Take MinnPost, for instance. According to Hindman’s analysis, 0.61 percent of all pageviews in the Minneapolis-St. Paul metro area went to local news sites. And only about 0.001 percent of total pageviews went to MinnPost (varying slightly by month).

But MinnPost’s Joel Kramer told us in March that, in January and February, MinnPost.com had received 921,000 visits. Each of those generated at least one pageview. The site has 5,700 daily email newsletter subscribers, 2,500 weekly email subscribers, and over 10,000 followers on Twitter.

In other words, while MinnPost may look like a rounding error in the overall scheme of Twin Cities web traffic, it is reaching many thousands of people. And even if those people are a small subset of the area population, a site like MinnPost can still have a significant positive impact on public affairs.

In Hindman’s previous book, The Myth of Digital Democracy, he advanced a largely similar argument based around political blogs. Here’s the book’s promo copy from its publisher:

Matthew Hindman argues that, though hundreds of thousands of Americans blog about politics, blogs receive only a miniscule portion of Web traffic, and most blog readership goes to a handful of mainstream, highly educated professionals. He shows how, despite the wealth of independent Web sites, online news audiences are concentrated on the top twenty outlets, and online organizing and fund-raising are dominated by a few powerful interest groups. Hindman tracks nearly three million Web pages, analyzing how their links are structured, how citizens search for political content, and how leading search engines like Google and Yahoo! funnel traffic to popular outlets. He finds that while the Internet has increased some forms of political participation and transformed the way interest groups and candidates organize, mobilize, and raise funds, elites still strongly shape how political material on the Web is presented and accessed.

You can see the DNA of that argument in the current paper. And I think Hindman’s right: If your goal for online media is to create a digital version of the New England town hall, then yes, you’re going to be disappointed that online media creates its own new class of media elites. But I’d argue that, if we’re judging online media’s value or worthiness to democracy, it’s important to do so through a lens that isn’t merely transposed from the days of big broadcast towers and giant metro newspapers.

I think, even taking Hindman’s facts on political blogs, that it’s impossible to argue that they haven’t had a significant impact on political discussion in America — despite their comparatively small readership and their power-law popularity structure. So I’d caution against anyone drawing similar conclusions based on this new paper about online media more broadly.

Local news does not equal “news”

It’s worth noting that, while the comScore data found little interest in local news, it did find substantially more interest in national and global outlets. Hindman’s analysis found that local news made up only about 19 percent of all pageviews to news sites measured. (The remainder is only defined as “nonlocal news sources,” but we can presume that a healthy chunk of that is made up of the big national news brands: CNN, MSNBC, Fox News, The New York Times, The Huffington Post, etc.)

The numbers were even more lopsided when you look at minutes spent rather than pageviews. Only about 15 percent of all time spent consuming online news was spent on local news sites.

While that’s not great news for local sites, it does indicate that people’s interest in online news more broadly isn’t in the same state of disrepair as their interest in local news. And it perhaps speaks to the wisdom of strategies that try to inject local news into national news brands — for instance, what MSNBC.com is doing with EveryBlock, or what AOL is trying to do in marrying HuffPo to Patch and Outside.in.

Watch your comparison sphere

One final note: Be cautious about reading too much into any statistics that look at online news as a fraction of total time spent online. That’s putting online news in competition not just with other traditional content sources, but with Gmail, and Facebook, and shopping on Amazon, and all the other bazillion things we do all day on the Internet. In other words, as more and more activities that traditionally took place outside the browser move inside it, it only makes sense that online news’ share might not keep up — even if online news consumption were held constant. (For example, if online news reading went up 10 percent, but total online usage went up 100 percent, online news as a share of online activity would drop — even though people were consuming more online news. Time spent shopping for shoes at Zappos shouldn’t count against time spent reading local headlines.

Again, that’s not to invalidate (or even to argue against) Hindman’s findings; his raw numbers of minutes spent are plenty low enough on their own, even without any comparison to the rest of the web. But if we are going to judge online news consumption, let’s use the numbers that make the most sense.

Separately, because the FCC-funded research process is pleasantly open, you can read Hindman’s initial draft of his paper, a peer review of it by Iris Chyi at the University of Texas, Hindman’s response to Chyi’s remarks, and the final version. Probably only of interest to the nerdiest of news nerds, but Chyi raises some good points.

March 02 2011

15:00

Dennis Mortensen: Are news orgs worrying too much about search and not enough about the front page?

Editor’s Note: This is a guest post from Dennis R. Mortensen, former director of data insights at Yahoo and founder of Visual Revenue, a New York firm that sells its predictive-analytics services to news organizations.

Dennis saw my talk at the Canadian Journalism Foundation in January and wanted to comment on the role of a news site’s front page in its success in assembling an audience. He argues that paying too much attention to SEO on current articles could backfire on news orgs.

In Josh’s talk in Toronto, he hypothesized that:

[The front page is] still an enormously powerful engine of traffic. I would say actually that for most American newspapers…it’s probably 70 percent in a lot of cases.

Josh is saying you should view the front page as a traffic channel unto itself, just as you’d think of Facebook or Google — something I wholeheartedly agree with. If you choose to view your front page as a traffic channel, you’ll also sign up for a different kind of data analysis — analysis that mixes external referrers with internal referrers. In other words, a link from the Drudge Report is no different than a link from your own front page, in the sense that they both should be viewed as channels to optimize.

I argue that the front page is the most important engine of traffic for news media publishers today. I would also argue that this whole notion of news media publishers being held hostage by Google — and the slightly suboptimal idea of optimizing your articles for search to begin with — is somewhat misguided. It certainly seems wrong when we look at the data.

In this analysis, it’s important to distinguish between two core segments: current article views and archived article views. To begin, I’ve chosen a set of very strict and non-favorable definitions to my conclusion. A current article is defined as an item of content that is directly being exposed on the front or section front page right now. Any other content not currently exposed on a front page or section front page is deemed to be an archived article.

We looked at a sample of about 10 billion article views, across a sample of Visual Revenue clients, and found that on any given day, 64 percent of views are on current articles, and 36 percent of views are on archived articles.

So on a typical day, for most if not all news media publishers, the largest portion of article views comes off of their current article segment — stories published today or perhaps yesterday and still being actively promoted. I find this analysis fascinating and almost empowering, for the simple reason that most current news events are initially non-searchable. If a revolution breaks out in Egypt, I won’t know until I’m told about it. Non-searchable translates into a need for the stories to be discoverable by the reader in a different way, such as on a front page, through RSS, or in their social stream — all channels the publisher either owns or can influence.

There is no doubt that search, as a channel, owns the archive. One can discuss the data of why that is and why it is or isn’t optimal — I’ll leave that for a later discussion. But today, let’s focus on the current article segment, by far the bigger of the two. Where do those views come from? Looking at the same dataset from our clients, we get a very clear indication of where one’s focus should lie:

Sources of current article views:

78 percent come from front pages
7 percent come from search
6 percent come from social media
5 percent come from news aggregators
3 percent come from news originators
1 percent come from RSS & email

(We’re defining “news originators” as sites where at least two-thirds of the stories excerpted and promoted on their front page are original pieces generated by the news organization — which includes most traditional media. “News aggregators” are sites where less than two-thirds are original, such as Google News, Techmeme, or Drudge.)

I doubt this front-page dominance is much of a surprise to most editors — but for some reason, it seems like we aren’t taking the appropriate action today. We have 78 percent of all views on current articles coming from the front page — that’s 49 percent of all your article views on any given day — and what do we do to optimize it? And why is it that so many news organizations think immediately of search when we write a new story, when search has minimal initial impact? Even worse, writing an SEO-optimized article excerpt title for your front page probably deoptimizes your results on current articles.

The front page is indeed still an enormously powerful engine of traffic. We now know that about half of your article views can be attributed to the primary front page or the section front pages — and with it a huge chunk of any news organization’s online revenue. The question, then, is what kind of processes and optimization methodologies have you applied to take advantage of this fact?

January 10 2011

18:00

Applying analytics to magazine sales at Hearst

It’s safe to say that analytics are no longer online news’ equivalent of fantasy baseball circa 1998: relegated only to the geekiest corners of the operation. News editors and executives not only know what kind of hourly, daily, and monthly traffic their sites get, but they can tell you who’s their best referrer and what stories will predictably provide a pageview explosion. (Of course there’s still a debate on what role those metrics should play for news organizations.)

But what about on the print side? Specifically magazines: When it comes to newsstand distribution, the numbers are largely a guessing game. And when you’re spending millions on printing and delivering magazines, that’s an expensive guessing game.

“There’s a lot of waste, in a sense,” said Charlie Swift, vice president of database strategy and marketing for Hearst Magazines. “You may put out 10 copies, sell 4 and 6 get tossed.” The flip side, of course, being when you don’t stock enough magazines at locations where they sell out. Distribution has largely been based off previous sales at given locations, but variables can come into play, such as where magazines are placed (near the checkout vs. on a newsstand) and demographics in an area (or as Swift told me: “If you’re selling Cosmopolitan in a retirement community, that’s probably not a good idea.”).

“We believe there’s a way to make [distribution] more efficient by putting analytics behind it,” Swift told me.

All of this is why Hearst held a competition to see who could build a better analytics model to predict newsstand sales for magazines. It’s a Netflix Prize-like approach to figuring out how well a magazine will sell at a given location, based off historical sales data. There were more than 700 participants in the competition, and last month Hearst and the Direct Marketing Association awarded a team $25,000 for their formula, which generated magazine sales estimates that were nearest to actual newsstand figures. Swift said they’re looking at how they can adapt the model to distributing different titles.

They’re hoping analytics can be a way of becoming more efficient and saving money — starting by wasting less paper. According to figures provided for the competition by the National Center for Database Marketing, less than 40 percent of all copies are sold, and the magazine industry saw close to a six percent drop in newsstand sales between 2009 and 2010. Swift said using analytics benefits readers “if we can be more efficient as an industry and take costs out that are not adding to the customer.”

What’s almost ironic about magazines getting turned on to analytics is that they’re historically known for holding on to information about readers from home subscriptions, direct mail, and surveys. Swift knows this better than most; his job is a relatively new position created at Hearst specifically to look at metrics. Like many organizations, Hearst’s magazine division is in the process of building a unified customer database that ties together information like bill payments, web visits, and promotions received.

Aside from the economic benefits of getting better data, Swift said he thinks magazine publishers realize the cost of storing and managing information has gone down. They’re also taking notice of how analytics are used in making decision for news sites. “The web changes the culture too,” he said. “There’s too much information to absorb — you have to process it, have to let analytics drive it in ways.”

Photo by YoungDoo Moon used under a Creative Commons license.

November 18 2010

17:30

Crunching Denton’s Ratio: What’s the return on paying sources?

There was a lot of buzz on Twitter yesterday about Paul Farhi’s piece in The Washington Post on checkbook journalism — in particular the way a mishmash of websites, tabloids, and TV news operations put money in the hands of the people they want to interview. (With TV, the money-moving is a little less direct, usually filtered through payments for photos or videos.)

But, just for a moment, let’s set aside the traditional moral issues journalists have with paying sources. (Just for a moment!) Does paying sources make business sense? Financially speaking, the justification given for paying sources is to generate stories that generate an audience — with the hope that the audience can then be monetized. Does it work?

There’s not nearly enough data to draw any real conclusions, but let’s try a little thought experiment with the (rough) data points we do have, because I think it might provide some insight into other means of paying for content. Nick Denton, the head man at Gawker Media and the chief new-media proponent of paying sources, provides some helpful financial context:

With the ability to determine instantly how much traffic an online story is generating, Gawker’s Denton has the pay scale almost down to a science: “Our rule of thumb,” he writes, “is $10 per thousand new visitors,” or $10,000 per million.

What strikes me about those numbers is how low they are. $10K for a million new visitors? There aren’t very many websites out there that wouldn’t consider that an extremely good deal.

Let’s compare Denton’s Ratio to the numbers generated by another money-for-audience scheme in use on the Internet: online advertising. After all, lots of ads are sold using roughly the same language Denton uses: the M in CPM stands for thousand. Except it’s cost per thousand impressions (a.k.a. pageviews), not cost per thousand new visitors, which would be much more valuable. What Denton’s talking about is more like CPC — cost per click, which sells at a much higher rate. (Those new visitors aren’t just looking at an ad for a story; they’re actually reading it, or at least on the web page.) Except it’s even more valuable than that, since there’s no guarantee that the person clicking a CPC ad is actually a “new” visitor. Let’s call what Denton’s talking about CPMNV: cost per thousand new visitors.

CPC rates vary wildly. When I did a little experiment last year running Google AdWords ads for the Lab, I ended up paying 63 cents per click. I ran a similar experiment a few months later with Facebook ads for the Lab, and the rate ended up being 26 cents per click.

What Denton is getting for his $10 CPMNV is one cent per click, one cent per new visitor. It’s just that the click isn’t coming from the most traditional attention-generating tool, an ad — it’s coming from a friend’s tweet, or a blogger’s link, or a mention on ESPN.com that sends someone to Google to search “Brett Favre Jenn Sterger.”

Doing the pageview math

And that $10 CPMNV that Denton’s willing to pay is actually less than the return he gets for at least some of his source-paid stories. Take the four Gawker Media pieces that the Post story talks about: the original photo of singer Faith Hill from a Redbook cover, to show how doctored the image was for publication; photos and a narrative from a man who hooked up with Senate candidate Christine O’Donnell; the “lost” early version of the iPhone 4 found in a California bar; and voice mails and pictures that allegedly show quarterback Brett Favre flirting with a woman named Jenn Sterger, who is not his wife. Gawker publishes its pageview data alongside each post, so we can start to judge whether Denton’s deals made financial sense. (Again, we’re talking financial sense here, not ethical sense, which is a different question.)

Faith Hill Redbook cover: 1.46 million pageviews on the main story, and about 730,000 pageviews on a number of quick folos in the days after posting. Total: around 2.2 million pageviews, not to mention an ongoing Jezebel franchise. Payment: $10,000.

Christine O’Donnell hookup: 1.26 million pageviews on the main story, 617,000 on the accompanying photo page, 203,000 on O’Donnell’s response to the piece, 274,000 on Gawker’s defense of the piece. Total: around 2.35 million pageviews. Payment: $4,000.

“Lost” iPhone: 13.05 million pageviews on the original story; 6.1 million pageviews on a series of folos. Total: around 19.15 million pageviews. Payment: $5,000.

Brett Favre/Jenn Sterger: 1.73 million pageviews on the first story, 4.82 million on the big reveal, 3.99 million pageviews on a long line of folos. Total: around 10.54 million pageviews. Payment: $12,000.

Let’s say, as a working assumption, that half of all these pageviews came from people new to Gawker Media, people brought in by the stories in question. (That’s just a guess, and I suspect it’s a low one — I’d bet it’s something more like 70-80 percent. But let’s be conservative.)

Expected under the Denton formula:
Faith Hill: 1 million new visitors
O’Donnell: 400,000 new visitors
iPhone: 500,000 new visitors
Favre: 1.2 million new visitors

Guesstimated real numbers:
Faith Hill: 1.1 million new visitors
O’Donnell: 1.17 million new visitors
iPhone: 9.56 million new visitors
Favre: 5.27 million new visitors

Again, these are all ham-fisted estimates, but they seem to indicate at least three of the four stories significantly overperformed Denton’s Ratio.

Reaching new audiences

The primary revenue input for Gawker is advertising. They don’t publish a rate card any more, but the last version I could find had most of their ad slots listed at a $10 CPM. Who knows what they’re actually selling at — ad slots get discounted or go unsold all the time, many pages have multiple ads, and lots of online ads get sold on the basis of metrics other than CPM. But with one $10 CPM ad per pageview, the 2.2 million pageviews on the Faith Hill story would drum up $22,000 in ad revenue. (Again, total guesstimate — Denton’s mileage will vary.)

Aside: Denton has said that these paid-for stories are “always money-losers,” and it’s true that pictures of Brett Favre’s manhood can be difficult to sell ads next to. Most (but not all) of those 10.54 million Brett Favre pageviews were served without ads on them. But that has more to do with, er, private parts than the model of paying sources.

But even setting aside the immediate advertising revenue — the most difficult task facing any website is getting noticed. Assuming there are lots of people who would enjoy reading Website X, the question becomes how those people will ever hear of Website X. Having ESPN talk about a Deadspin story during Sportscenter is one way. Having that Redbook cover emailed around to endless lists of friends is another. Gawker wants to create loyal readers, but you can only do that from the raw material of casual readers. Some fraction of each new flood of visitors will, ideally, see they like the place and want to stick around.

Denton publishes up-to-date traffic stats for his sites, and here’s what the four in question look like:

It’s impossible to draw any iron-clad conclusions from these squiggles, but in the case of Jezebel and Deadspin, the initial spike in traffic appears to have been followed by a new, higher plateau of traffic. (The same seems true, but to a lesser extent, for Gizmodo — perhaps in part because it was already much more prominent within the gadget-loving community when the story broke than, for example, 2007-era Jezebel or 2010-era Deadspin were within their target audiences. With Gawker, the O’Donnell story is too recent to see any real trends, and in any event, the impact will probably be lost within the remarkable overall traffic gains the site has seen.)

Fungible content strategies

I’ve purposefully set aside the (very real!) ethics issue here because, when looked at strictly from a business perspective, paying sources can be a marker for paying for content more generally. From Denton’s perspective, there isn’t much difference between paying a source $10,000 for a story and paying a reporter $10,000 for a story. They’re both cost outputs to be balanced against revenue inputs. No matter what one thinks of, say, Demand Media, the way they judge content’s value — how much money can I make off this piece? — isn’t going away.

Let’s put it another way. Let’s say a freelance reporter has written a blockbuster piece, one she’s confident will generate huge traffic numbers. She shops it around to Gawker and says it’ll cost them $10,000 to publish it. That’s a lot of money for an online story, and Denton would probably do some mental calculations: How much attention will this story get? How many new visitors will it bring to the site? What’s it worth? I’m sure there are some stories where the financial return isn’t the top factor — stories an editor just really loves and wants to publish. But just as the Internet has turned advertising into an engine for instantaneous price matching and shopping into an engine for instantaneous price comparison, it breaks down at least some of the financial barrier between journalist-as-cost and source-as-cost.

And that’s why, even beyond the very real ethical issues, it’s worth crunching the numbers on paying sources. Because in the event that Denton’s Ratio spreads and $10 CPMNV becomes a going rate for journalists as well as sources, that means for a writer to “deserve” a $50,000 salary, he’d have to generate 5 million new visitors a year. Five million is a lot of new visitors.

There’s one other line Denton had in the WaPo piece that stood out to me:

“I’m content for the old journalists not to pay for information. It keeps the price down,” Denton writes in an exchange of electronic messages. “So I’m a big supporter of that journalistic commandment – as it applies to other organizations.”

When we think of the effects of new price competition online, we often think of it driving prices down. When there are only a few people competing for an advertising dollar, they can charge higher rates; when there are lots of new competitors in the market, prices go down. But Denton’s basically arguing the equally logical flipside: I can afford to pay so little because there aren’t enough other news orgs competing for what sources have to offer. Let’s hope we don’t get to that same point with journalists.

October 29 2010

14:00

How does audience engagement work in the newsroom?

So…how’s that Twitter thing working out for you? I’m sure American Public Media will be less glib than that when asking journalists how audience engagement works for them.

APM’s Public Insight Network is surveying journalists about their methods of reaching out to readers, but perhaps more importantly, asking them if they think it’s doing them any good.

The survey was launched last week and the Public Insight Network is hoping to poll the most connected journalists they can find this weekend at the Online News Association conference in Washington, D.C. (Though anyone can take the survey online.) They plan to produce a white paper with the findings and potentially find new partners for the expanding network.

I emailed Andrew Haeg, editor of the Public Insight Network, to ask why they want to examine engagement now and why tap ONA. “ONA has become the go-to conference for journalists searching for new ways to create distinctive content that cuts through the noise of the Internet, and audience engagement has emerged as a major piece of any news operation’s efforts to stand out online,” Haeg wrote.

In their zeal to get involved in social media, Haeg said news outlets have taken a “shoot first, ask questions later’ kind of approach. Now that more journalists have audience engagement experience under their belt, we’re curious to find out how efforts are measuring up.”

This seems to jibe with recent signs that more newspapers, magazines and other news outlets are past the introduction phase with social media, as well as the fact that both Twitter and Facebook have people dedicated to working with news organizations. At the same time a number of media start-ups, including TBD, the Honolulu Civil Beat, and Voice of San Diego have made audience engagement a priority from launch.

The insight network’s open-ended survey asks basic questions about journalists’ familiarity and comfort using Twitter, Facebook, and other platforms to reach out to readers. The big question appears to be whether audience engagement is important and a worthwhile use of time. Haeg said they also want to find out what the expectations are when it comes to online outreach and how journalists gauge success in engagement.

“Is it primarily a way to drive traffic?” Haeg asked. “Does it afford journalist the chance to gather new information, get in touch with sources they otherwise couldn’t have, and seek out new stories? Does it feel like a lot of busy work that’s not adding up to much?”

In particular, the question of “what is engagement” may be of interest to future-of-news watchers becoming skeptical of what the word means in relation to news. Is it just using Twitter and Facebook, is it talking to readers in comments, is it publishing user-created content? In looking at what mainstream media outlets are successful in social media engagement, ReadWriteWeb based its recent findings on Postrank’s analysis, which studies RSS feed items and tallies comments, bookmarks, Diggs, and mentions on Twitter. Over at the Reynolds Journalism Institute, Joy Mayer is taking the long view on engagement, examining how various news outlets define the term.

Haeg said they plan to release the white paper after ONA and ask respondents to test out new engagement tools the insight network creates.

October 28 2010

16:00

Engagement: Where does revenue fit in the equation?

Our post earlier this week about philly.com’s seven-part equation to measure user engagement has sparked a lively debate in the comments. The central question: Should a news site’s engagement equation factor in revenue? If so, how?

To recap, Philly.com’s equation puts a numerical value on user engagement by calculating what percentage of the site’s users fulfill certain criteria, including viewing multiple pages, spending more than six minutes on the site, leaving comments, sharing content through social media, or returning regularly. The equation allows philly.com to track how the site is doing in terms of these individual categories of engagement, as well as averaging them out to obtain an overall engagement percentage.

But several people, starting with Sonia Meisenheimer of the St. Petersburg Times in the post itself, questioned the usefulness of an equation that doesn’t take into account how user behavior affects the news organization’s bottom line. Ravi Pathak propsed adding revenue as a coefficient, and Ophir Prusak suggested that factoring in banner-ad clickthrough rates might be a good way to do this. Jim Novo put a different spin on the question, asking:

Another way to say this: does it matter more to you what kind of content engaged visitors in the past, or what kind of content attracts visitors who are likely to remain engaged in the future?

He suggested focusing exclusively on “recency,” or the likelihood that a user will visit the site again, as an engagement metric that speaks more directly to revenue.

But Eric T. Peterson, the author of the white paper upon which philly.com’s engagement equation was based, jumped in to defend the idea of keeping revenue calculations separate from engagement:

Even in the media and publishing model, engagement and revenue are different aspects of consumer behavior. A consumer can be very engaged with your site but not be tremendously profitable…but you still want a way to measure their engagement independent of profit. Same for satisfaction and engagement — they are different aspects of the consumer experience…Work to understand what my (or any) engagement metric can tell you about your audience first, then go looking for the relationship between engagement and money.

More than a dozen people have chimed in on the issue, and the debate is still going on. You can read the full thread here.

October 26 2010

14:00

Getting beyond just pageviews: Philly.com’s seven-part equation for measuring online engagement

As web analytics reports become a mainstay of news meetings, there’s a lot of nervousness about how attention to clicks will affect news coverage, and about the perceived incentives to produce high-trafficking junk news. Earlier this week, a web research company released a good-news study arguing that stories about substantive issues like unemployment and mortgage rates can actually bring in more revenue per pageview than celebrity crotch shots.

But two months ago, philly.com, home of the Philadelphia Inquirer and Daily News, began analyzing their web traffic with an “engagement index” — an equation that goes beyond pageviews and into the factors that differentiate a loyal, dedicated reader from a fly-by. It sums up seven different ways that users can show “engagement” with the site, and it looks like this: Σ(Ci + Di + Ri + Li + Bi + Ii + Pi)

I spoke with Chris Meares, senior data analyst at Philly.com, about how the equation works, and what it’s revealed so far about the newspaper’s users on the web. The first step in measuring engagement, Meares explained, is identifying which web behaviors show that users are “engaged.”

Working off a white paper called “Measuring the Unmeasurable: Visitor Engagement” by Eric T. Peterson and Joseph Carrabis, Meares sat down with Ryan Davis, the president of Philly.com, and Wendy Warren, the vice president for content, to hash out the meaning of engagement.

One possibility they considered was measuring engagement simply through how many visitors left comments or shared philly.com content on a social media platform. But that method “would lose a lot of people,” Meares said. “A lot of our users don’t comment or share stories, but we have people — 45 percent — [who] come back more than once a day, and those people are very engaged.”

They ultimately decided on seven categories, each with a particular cutoff:

Ci — Click Index: visits must have at least 6 pageviews, not counting photo galleries

Di — Duration Index: visits must have spend a minimum of 5 minutes on the site

Ri — Recency Index: visits that return daily

Li — Loyalty Index: visits that either are registered at the site or visit it at least three times a week

Bi — Brand Index: visits that come directly to the site by either bookmark or directly typing www.philly.com or come through search engines with keywords like “philly.com” or “inquirer”

Ii — Interaction Index: visits that interact with the site via commenting, forums, etc.

Pi — Participation Index: visits that participate on the site via sharing, uploading pics, stories, videos, etc.

Those are largely the same as in the Peterson/Carrabis white paper, although they use a Feedback Index (“captures qualitative information including propensity to solicit additional information or supply direct feedback”) in place of Philly.com’s Participation Index.

The next step was to track the percentage of overall visits that satisfy each of these categories. What percent of visits to the site lasted at least five minutes? What percent included a comment or other interaction? For instance, in a recent measure of the Click Index, out of 3.9 million total visits, 698,000 were visits where a user clicked through at least six pages — which comes out to a 17.9 percent engagement rate.

As well as paying attention to the engagement rates for each category, Meares also averages the seven individual percentages to create an overall engagement score — the average percent of visits to the site that broadly qualify as “engaged.”

Because the engagement percentage typically goes down as total site pageviews go up (new visitors are, by definition, not loyal ones), Meares multiples the overall engagement percentage by the total number of pageviews to get a estimate of the total number of engaged visits.

Last week, the overall engagement was 31 percent, which translates into an estimated 1.221 million engaged visits.

Month to month, the overall engagement score for philly.com has hovered around 35 percent, Meares said. He also tracks the levels of engagement for different areas of the site, including news, sports, and living. While he said the sports score isn’t actually as high as the 73-percent figure a Philly.com exec gave us last week, it is higher than the average engagement level across the site. In September, sports page visits were 46.6 percent engaged, while news page visits were only 34.4 percent engaged.

Tracking site visits with this level of specificity is time-consuming — Meares says he devotes about a third of his full-time job to analysis of the engagement equation — but it has produced some interesting information. For instance: “We’re definitely seeing the impact of social media and how it provides engaged visitors.” While Google and Yahoo provide a lot of traffic, the visits that they send to Philly.com don’t tend to be engaged. Only 20.34 percent of visits that come through Google are engaged visits. In comparison, 33.64 percent of visits that come via Facebook are engaged.

More broadly, Meares said, tracking engagement allows Philly.com to put traffic data in perspective. If overall traffic for the site is down, but the number of engaged users are up, that still means the site is doing well, Meares said.

Newspapers in other markets have come to Philly.com for advice about measuring engagement on news sites. “It’s a really big challenge,” said Sonia Meisenheimer, digital marketing strategist for the St. Petersburg Times’ tampabay.com. “There’s not an expressed moment of loyalty. You have to basically be a diviner with a divining rod and go around and tap on all these different things.”

Meisenheimer said she found philly.com’s engagement equation “fascinating” but impractical for her requirements. “They can do this because they have a web analytics person full time looking at ‘how do we measure success?’” she said. “We could never sustain the reporting and tracking.”

While the equation might provide interesting feedback to editors, Meisenheimer said she thought the results it produced were too complicated. In a competitive market, businesses want to compare different web outlets on apples-to-apples factors like total audience and local audiences. She said tampabay.com would measure engagement in a much simpler way, focusing on registration numbers and their brand index, or how many people come to the site through a bookmark or by searching for terms like “St. Pete Times” or “tampabay.com.”

Internally, she said the most important number for tampabay.com is simply revenue per unique visitor. “My question to philly.com is: how does this help you make more money? Because I don’t see that in the equation.”

October 25 2010

14:00

Building a university sandbox for news orgs: UNC’s new digital newsroom nearing Nov. 1 launch

A journalism school launching an outward-facing online news outlet is nothing new these days, as more schools are creating in-house laboratories for students to learn online skills. Next Monday, though, there will be a new and interesting entrant to the field: The University of North Carolina’s Reese Felts Digital News Project is launching a news site Nov. 1 that will cover the campus and region with a 21-student staff. What makes the project different is its secondary purpose: It wants to be an R&D lab for the news industry, using its students as testing grounds for new ideas while sharing the results with the rest of us. Essentially, the students are taking requests.

Monty Cook, the project’s executive producer who had been senior vice president and editor of The Baltimore Sun and baltimoresun.com, describes a dual imperative to both give students needed skills while pushing the industry forward. The editorial focus will be on long-form journalism, particularly investigative journalism in many formats, including documentary videos and data visualization. To create and present that content, the students will be cooperating with outside companies that need a sandbox. There’s space set aside so that a browser plug-in, news application, or emerging social media platform could test a product on the site for 30 to 60 days, using the staff in a trial run.

“We’re not here to make anyone money,” Cook said. “But if we can help provide greater understanding not only for ourselves and our students but for the companies that are working hard to make the transition, then we should do that.”

That industry-aiding focus means openness. Cook said they will open-source the WordPress theme they built for the site’s back end, and iPhone and Android apps will also be available. And unlike news organizations that play stingy with their internal metrics, Cook said they will be willing to share the site’s Omniture numbers. That could come in handy as the students experiment with alternate forms of storytelling or reporting, as those lessons would be shared through a research component of the site. Cook said students will be reflecting on their successes and failures, while other UNC faculty members will contribute their thoughts and research. Some news organizations have already asked if Reese Felts students could eventually train their journalists.

As with any startup, the initial version will lack many of the features Cook visualizes down the road. The staff — 19 undergraduates and two graduate students, all paid a stipend, plus freelancers and volunteers — still needs to learn some of the skills they’ll need to produce ambitious content, Cook said. And as of now, don’t expect any revolutionary business ideas. The site will not have advertising, and is paid for by a major gift from late UNC alumnus Reese Felts.

The largest news organizations can afford their own R&D efforts and can try fresh ideas on their own. But a radio station without the resources to build a mobile app could watch as the students fine-tune theirs, or a mid-sized newspaper can observe what Cook says are exciting ideas on how to moderate discussions. The key, Cook said, is that the program is considered an audience research lab first, news organization second. And, incidentally, the students will get to learn some new skills, too. “We’re looking to do experimental digital news, and that means getting them to think differently about their approach to both newsgathering and news dissemination,” Cook said.

October 19 2010

14:00

NAA finds a more favorable website stats vendor — but misses the readership shift to mobile news

When last we checked on the Newspaper Association of America’s web stats (and other data) back in April, the monthly website usage information that the nation’s daily newspaper organization was publishing came from Nielsen Online, and it wasn’t all that pretty.

The NAA tried to put the best spin on the data, but as we pointed out at the time, time spent at newspaper sites was in the doldrums and getting gradually worse, with three of the seven shortest attention spans measured by Nielsen occuring in the first quarter of 2010: 34:10 minutes in January, 31:39 minutes in February, and 32:21 minutes in March. For context, consider that at the time, also according to Nielsen, the average Facebook user was spending nearly seven hours on the social networking site.

It looks like NAA was not happy with those first quarter web stats. It published April data from Nielsen but offered no further updates for four months. At that point, I inquired whether NAA had decided to stop publishing the data, and was informed by Jeff Sigmund, director of communications, that “a new methodology” was in the works.

The new methodology turns out be be Comscore. Last Thursday, NAA posted Comscore data for September, and simultaneously wiped all the old Nielsen data off its site. The reason for the switch is clear: Comscore’s results are more favorable to newspapers than Nielsen’s in several categories, as trumpeted in an NAA press release.

As part of the switch, Comscore provides NAA with data specific demographic slices, something it didn’t get from Nielsen. For example, it reports: “More than 55 percent of adults in the 25-to-34-year-old demographic visited a newspaper website in September. During that same time period, 52 percent of this age group visited Yahoo! News Network, 22 percent visited CNN and 24 percent visited MSNBC.” We’d love to drill a little deeper there, but that particular comparison seems to suggest that, with no real statistical difference between Yahoo News and the entire newspaper business in reach among 25-to-34-year-olds, Yahoo offers a vastly more efficient one-stop ad buy than newspapers do.

Similarly, NAA says Comscore found that “one-in-four (25 percent) of adult newspaper website visitors come from households earning at least $100,000 a year, compared to 21 percent of all adult Internet users.” News consumption and newspaper readership have always skewed a bit toward higher income strata, so that’s not surprising — in fact, that four percent advantage is not particularly impressive, and here again, advertisers can easily find sites that are far more efficient in reaching high-income consumers.

When compared with the old Nielsen data, the benefits of the vendor switch are obvious. In March, Nielsen found 72.1 million unique visitors, which was about equal to the average for the previous 9 months. But in September, Comcast identified 102.8 million UVs, “almost two-thirds (61 percent) of all adult Internet users” according to the NAA release. (The release explains that Comcast is now NAA’s preferred methodology because “it more accurately reflects the true size of the newspaper Web home and at-work online audience,” but it offers no explanation of the major discrepancies between the old and new systems. “More accurately” seems to mean “they found more of them.”)

Time spent, or engagement, is the metric that matters most to advertisers these days. Unique visitors, no matter how impressive a slice of the total web audience they represent, don’t deliver customers to advertisers. They key is whether site visitors are engaging — interacting — with the content and the advertising on the site, and that kind of engagement still eludes most online newspapers.

The NAA release says “the findings point to engagement,” without putting that engagement in context or mentioning the specific per user/per visit “time spent” portion of Comscore’s findings, which are posted in the “Trends and Numbers” section of NAA’s site. There, Comscore reports 3.8 minutes per September visit, and 8.5 visits per visitor, for the month. That’s 32.3 minutes per visitor, total, for the month of September — a result slightly below the first-quarter average of 32 minutes, 43 seconds found by Nielsen. So in the engagement metric, “more accurate” means “almost exactly the same.”

So, while it’s no doubt statistically invalid to directly compare the Nielsen and Comcast findings, what NAA seems to have bought by switching is a larger audience spending about the same amount of time per visitor. And that “time spent” by either method boils down to an average of about a minute per user per day — still only about one half of one percent of all time spent online.

Comscore measured a total of 3.3 billion minutes spent at (U.S.) newspaper websites (compared to an average of 2.3 billion in the first quarter — the increase being due to the improved UV count) which sounds like a lot until you consider that (in August, and globally) we collectively spent 41.1 billion minutes at Facebook. Those figures should not be directly compared, but it’s clear that newspapers, collectively, enjoy just a fraction of the attention the top social network commands.

Perhaps more important than all of this statistical nitpicking, though, is the fact that while the NAA fiddles with methodology and stat sourcing, the audience is in the middle of another major shift in its digital news consumption from web browsers to mobile platforms — smartphones, e-readers and tablets. By sometime in 2012, cumulative sales of iPads, alone, will likely exceed the number of home-delivered newspaper subscriptions. The majority of mobile content consumption is likely to fall on the leisure side of the consumer’s online time — enabling deeper, longer engagement than the fleeting workday news consumption being measured by Nielsen and Comscore. Many newspapers are in fact working to follow their audience in this shift (though few are leading them there), but NAA is still stuck in an earlier mode of touting questionable browser traffic stats.

NAA’s vendor switch may mean a somewhat more impressive monthly press release, but it misses the mark of what it should really be doing to help its members find a digital audience. It’s time for NAA to dig much more deeply and broadly into this shifting landscape — to begin measuring and comparing news consumption in print, in broadcast, on computers and on mobile devices. So far, few news media have found ways to make their digital versions profitable, let alone self-sustaining in the absence of their legacy print versions. Will they ever get there, when their national trade organization is still refining “methodology” for counting its web browser audience, while that very audience is rapidly shifting its consumption to a whole new generation of digital devices in which usage of apps, rather than websites, is what counts?

Image by Eric Skiff used under a Creative Commons license.

October 08 2010

16:00

Twitter data lets NPR glimpse a future of app-loving news junkies

Conventional wisdom tells us that if you hook up your website to Twitter and/or Facebook you should see some increase in online traffic.

But beyond more eyeballs and pageviews, what’s the value of all those followers and “likes” to a media organization? Particularly if you’re a giant like NPR?

In surveying their Twitter followers (and previously Facebook fans), NPR got more than just handy metrics to let them know they’ve got a good grasp on social media: They got a glimpse at their future, a population of news junkies who live in a world of apps that allow them to consume, share and interact with information.

“Gadgets and news consumption are a major part of Twitter in general, so it makes sense we’re a subset of those activities,” said Andy Carvin, NPR’s senior strategist for social media.

It also makes sense that NPR wants to monitor its emerging platforms as they try to transform into a digital media company. Facebook and Twitter combined now account for 7-8 percent of traffic to NPR.org, an amount that has doubled in the last year. The survey provides a measure of context to those traffic numbers. Carvin said Twitter links on average account for 500,000 pageviews per month, while Facebook averages closer to 3 million.

Here’s what we know from the numbers: People who follow NPR are bullish on social media (they follow more than one account connected to the company); they want live or breaking news delivered to them; they primarily get their news online. Anecdotally all of this may seem unsurprising — if you’re on Twitter there’s a strong likelihood you know your way around an iPhone and/or the Internet.

But the value here is that NPR now knows what its audience of the future looks like. While average public radio listeners are well into their 50s and tune in to NPR four hours a week, NPR’s typical Twitter and Facebook fans are in their 30s and listen on average two hours a day, Carvin said. “This gives us a great opportunity to reach out to audiences that are younger than our radio audience and hopefully retain them over the next several decades,” Carvin said.

By breaking down the data, they’ll be able to fine tune how they use Twitter and Facebook — from choosing which types of stories to post (breaking or topical news for Twitter vs. features and conversations starters on Facebook), to when to post content, to how much to encourage the staff to keep at it.

This all comes at a time when some say NPR is awakening the sleeping giant by becoming a cross-platform company that reaches from radio, to mobile devices and online media. In 2008 NPR received grant funding from the Knight Foundation for the specific purpose of bringing the staff up to speed on new media, including blogging, multimedia storytelling, and social networking. “Having data like this along with encouragement from peers who use social media will help us move forward and expand social media literacy in our reporting capacity,” he said.

Though NPR has been prolific in its creation of Twitter accounts and Facebook pages, Carvin said the reporters, producers and hosts behind the scenes need some confirmation that what they’re doing is paying off. “It certainly didn’t hurt that Scott Simon zoomed to have 1 million fans on Twitter,” he said.

And don’t expect the number crunching to stop here. Thanks to NPR’s Audience Insight and Research team, the company studies everything from the wants of mobile users to comparing hour-by-hour visitors to NPR.org and radio listeners. Though it may not be unusual for a media organization to do research on news coverage or design factors, fielding responses on how you and your audience use Twitter is fairly new.

“I feel that being open represents the mission and values of NPR,” Carvin said. “Literally, if you look at our mission statement, it says we exist to create a more informed public.”

September 28 2010

08:54

September 23 2010

17:30

What impact is SEO having on journalists? Reports from the field

Last week, I wrote that SEO and audience metrics, when used well, can actually make journalism stronger. But I got pushback from journalists who complained that I was parroting back management views rather than the on-the-ground experiences of the reporters who have to deal with SEO-crazy bosses. So the natural next move was to gather more evidence.

It seems that whether SEO makes your journalistic life miserable you depends on how smart your news organization is about using SEO — and how your news organization does in making you feel invested in the process of combining SEO with quality content production. Organizations that understand the power of SEO and social news to drive traffic — rather than chase traffic — will keep their reporters in the loop and make them happy. Even if an organization has a good SEO strategy, it still needs to be communicated effectively to the newsroom, so journalists don’t feel like they’ve been turned from trained professionals into slaves to Google Trends.

Some journalists I communicated with (who shall remain nameless so they can keep their jobs) say SEO is pushing them to the brink. The demand that every story generate traffic creates, in their minds, horrible pressure to produce work that will be measured only by how much it is read. The more they can seed their work with SEO terms and then promote their work on social media platforms, the better their metrics and the happier their bosses. But that formula can make for some very exhausted journalists.

I emailed with one unhappy Washington Post reporter. She described a scene not unlike the one described by Jeremy W. Peters in The New York Times, with the majority of morning editors’ meetings focused on looking at web numbers and “usually, making coverage decisions based on that.” The reporter’s complained:

If my blog has an awesome day, I get complimented on it. If I spend weeks digging into a really juicy story, I don’t hear anything from anyone (well, unless it gets picked up by Gawker or it gets epic levels of hits). So what should I spend my time doing?

There have been several instances when reporters (or content producers) have been told by [SEO people] to drop everything they are doing and file some pithy blog post about the hot topic of the moment, which usually fades by the time we can get a story up.

She was also frustrated that that the new SEO people in the newsroom seem to have “unilateral power” and The Washington Post’s commitment to SEO was changing the way this reporter was writing.

“We are told to put SEO-powered words in our headlines, which I understand, even though it takes the life out of our heads,” she continued. “But every now and then, we have also been told to get these SEO-charged words into our first sentence or lede of a story. Wonder why a lede suddenly sounds robotic? This is why.”

But then I spoke with another Post reporter, who said “SEO has not really affected my work that much.” And while he has opinions about SEO, the reporter said that he does not “have a first-hand account of how SEO has hurt or helped me.” Have his editors not cracked the whip? Or has SEO become a natural part of his work?

Whatever it is, these two reporters aren’t thinking about SEO in the same way — which would seem to indicate that Post management could probably stand to improve the way it communicates with its staff about what it wants. Mixed communication about innovation is a common issue at times of change and the Post should not be singled out — their newsroom just happens to be one I reached out to.

Different newsrooms, different perceptions

I then turned to the crazy and wild world of online-only publications, thinking that SEO might be unusually disruptive for their journalists working under the commands to boost traffic and engage with audiences. I found this hypothesis to be untrue, even at traffic monster The Huffington Post and aspiring traffic monster The Daily Beast. For another perspective, I also spoke to the kind folks at GigaOM, the tech news blog, where technology is nothing to fear, and who actually went on the record. First, HuffPost, where my source emailed:

SEO guides the content we decide to write, but only to a certain degree…at HuffPost in particular, I know this is true because of how effectively we utilize organic SEO. Almost all of our posts are written, or should be written with SEO in mind…Sometimes SEO can determine an entire post. We have people in the office that are pretty hot on the top Google searches, and sometimes an entry will be created to utilize the traffic we get some traffic.

To be clear, organic SEO is the way that SEO works based on algorithms and natural searches. So HuffPost is really good at playing this game. But to play this game, one has to be aware that the algorithms are always changing. But does SEO do anything to their reporting or writing or actual gathering of news at HuffPost? My source:

I would say that SEO rarely impacts the actual reporting that we do. For a website as large and as SEO focused as HuffPost, instances of certain words within the article won’t actually help the search value of a post.

What about at The Daily Beast, which aspires to HuffPost levels of hyper-readership? Are work practices changing there? The source I spoke to didn’t find SEO or audience metrics onerous, simply seeing them as a small part of his job. And like my source at HuffPost, this was not an assault on journalism but the new reality — and one that didn’t actually affect the content of stories.

As my source told me, SEO gets used in pretty predictable ways, adding tags into stories and putting the SEO term in the URL slug — which isn’t same as the headline. But this journalist says the pressure of SEO has no significant impact on what he chooses to report or on how he writes.

The value of metrics

The folks at GigaOM were positive about how SEO was helping to change the way they do things — for the better. Probably because they understand the argument I made last week — that paying attention to the audience brings you in better touch with what the audience cares about. And they’re smart about how to use SEO to their advantage. Liz Gannes, a senior writer for GigaOM, listed off in an email her understanding of how SEO was affecting her work:

Metrics only get you so far…if there’s no spark of idea driving a story and voice behind it, it’s bound to be boring.

Until very recently, the most a journalist could hope for after pushing a perfectly polished piece out into the universe was feedback from (at best) a few dedicated readers or haters…Metrics give us a peek into what happens to our stories after we hit publish.

Metrics have helped Gannes think about ways that other reporters, editors and bloggers can use metrics to do better journalism:

— Recognize new and emerging topics
— Figure out the peak times of audience interest so a story will find its audience
— Help readers who have found old articles see a more recent one through links
— Understand when an article is clear and readable, or when it’s become too complicated
— Identify good sources of referral traffic
— Know when to get involved in comments and social media, potentially as inspiration for further articles.

Is SEO foe or friend for the journalist? Constant harping from editors, driven by fear of the future and the need to monetize the web, can make it feel to some journalists that SEO is destroying news judgment and their craft. But the best solution to this complaint is to figure out how to use SEO more effectively to make journalism better — and make the lives of journalists easier.

Metrics can be good for journalism and for journalists; it just takes putting aside the fear of the now to think about future strategies for building good content that will keep readers coming back. As I’ve said before, good content and high readership levels are not mutually exclusive: good stories will be found, and SEO can help.

September 14 2010

18:30

“Squeezing humanity through a straw”: The long-term consequences of using metrics in journalism

[Here's C.W. Anderson responding to the same subject Nikki Usher wrote about: the impact of audience data on how news organizations operate. Sort of a debate. —Josh]

One way to think about the growing use of online metrics in newsrooms (a practice that has been going on forever but seems to have finally been noticed of late) is to think about it as part of a general democratization of journalism. And it’s tempting to portray the two sides to the debate as (in this corner!) the young, tech-savvy newsroom manager who is finally listening to the audience, and (in this corner!) the fading fuddy-duddy-cum-elitist more concerned with outdated professional snobbery than with what the audience wants.

Fortunately, actual working journalists rarely truck in such simplistic stereotypes, arguing rightly that there isn’t a binary divide between smart measurement and good journalism. As Washington Post executive producer and head of digital news products Katharine Zaleski told Howard Kurtz:

There’s news we know people should read — because it’s important and originates with our reporting — and that’s our primary function…But we also have to be very aware of what people are searching for out there and want more information on…If we’re not doing that, we’re not doing our jobs.

Or as Lab contributor Nikki Usher put it: “[I]f used properly, SEO and audience tracking make newsrooms more accountable to their readers without dictating bad content decisions — and it can help newsrooms focus on reader needs.”

At the level of short-term newsroom practices, I agree with Usher, Zaleski, and every other journalist and pundit who takes a nuanced view of the role played by newsroom metrics. So if you’re worried about whether audience tracking is going to eliminate quality journalism, the quick answer is no.

My own concerns with the increased organizational reliance on metrics are more long-term and abstract. They have as much to do with society than with journalism per se. They center around:

— the manner in which metrics can serve as a form of newsroom discipline;
— the squishiness of algorithmically-afforded audience understanding;
— the often-oversimpistic ways we talk about the audience (under the assumption that we’re all talking about the same thing); and, finally
— the way that online quantification simplifies our understanding of what it means to “want” information.

Big topics, I admit. Each of these points could be the subject of its own blog post, so for the sake of space, I want to frame what I’m talking about by dissecting this seemingly innocuous phrase:

“We know what the audience wants.”

Let’s look at the words in this sentence, one at a time. Each of them bundles in a lot of assumptions, which, when examined together, might shed light on the uses and the potential long-term pitfalls of newsroom quantification.

“We”: Who is the “we” that knows what kind of journalism the audience wants? Often, I’d argue, it’s executives in our increasingly digitized newsrooms that now have a powerful tool through which to manage and discipline their employees. In my own research, I’ve discovered that the biggest factor in determining the relationship between metrics and editorial practices are the ways that these metrics are utilized by management, rather the presence or absence of a particular technology. Philosopher Michel Foucault called these types of practices disciplinary practices, and argued that they involved three primary types of control: “hierarchical observation, normalizing judgment, and the examination.” Perhaps this is fine when we’re trying to salvage a functional news industry out of the wreckage of a failed business model, but we should at least keep these complications in mind — metrics are a newsroom enforcement mechanism.

“Know”: Actually, we don’t know a whole lot about our audiences — but there’s a lot of power in claiming that we know everything. In other words, the more data we have, paradoxically, the less we know, and the more it behooves us to claim exactitude. While smart thinkers have been writing about the problem of poor web metrics for years, a major new report by the Tow Center for Digital Journalism at Columbia has thrown the issue into stark relief. As report researcher (and, full disclosure, friend and colleague) Lucas Graves writes:

The Web has been hailed as the most measurable medium ever, and it lives up to the hype. The mistake was to assume that everyone measuring everything would produce clarity. On the contrary, clear media standards emerge where there’s a shortage of real data about audiences…The only way to imbue an audience number with anything like the authority of the old TV ratings is with a new monopoly — if either Nielsen or comScore folds or, more likely, they merge. That kind of authority won’t mean greater accuracy, just less argument.

There’s a circular relationship here between increased measurement, less meaningful knowledge, and greater institutional power. When we forget this, we can be uncritical about what it is metrics actually allow us to do.

“The Audience”: What’s this thing we insist we know so much about? We call it the audience, but sometimes we slip and call it “the public.” But audiences are not publics, and it’s dangerous to claim that they are. Groups of people connected by the media can be connected in all sorts of ways, for all sorts of reasons, and can be called all sorts of things; they can be citizens united by common purpose, or by public deliberation. They can be activists, united around a shared political goal. They can be a community, or a society. Or they can be called an audience.

I don’t have anything at all against the notion of the audience, per se — but I am concerned that journalists are increasingly equating the measurable audience (a statistical aggregate connected by technology, though consumption) with something bigger and more important. The fact that we know the desires and preferences and this formerly shadowy and hidden group of strangers is seductive, and it’s often wrong.

“Wants”: Finally, what does it mean to want a particular piece of information? As Alexis Madrigal notes in this short but smart post at The Atlantic, informational want is a complicated emotion that runs the risk of being oversimplified by algorithms. Paradoxically, web metrics have become increasingly complex at the same time they’ve posited increasingly simplistic outcomes. They’re complex in terms of their techniques, but simple in terms of what it is we claim they provide us and in the ultimate goal that they serve. Time on site, engagement, pageviews, uniques, eye movement, mouse movement — all of these ultimately boil down to tracking a base-level consumer desire via the click of a mouse or the movement of the eye.

But what do we “want”? We want to love a story, to be angry about it it, to fight with it, to be politically engaged by it, to feel politically apathetic towards it, to let it join us together in a common cause, for it make us laugh, and for it to make us cry. All of these wants are hard to capture quantitatively, and in our rush to capture audience data, we run the risk of oversimplifying the notion of informational desire. We run the risk of squeezing humanity through a digital straw.

So — will an increasing use of online metrics give us bad journalism? No.

Will they play a role in facilitating, over the long term, the emergence of a communicative world that is a little flatter, a little more squeezed, a little more quantitative, more disciplinary, more predictive, and less interesting? They might. But take hope: Such an outcome is likely only if we lose sight of what it is that metrics can do, and what it is about human beings that they leave out.

17:30

Why SEO and audience tracking won’t kill journalism as we know it

[I'm happy to introduce Nikki Usher, a new contributor here at the Lab. Nikki is a Ph.D. candidate at USC Annenberg and, before academia, was a reporter for The Philadelphia Inquirer and elsewhere. Here she tackles the question of using metrics in journalism; later today, we'll have a different take on the same topic from C.W. Anderson. —Josh]

Last week, The New York Times featured the scary tale of how some newspapers, including The Washington Post and The Wall Street Journal, are (shockingly!) changing their coverage after using online metrics to figure out what their audience wants to read. And Gene Weingarten, in an amusing takedown of search engine optimization, insinuated earlier in the summer that just by putting Lady Gaga in his column, he’d get more hits.

Jeremy W. Peters had another Times piece about much the same concern: young journalists doing “anything that will impress Google algorithms and draw readers their way” and the scary “big board” that Gawker keeps in its newsroom tracking the 10 most popular blog posts, along with pageviews per hour.

This concern that audience tracking, writing for Google, and SEO will somehow destroy the ability of news organizations to keep news judgment apart from audience demands is misplaced. Instead, being more attentive to audience demands may actually be the best thing that news organizations can do to remain relevant and vital sources of news.

With monetization tied to clicks, and real-time Omniture data a feature of more and more newsrooms, it’s easy to worry that audiences will dictate news coverage. But how about the opposite argument: that journalists, for too long, have been writing about what they think their readers ought to know, and not enough about what their audiences want to know.

Journalism has always depended on having an audience to consume its work and has spent much of the past century trying to figure out exactly what that audience wants to know. Now, journalists have better tools than ever to figure out who their audiences are, learn what they want, and in real time, track their behaviors in order to be more responsive to their needs. This isn’t a bad thing — it turns journalism away from the elitism of writing for itself and back to writing what people are actually looking for.

But what about the concerns that journalists are going to spend all their time writing about pets, or Lady Gaga? The truth is that many of the newsrooms I’ve spoken with are smarter than that. They aren’t abandoning journalism principles; they see metrics as a way to ensure their journalism will be read.

SEO at the Christian Science Monitor

In my academic work, I’ve been following the evolution of The Christian Science Monitor as it has moved from a print daily to a website with a print weekly. Over the course of this evolution, I’ve watched the newsroom grow increasingly sophisticated about audience tracking. When I asked John Yemma about his views on SEO, he had this to say in an email about its impact on the newsroom:

Search engines remain a powerful and preferred tool for online readers. We have no choice but to become adept at SEO if it helps us reach readers where they are. This is nothing new in the news business. In the pre-Web days, newspapers periodically redesigned and reformatted. Editors frequently admonished reporters to write shorter, to use simple and direct language, to “think art” when they were on an assignment — all in the interest of reaching readers.

SEO, at its essence, is about editors thinking the way readers think when they are searching for news. At the Monitor, as at almost every other publication, we work diligently to emphasize key words. But that is only one tool in the toolkit. We try to respond quickly when a subject we know well (international news, for instance) is trending. This gives us an opportunity to offer related links that invite readers to dive deeper into our content. If SEO is about acquisition, related links are about retention. In the past year, we have tripled our online traffic with this strategy.

Does that mean we just write plain-vanilla headlines or merely follow Google/Trends? No. A clever headline can still be a powerful draw, especially on our home-page or in social media. And we still report stories that we know are important even if readers don’t agree. But we are much more attuned these days to what readers will respond to. If our journalism is not read, our work is not effective.

Trend tracking at TheStreet.com

At TheStreet.com, the organization has hired a full-time “SEO guy,” John DeFeo, to monitor trends on Omniture, watch search terms, and optimize TheStreet’s content after it is written so it can be found via search.

The result: Traffic has improved. When I was in TheStreet’s newsroom conducting field research, I did see DeFeo make a suggestion that someone bang out a quick story on a children’s Tylenol recall after seeing it trend on Yahoo. But should we see that as being overly responsive to audience demands? Or should we see it instead as a chance for TheStreet to provide its unique comment on what such a recall might mean for Johnson & Johnson stockholders — and at the same time know that the story will have a chance at reaching an audience because it is trending?

Glenn Hall, Editor at TheStreet, defends SEO journalism as being the core of the basic principles of journalism itself. In an interview, Hall said:

Good journalism is not mutually exclusive with SEO. We have proven over and over again that our best journalism tends to get the best page views. SEO is a tool to make sure the best stories get noticed…SEO increases visibility where users are looking. People consume content differently than they used to through a newspaper.

Hall explains to his staff that SEO is in line with the best practices of journalism. He believes that simple declarative sentences, clear and to the point, makes good sense for both journalism and SEO. And, as he notes, SEO doesn’t have the final say on a story’s success or failure: “It doesn’t matter how good the SEO is if the content isn’t good.”

The new news is social

Nick Bilton, the Times tech blogger, writes in his new book, I Live in the Future & Here’s How It Works about the “consumnivore” — an information-hungry consumer who wants the latest news now. But for this new information consumer, information isn’t just a quest for information. It’s also a social experience, shared with people from Twitter, Facebook, email, or other social media. In other words, if you aren’t looking for news, the news will find you. Good journalism will still be found, even without the high-energy SEO pumping of a daily newsroom — largely, I think, because of the new power of news as a social experience.

This isn’t a myth. At the Pulitzer celebration at The New York Times on April 12, 2010, New York Times Magazine editor Gerald Marzorati noted the following in his celebratory speech for sharing the Pulitzer with Propublica for Investigative Reporting for a story about a New Orleans hospital during Katrina: “[Long form journalism is] our most viewed and most emailed…It does matter to readers. It stops the reader. It slows the reader down.”

Was Memorial Medical Center, the hospital in the story, a hot search term? Probably not. Were 13,000 words likely to produce the quick hits of information that the consumnivore hungers for? No. But the story still reached a substantial audience, person to person. And as it was read by more and more people, it likely climbed up Google’s rankings for those people who were searching for articles about Katrina.

So, if used properly, SEO and audience tracking make newsrooms more accountable to their readers without dictating bad content decisions — and it can help newsrooms focus on reader needs. What is a story if it is never read? SEO won’t kill journalism; it will only enhance how we find and use news.

August 19 2010

16:11

NPR’s audience, hour-by-hour, platform by platform

NPR has looked at the behaviour of its audiences across all platforms, charting how content is consumed hour-by-hour over the course of an average week.

Radio remains NPR’s strongest distribution platform, with an audience of around 2,500,000 average quarter hour listeners on weekday mornings at 7:00am. Peaks in broadcast figures follow commuter times, with another high of around 2,000,000 at 5:00pm. Online audiences peak in the early afternoon with around 75,000 average unique visitors at 2:00pm, falling off gradually over the course of the rest of the day. Note the dual axes for comparing radio and online audiences.

Metrics for mobile show that NPR readers favour the iPhone over Android, iPad or mobile web, and peak earlier in the day than the online browser figures, with a little more than 8,000 unique visits at around 8:00am.

Similar Posts:



Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl