Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

March 10 2011

17:00

“Journalists have lost control of the story”: Twitter, tech bubbles, and the nostalgia of the technology press

Editor’s Note: I’m very happy to welcome Tim Carmody — who you may know from Snarkmarket, kottke.org, Wired.com, Twitter, or elsewhere — as a contributor to the Lab. Here he looks at how the increasing speed of media opens us to manipulation — and false nostalgia.

There’s nothing new about speculation bubbles, especially in the technology industry. It’s nearly impossible to be certain which new ideas or products will be able to do what they’re supposed to be able to — let alone whether they’ll be able to do so at cost or scale, if they’ll be adopted by the market, or if a competitor will get there first and better. And when everything’s happening quickly and everything seems exciting, it’s nearly impossible to tell a bubble from a real boom.

The only sure strategy for an investor or inventor is to get in early, push the company as hard as you can to attract attention and investment, and try to sell high, neither too late or too soon. When the economics of money and attention move too far past the economics of the underlying value, you get a bubble. When the money and attention slow, then stop, then rush in the opposite direction, the bubble bursts. The boom is over, if it ever existed at all.

There’s also nothing new about the press’s role in helping to inflate bubbles, worrying over them, and watching them burst. What is new, according to Federated Media’s John Battelle and Thomson Reuters’ Connie Loizos, is how the accelerated news cycle of blogs, Twitter, and other digital media forces the technology press to work at the same speed as the investors they cover — with the same worries about getting in early and beating competitors trumping the real value of the product. In this case, though, the product is their own journalism.

“For several years now,” writes Loizos, “savvy investors have been effectively gaming Twitter and mastering the ability to trumpet their investments in 140-word sound bites.” The credibility (in both senses) of the technology press, when mixed with Twitter’s easy ability to quickly pass on information without comment, gives those trumpet bursts an amplifier. “Journalists have lost control of the story. In rushing to retweet the latest auction results from SharesPost, we’re not thinking about what we’re writing or questioning what we’ve been told.”

Loizos elaborated on her argument in an email. “Thanks to Twitter and, to a lesser extent, other social media like Quora, information about startups and financings has become much more porous,” spreading good and bad information equally quickly, and in volume. “The first story out wins. For example, that first ‘scoop’ is what gets the most real estate by powerful aggregators like Techmeme, while every other story gets scuttled underneath it.” It also changes the relationship between a reporter’s sources and her audience. “[Now] we’re not just competing against one another as journalists but also against savvy investors and entrepreneurs who know they can reach just as broad an audience by delivering their news themselves via Twitter and their blogs.”

Loizos is a veteran of the last tech valuation boom and bust, reporting for the first-generation tech magazine The Industry Standard, founded by Battelle. Battelle’s Federated Media has since gone on to partner with a who’s who of current tech culture and business sites, from Boing Boing and TechCrunch to Business Insider and GigaOm. He sees a problem too, possibly bigger than VCs driving their investments.

The real bubble, or at least the more troubling one, is the “Internet interest bubble.” Here the press is not peripheral but central to the story.

In the new media landscape, “we have migrated to a more free-wheeling discourse driven by any number of interested parties,” Battelle writes. In addition to investors, we see “bankers trying to influence any number of outcomes, and sources within all manners of companies pushing their own agenda on Twitter, Quora, or in private conversations with bloggers and other media outlets…The tweets, conference utterances, and blog posts of these sources are instantly turned into ‘news stories’ by the post-cambrian publishing explosion of sites covering the narrative that was once the province of first-generation Internet magazines” like Battelle’s Standard.

Churnalism, in other words, is a much bigger problem than just press releases and wire stories. It’s everywhere — and creating an echo chamber unprecedented in its size and reach.

“Millions upon millions of people visit these tech news sites, because the narrative they chronicle is more important than it’s ever been,” Battelle writes. “Our industry impacts a huge swatch of society and culture, and increasingly is understood to be the core driver of pretty much all of business today.” And apart from contributing to a tech bubble, Battelle and Loizos think that the echo chamber crowds out better analysis and better stories in our news sources:

But where’s the bigger picture? Where’s the hold-on-a-minute-let’s-think-this-through-and make-a-few-phone-calls-and-see-how-it-develops approach? Where’s the conceptual scoop? The second-day (or even second week) analysis?

“There are stories about healthcare startups that are transforming lives that no one is reading,” Loizos told me. “I think behind-the-scenes profiles of employees who truly make Valley companies valuable are fascinating, but people don’t make time to write them because there’s still this unquenchable thirst for the same stories being written again and again: about the hottest new startup, the hottest new venture capital firm, the hottest new valuation, the hottest new application.” There’s also the comfort of the familiar: “in the tech universe, people could read about Twitter and Facebook” — or Apple and Google, etc. — “all day long and journalists — saddled with driving eyeballs — are giving them what they want.”

“It’s an exciting time, but it’s also pretty screwed up,” she adds.

Both Loizos and Battelle show some nostalgia for the tech coverage produced by magazines like the Standard in the 1990s — partly for the quality of the reporting or at least the relative sanity of print’s slower pace. But Owen Youngman, Knight Professor of Digital Media Strategy at Northwestern University’s Medill School of Journalism, is skeptical that things were any better a decade ago.

“In my memory,” Youngman told Loizos, “a lot of glossy magazines back then were by and for the same people that are running up valuations today, and they could make even the wispiest of ideas seemed substantial.” In an email, he added that “the nostalgia is more about the former number of high-gloss, high-profile, high-paying outlets for tech journalism, not necessarily for the journalism itself.”

In a recent article for The Atlantic, James Fallows voices a similar skepticism about our ability to accurately measure journalism’s present against its past.

“When I recently talked to people in the news business, historians, political scientists, and others about the current predicament of the news, every previous era looks innocent,” Fallows writes. Flux in journalism isn’t the exception, but the rule; and what seem to us like venerable staples like Time, Nightline, or NPR are both younger and were more radical than we typically remember. Ultimately, even that is the wrong question: “While it’s interesting and even useful to know whether today’s journalism marks a descent from past standards, what matters more is how it suits today’s needs.”

At the same time, even VCs themselves are balking at the speed of the market and how social media are disrupting their own practices. AngelList plays a similar role for investors and entrepreneurs that TechMeme plays for journalists and readers, using aggregation, filtering, and social media to manage the flow of information and create new opportunities for both. In “Why I Deleted My AngelList Account,” influential VC Bryce Roberts detailed how this approach conflicted with his own investment strategy and style:

At the earliest stages, it’s nearly impossible to pick the next Google so throw a lot of darts in the dark and hope you hit it. That high velocity, light touch style is certainly a viable approach to investing. It’s just not my style.

I tend towards a more concentrated approach to seed investing where we make fewer, larger, investments and take an active role in working with the companies we fund. Frankly, I just don’t buy the notion that making an investment is akin to throwing a dart in the dark. Worse, I think it’s a dangerous idea to promote…

Real or perceived, organic or manufactured, AngelList is in the business of generating heat. As I’ve said here and elsewhere, I tend to be interested in ideas and companies that most investors aren’t, so heat is generally a false signal for me.

Johnson’s post quickly drew a sharp response from Internet entrepreneur/provocateur Jason Calacanis. “Let’s be honest and just say what’s happening here: you’re pissed that you now have hundreds of angels swarming on deals that you used to be able to snap up at half the price…There are now *hundreds* of qualified and unqualified angels who are driven by sport and not return! They are betting with their own money — not some LP’s” — limited partners who invest in a venture capitalist’s aggregated fund rather than make individual investments — “and [they're] more excited by private companies than 4% muni bonds.”

The language is very different, but it’s not dissimilar to Youngman’s critique of journalistic nostalgia — or for that matter, Nick Denton’s defense of Gawker’s approach to web journalism to Fallows. People want what they want — and what they want is low-opportunity-cost fun. Nobody wants “to eat their vegetables,” to use Denton’s phrase for high-substance, high-prestige investigative journalism. These outlets need the support of institutions or nonprofits, not advertising and eyeballs alone.

It’s clear that both technology companies and technology journalism are on the cusp of something. Whether it’s a bubble or a boom, we can’t know. In the meantime, we have all of the problems of indeterminacy: practices and standards held over from an earlier period jostling against emerging conventions which offer something new.

Blogs and social media offer both entrepreneurs and journalists new modes of engagement with each other and a different kind of conversation with their readers. At the same time, the demands of traditional news formats can actually push us into stories that privilege new forms of manipulation. Reporters seeking a news peg for an analysis-driven story about a popular company can find quotes from blogs, Twitter, or Quora as easily as they can from a company’s press release, putting the same texts and voices into circulation.

Finally, news outlets have to recognize that a big part of their readership is driven by popular speculation, particularly if their coverage focuses on hot startups, big IPOs, and new deals. If a valuation bubble bursts, those eyeballs vanish too. Investing in deep analysis, conceptual scoops, alternative content, experimental storytelling — and the reporters who can produce those stories — is a terrific hedge against that dangerous future.

September 20 2010

17:30

In a hamster-wheel world, is there room for journalistic creativity? Evidence from The New York Times

The essential question facing newsrooms today is this one: Does more speed and more content come at the cost of creativity? Does the “hamster wheel,” as described by Dean Starkman in this month’s Columbia Journalism Review cover story, reduce journalists’ capacity to move stories forward instead of playing catch-up online? And does the demand for speed and the hunger for clicks come at the cost of thoughtful editing and crafting of stories?

This question is at the core of my (almost written) dissertation — when news is made in an online newsroom, what happens to the invention process? But it is my contention, after having the privilege to spend five months, day-in and day-out at Business Day at The New York Times, is that even though many journalists there often feel like wire reporters, many also feel that writing a story five times actually makes their work better.

But that division in sentiment is not the heart of the argument. My sense so far is that there are five factors that encourage creativity in newsrooms even at a time when journalists are producing more with less.

— Newsrooms, recognizing that news is everywhere, need to differentiate their content.

What makes a story in The New York Times business section — even if it the third time it is written by a staffer — different from the same story in The Wall Street Journal? The hope and aspirations by editors, at least, and the goal expressed to me by the close to 50 journalists I interviewed on the business desk, was that the intention of every story was to provide “added value” — something that other newsrooms wouldn’t have.

Most journalists referred to the news that everyone else has as “commodity news” — the news that you can get anywhere. But if newsrooms are to survive, newsrooms as they produce multiple iterations of the same story throughout the day must provide something different than their competitors. This challenges the journalist further to provide a different take, and the most successful journalists will be able to distinguish their content. Those who win that battle will also win the click battle, eventually.

— Newsrooms that still have a print cycle have to pause and think about the day ahead.

If print newsrooms are to remain competitive, there is necessarily a point at which journalists must think about what’s going to be in the paper. If a developing news story has been up on the web all day for readers, newspapers won’t maximize their return by just plopping that story into the next day’s paper. At The Times, the story in the print paper was viewed much more as a “second-day story” — even if the event had taken place the same day the story was written. A few staffers said that The Times in print was become more like a daily news magazine than a daily paper, giving people a step back from the daily hubbub of the news to provide a deeper and richer story.

Take, for instance, a Goldman Sachs earnings report. In the morning, it might be a routine earnings story with the numbers. Over the course of the day, a reporter might differentiate that story with different questions asked to bigwigs at Goldman, different snippets of life from the trading floor, and perhaps proprietary reporting gleaned from sources, or even takes on the earnings from academics. At a certain point in the day, the rewriting of the story stops and it becomes time to look for the big picture — there has to be enough that people who haven’t been following the story all day have enough to understand the story, but there will be a larger tale, perhaps about the broader significance of the numbers, or what larger trends at the bank might mean, or other take-outs that make the story different from competitors.

And don’t forget: This final print story is the final story that ends up online.

— Speed only applies to certain kinds of stories.

Only certain stories lend themselves to the kind of developing coverage that would require multiple rewrites. Hearings on Capitol Hill might lend themselves to something like the attention of a live-blogger plus the attention of a reporter tasked with covering the take-out stories as they develop, from pre-written statements to the actual question and answer period. On the day Apple’s iPad was introduced, I saw multiple stories being written as the story developed — and the attention of everything from a live-blog to all-hands on deck with Twitter and The Times’ Bits blog. These stories require constant updates because something new is happening as the day develops. There is more to add to the story. And determining which updates are worth including is the careful task of editors and reporters who must again decide where added value comes in.

But it is my thinking that newsrooms, even those with increasingly limited resources, also understand the importance of pacing and managing staff. A newsroom that has everyone devoted to playing catch-up will not have the substantive stories that will distinguish their news from all the other products out there. Thus some reporters have to be taken off the breaking news bandwagon — this may be for particular days, or it may be that some reporters simply do not have to do regular breaking news. This is one way to keep coverage fresh and inventive. A newsroom that can figure out how to allocate resources will be a newsroom that continues to remain creative and one that ultimately will keep readers coming back.

— The audience does want more, now more than ever. And journalists can listen, too.

Presuming a developing story will go through multiple iterations, it is reasonable to suspect that audiences will be checking back. Or that since the audience is fragmented, there are different audiences checking in at different points of the day. Shouldn’t we have something for audiences that choose to follow the story over the course of the day? And shouldn’t we have something for audiences that choose to read just once, say at 3 p.m., instead of the moment that the news is breaking? My thinking is that the audience wants more from journalists because journalists can provide more; the voracious news consumer (and ultimately the news consumer who will be most valuable when news organizations switch to paywalls or meter models) will be checking the site frequently. And for those drop-in audiences, people who maybe check once in the morning and once in the evening, don’t they also deserve something new?

But there’s another element here: Developing stories also are also the ones, I’ve observed, to be most likely to be open to comments. Though not all journalists have gotten the hang of reading comments or monitoring Twitter, they are definitely reading reader email as it comes in. And this audience feedback provides journalists with new opportunities for direction for stories, and a sense of how their stories are being received in a way that they never could have had before. This sense of instant feedback on a story’s progression has the opportunity to shape reporting. This is still early in its development, and at this point most likely to affect reporters who can monitor Twitter and check comments as part of their regular routine. But I see great opportunity for audience feedback shaping developing news in the future.

— Speed also means more attention has to be devoted to more than just the text of the story.

If a story is going to be big enough to merit multiple rewrites — if it is a developing story all over the web — you better hope that the story isn’t just text. The room for creativity does depend on the capacity of newsrooms. At The Times, the newsroom is privileged to have an amazing staff of web producers, graphics folks, photo staff, and videographers who can create a multimedia package to go along with a developing story to make that story stand out. But the lesson is true for newsrooms that do not have the same depth of resources as The Times (which, it must be said, often can’t do everything it wants, either).

There are other ways for stories to become more than just stories. Without multimedia, reporters even as they go about collecting their moment-by-moment updates, can also be engaged with conversations on social media platforms. This is an adjustment for reporters, and it certainly adds another layer to the concern about speed and burnout, but it inspires, as I noted before, creativity — and it adds further interest to the story. I have seen reporters working on intense deadline pressure on competitive stories use social media to enhance their work — extending the story beyond text. For those looking for some inspiration, check out what Michael de la Merced, Brian Stelter, and Micki Maynard (who has since left the paper) have been able to do on big stories.

So am I setting up an unfair example?

Certainly you could argue that a place like The Times is an outlier and an unfair place to start talking about creativity under pressure. But I don’t think that The Times is doing anything that other newsrooms aren’t, except for perhaps the amazing multimedia opportunities. The Times is still in fierce competition to distinguish coverage, reporters are still writing multiple times during a day on developing stories, and the challenges on journalists to do ever more are common to all newsrooms. But I see incredible opportunities for the hamster wheel to produce even better journalism — it just might take some time to figure out.

June 29 2010

23:01

What's the fastest way to compare two large CSVs against each other?

So here's the nut. Imagine you've got two huge CSV files, snapshots from the same database taken on different days.

They share a common identifier, and always have the same set of fields. But there are amendments, omissions and additions made between snapshots that can only be detected by comparing records against each other. What's the fastest way to loop through a fresh snapshot and compare it against the previous snapshot for changes, additions and omissions?

Below is a roughed out Python routine I've written with a fake data set. Basically, it sets the unique ID as the key to a dictionary that contains what amounts to a CSV DictReader pull from two imaginary CSV files: one with the older "existing" snapshot; and another with the newer "fresh" snapshot.

It seems to work okay in testing, but when you run it over a large data set, it goes pretty darn slow. I'm curious whether anyone here knows a way I can make it run quicker.

existing_dict = {
    'A1': {'name': 'John', 'gender': 'M', 'value': 10},
    'A2': {'name': 'Jane', 'gender': 'F', 'value': 10},
    'A3': {'name': 'Josh', 'gender': 'M', 'value': 20},
    'A4': {'name': 'John', 'gender': 'M', 'value': 20},
    'A5': {'name': 'Janet', 'gender': 'F', 'value': 15},
    'A6': {'name': 'January', 'gender': 'F', 'value': 10},
}

fresh_dict = {
    'A2': {'name': 'Jane', 'gender': 'F', 'value': 10},
    'A3': {'name': 'Josh', 'gender': 'M', 'value': 20},
    'A4': {'name': 'John', 'gender': 'M', 'value': 20},
    'A5': {'name': 'Janet', 'gender': 'F', 'value': 15},
    'A6': {'name': 'January', 'gender': 'F', 'value': 5},
    'A7': {'name': 'Jessica', 'gender': 'F', 'value': 10},
}

def compare():
    """
    Compares two data dicts against each other.
    """
    # Set some counters to report outcome
    nochanges, amendments, omissions = 0,0,0

    # Loop through the existing data...
    for id_, existing_data in existing_dict.items():
        # Try to find the corresponding record in the fresh data
        fresh_data = fresh_dict.get(id_, None)
        # If it's there...
        if fresh_data:
            # Determine if there are any changes
            if is_diff(existing_data, fresh_data):
                amendments += 1
            else:
                nochanges += 1
            del fresh_dict[id_]
        else:
            omissions += 1
    additions = len(fresh_dict.keys())
    return nochanges, amendments, omissions, additions

def is_diff(existing_row, fresh_row):
    change_list = [field for field in existing_row.keys()
        if existing_row.get(field) != fresh_row.get(field)]
    if change_list:
        return True
    return False

if __name__ == '__main__':
    print "No change:%s; Amendments:%s; Omissions:%s; Additions:%s;" % compare()

That's pretty much it. Here's what the imaginary CSV files might look like, if it helps.

existing.csv

id,name,gender,value
A1,John,M,10
A2,Jane,F,10
A3,Josh,M,20
A4,John,M,20
A5,Janet,F,15
A6,January,F,10

fresh.csv

id,name,gender,value
A2,Jane,F,10
A3,Josh,M,20
A4,John,M,20
A5,Janet,F,15
A6,January,F,5
A7,Jessica,F,10

November 23 2009

16:55

The half-life of news

At a Yale conference a week ago, Thomson Reuters CEO Tom Glocer talked about the life cycle of the value of news in his business.

When a piece of financial news come out, it is at its most valuable for a very short time, he said. I asked him later how long that is. “Milliseconds,” he replied. Milliseconds. That’s as long as a computerized trader has to take advantage of news before the market knows it, before the news is knowledge and is thus commodified and loses its unique and timely value.

Reuters still gets high value out of its news in stages, turning this tidbit into a headline and a story and selling it as part of its financial data services and then its wires. It finally lands on Reuters’ web site, visible to consumers, where Reuters collects ad revenue directly. That, Glocer said, is about 2% of Reuters’ revenue.

Of course, one can’t view this timeline in isolation. The news is being spread in all kinds of vectors: other news organizations get it and it’s masticated and repeated in print (slow), on broadcast (faster), on websites (faster), by aggregators (faster), by conversation (aka Twitter – getting faster all the time). The faster that distribution is, the quicker news becomes knowledge and thus a commodity, the faster it loses its unique, saleable value. And that chain is getting only faster.

And that, ladies and gentlemen, is one reason why trying to lock up the value of news behind a wall won’t work, in my estimation.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl