Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

August 09 2012


rt.ly: Stories about food being clicked statistically disproportionately in Brooklyn?

bit.ly's realtime engine and lab experiment is making nice progress. Latest feature improvements: city level searches and pagination.

Bit.ly (via email) :: We added a few improvements to rt.ly (bit.ly's real-time service) in the last week. For example, you can now do high-resolution location queries down to the city level, like “show me all of the stories about food that are being clicked statistically disproportionately in Brooklyn? We also added pagination, so you can see the full set of results that match a particular query.

Try this realtime search: stories about food that are being clicked statistically disproportionately in Brooklyn


Reuters, Gizmodo Hacks Are Cautionary Tales for News Orgs

The Syrian civil war is also a propaganda war. With the Assad regime and the rebels both attempting to assure their supporters and the world that they are on the brink of victory, how the facts are reported has become central to the struggle. Hackers working in support of Assad loyalists this week decided to take a shortcut, attacking the Reuters news agency's blogging platform and one of its Twitter accounts, and planting false stories about the vanquishing of rebel leaders and wavering support for them from abroad.

The stories and tweets were unconvincing, and none spread much further than their home sites. The majority of readers disseminating the repurposed Twitter stream appeared to be Assad partisans, either keen to spread the misconceptions or to believe them themselves.

The attacks demonstrate, however, how media institutions are at risk of targeted attacks by state-supported electronic activists -- and that hackers will attempt to leverage the outlying parts of a large organization to take wider control, or at least the appearance of wider control.

Neither Reuters' blogging site nor its minor Twitter accounts feed the company's authoritative wire service, but as a consequence they may not have the same levels of heavy protection against misuse. A weak password used by a single person could have granted an outsider the power to post publicly to either service.

Even individual journalists are at risk

Even when a hacker's target is an individual journalist and not his or her media organization, things can escalate to affect the institutions journalists work for. When the tech reporting site Gizmodo's Twitter account was taken over on Friday, it was through an attack on one of its former reporters, Mat Honan. Gizmodo's reporting has made it unpopular in some quarters, but Honan says that he was the target, and that Gizmodo was "collateral damage." His Twitter account was linked to Gizmodo's corporate account, and the attackers used one to post to the other.

Thumbnail image for mathonan.png

Honan's story should give anyone pause about their own digital safety, especially if they rely on external companies. His Twitter account was taken over by a hacker who persuaded a tech support line operator to reset the password to his Apple account. The attacker used this account to change his linked Gmail and Twitter account information, and then proceeded to use the "remote wipe" feature on the latest Apple iPhone and laptops to disable and delete the content of his phone, iPad and Macbook. As a
freelancer, Honan did not have offline backup of his work. (Honan says he is waiting for a response from Apple the company; meanwhile, Apple tech support is helping with damage control.)

Honan has corresponded with an individual who claims to be his hacker, and says that the real intent of the compromise was his three-letter Twitter account. Whether it's by common cybercriminals or state-supported propagandists, journalists are being targeted as individuals. The organizations that employ them need to invest resources and training to improve their cyber-security; not least because when one person's security is compromised, everyone who relies on that person is also under threat.

Danny O'Brien is the Internet advocacy coordinator for the Committee to Protect Journalists. O'Brien has been at the forefront of the fight for digital rights worldwide, serving as an activist for the Electronic Frontier Foundation. He was an original staff member for Wired UK magazine and co-founded the Open Rights Group, a British digital rights organization. He's also worked as a journalist covering technology and culture for the New Scientist, The Sunday Times of London, and The Irish Times. Follow on Twitter: @danny_at_cpj

cpj-logo-name.jpgA version of this post originally appeared on CPJ's Internet Channel. The Committee to Protect Journalists is a New York-based, independent, non-profit organization that works to safeguard press freedom worldwide. You can learn more at CPJ.org or follow the CPJ on Twitter @pressfreedom or on Facebook here.

This is a summary. Visit our site for the full post ».

Sponsored post

How User-Centered Design Powers FrontlineSMS, Version 2

I'm going to be honest: When I first joined FrontlineSMS, I had no idea how much goes into the design of software. Every screen, every button and every function has principled thought behind it.


In 2011, we worked alongside Gabriel White, a user experience designer from Small Surfaces, to help translate FrontlineSMS users' needs into the new design of Version 2. I came to realize that no matter how advanced and amazing a piece of software might be, it has no relevance if users can't access it or work out how to use it. I think that the user interface -- that point of contact between a user and the functionality (or what the software can do) -- is the most important entry point in the way users experience a tool.

It's now been over a year and a half after the design work first began, and I recently spoke with Gabe to share his reflections on how we ensured users' priorities were central to the design of Version 2.

What user design experience involves

I'm sure that for many of us it's not clear what User Experience Design really involves, so I asked Gabe to explain. "To me, it means creating products and services that address real user needs, and defining how people can interact with software in a way that's useful and meaningful. The most important things to consider in this process are what you (as an organization) are trying to achieve by creating the product or service; what the needs of the end users are; and then bringing those two sets of goals together through a design solution that is usable, useful and engaging."


At FrontlineSMS, we have always endeavored to put our users first and be responsive to their needs -- to make our software work better for them. This user-centered design process is at the heart of Version 2. I was curious to ask Gabe how he got involved in the FrontlineSMS project. "I decided to move to Uganda to focus my work on projects which were meaningful to me in terms of positive social impact," he replied. "I found out about the Mobiles for Development Conference in Kampala in 2010. I'd heard that FrontlineSMS' founder, Ken Banks, was going to be there, and the FrontlineSMS project was exactly the kind of initiative I wanted to get involved in. So I basically cornered him and said, 'We have to have a coffee together!' When I later found out that he was thinking about how the user experience would evolve in the then-upcoming Version 2 of the software, it felt like serendipity. Working with FrontlineSMS turned out to be one of the highlights of my design career."

step one: personas

The first step in working together was when Gabe asked us to draw up profiles representing the characteristics of different types of FrontlineSMS users ("Personas" in design-speak). We asked volunteers who represented diverse projects using FrontlineSMS to be involved in the design process. Gabe explained the importance of this: "It's really critical to involve users throughout the entire process so that you can continuously ensure that you address users' real needs in appropriate ways. First, we interviewed existing users of the software to understand their aspirations and pain points. This helped us frame the problems we wanted to solve with Version 2. As I began to craft a design solution, it was important to continue to engage end users through the process. So even when we had only very early design concepts, I shared the alternative solutions with users to understand how effectively the design ideas met the needs I'd earlier uncovered."

"One of the things we found was that, while it was often easy to do basic things in Version 1 of the software, it was sometimes harder to do more sophisticated things with it. For example, FrontlineSMS users often want to use the tool to gather together messages from a group of people on a range of specific topics, or create a poll and easily understand the responses. Essentially, it's great to be able to gather or disperse information using FrontlineSMS, but that's only the beginning of the story -- it's often what users do with all those messages afterwards that counts. Making it easier for people to use FrontlineSMS to do more sophisticated things was critical as we thought about building the new software."

the inspiration behind activities

This speaks volumes to a central feature of Version 2: the "Activities" which guide users through common tasks like announcements and polls, so I was keen to know more about where the inspiration for this came from. "In the research we found that most people were wanting to use the software to carry out three or four core types of tasks (such as conducting a poll)," he said. "Version 1 of FrontlineSMS required users to put the pieces together themselves when doing these tasks, which meant that many users were unable to unlock the full potential of the software. I realized we needed to do two things: Make it easier for people to do more complex things with the software, and also help people appropriately manage the information that was coming in and going out in relation to each of these different activities. So we created this idea of Activities -- if we know you wanted to create a poll, for example, we could guide you through the steps of setting it up, and then help you manage and understand the responses coming back in. With Activities, people do not need to put the pieces together themselves -- the software now supports them through the whole process by providing pre-packaged sets of tools."

Activities FrontlineSMS 2.jpg

Moreover, the system was designed to inspire people to make the most of FrontlineSMS and explore more sophisticated uses of SMS. Gabe elaborated: "Activities expose people to the possibilities of what they can do with the system. FrontlineSMS users have always been aware there was potential, but some didn't know they could do more advanced things with the software. Activities make it much more explicit and easy to understand. It's now more obvious about potential possibilities and so makes everything much more approachable."

the elements of design

When we presented early designs to users to seek their feedback, one person highlighted the power of the "email metaphor," particularly in reference to the ability to star messages or select multiple messages using check boxes. I wondered to what extent Gabe's design was influenced by online tools like Gmail and Facebook. His response: "As a designer one of the things I think about is: What are the design approaches or metaphors that people are familiar with and makes most sense to them? Design most often is not about creating completely new and radical solutions; rather it's about bringing together elements and metaphors that people already deal with in novel and interesting ways."

Gabe's approach was logical and meticulous, sticking to predictable behavior to ensure the usability of the user interface. It wasn't until after building user personas, choosing the task-based "Activity" concept and creating over 100 pages of design documentation that we first saw the first line of Version 2 code and a blue hyperlink for "Inbox" in summer 2011. Now that it's fully working software, I sometimes have to rub my eyes to believe how far we've come. What I love the most is hearing what people think, because that is what's central to user interface design. So find out about what's new in Version 2 here and share your ideas on what you think of the design on our forum here.

Gabriel White's company Small Surfaces designs user interface solutions for smartphones, tablet computers and beyond. His award-winning designs have helped organizations including FrontlineSMS, Ushahidi, World Vision, and Refugees United, as well as business leaders like Google, Samsung, Hewlett-Packard, Microsoft, Sandisk and Kodak deliver innovative, next-generation products and services. Gabe continues to work on new features and designs for FrontlineSMS.

Amy joined FrontlineSMS at the beginning of 2011 and is coordinating the FrontlineSMS:Radio project. This is a tailored version of FrontlineSMS's free and open-source software which is customized for radio DJs to help them interact with their audiences via text message. The project has involved offering user support to the growing community of radio users who are interested in solutions for the management of SMS and translating their needs into the software development process. Previously, Amy has worked for the Unrepresented Nations and Peoples Organisation, Amnesty International and Action Against Hunger.

A version of this post originally appeared on the FrontlineSMS blog.

August 08 2012


How the Knight Lab's Babl App Helped Lollapaloozans Deal with Storms

This post was written by Jordan Young of the Knight News Innovation Lab.


This past weekend marked the annual music carnival known as Lollapalooza" held in Chicago's Grant Park. As you'd expect, close to 100,000 people attending a large event can generate a lot of hot conversations on social media outlets.

The Knight News Innovation Lab recently released a mobile application, Babl, which gives users a unique way to share and discover news. This iPhone app offers a visual alternative to reading through a scrolling list of tweets. Babl users can create their own conversation topics by entering a title and keywords. The app uses the terms entered to create and display a collage of tweeters' photographs that can then be tapped to reveal their individual tweet.

behind the scenes

Prior to Lollapalooza, we set up a featured topic for the opening day of the fest allowing any user to sample the news, conversation and entertainment as it happened. We thought it might be fun to see the app in action during a lively event -- and apparently Mother Nature agreed by bringing severe thunderstorms to the Chicago area and forcing an evacuation of the park.

Thumbnail image for IMG_2620.jpg

Through Babl, we were able to participate in Twitter conversations about Lollapalooza throughout the weekend, starting on Friday as people filed into Grant Park to see their favorite artists and dance like neon-clad wild animals. On the afternoon of Day 2, tweets brought us the first news of the show being suspended due to an incoming tempest. Babl users were able to view reports like official news tweets, tweets from artists, and tweets from the herd of people as they were being evacuated into the streets of downtown and parking garage shelters -- most attendees opted for bars.

A few hours later, all the weather drama subsided and Babl displayed tweets of people re-entering the gates and enjoying the rest of the evening through Sunday's closing. Babl enabled us to easily view the local and global tweeters participating in a conversation topic, and gave us a rich media experience of an event in real time.

Jordan Young has been part of the Knight News Innovation Lab since its launch in August of 2011. She is a freelance blogger, contributing writer for Illinois Meetings + Events Magazine, and aspiring publisher. You can reach her at knightlab@northwestern.edu and on Twitter: @knightnewslab.

KnightLogo.jpgEstablished in 2011 with a $4.2 million grant from the John S. and James L. Knight Foundation, the Knight News Innovation Lab is a joint initiative of Northwestern University's Robert R. McCormick School of Engineering and Applied Science and the Medill School of Journalism. In partnerships built across the Chicagoland region -- from neighborhood bloggers to large media companies -- the Lab invents, improves and distributes technology that help build and sustain a better informed citizenry and a more innovative publishing environment.

August 07 2012


From Japan to Burning Man, LocalWiki Heats Up the Summer

We want to let you know about some really fantastic stuff that's happened in the LocalWiki world over the past month.

Tallahassee, Fla.

The folks spearheading the TallahasseeWiki project held their first two in-person CampWiki workshops. The idea behind the workshops is to introduce community members to the TallahasseeWiki, get them excited, answer questions and start building out the project.

Below are some photos of the CampWiki workshops:




Their meetup even made the front page of the Tallahassee Democrat!

Olympia, Wash.

OlyWiki held its first little in-person meetup. Unfortunately, they didn't take any pictures, so here's a photo of Seth Vincent, the project leader, putting up some flyers around town:


Oakland, Calif.

A group of really great folks have started laying the groundwork for an OaklandWiki project, and during the recent Code for Oakland hackathon there were around 30 people digging into OaklandWiki. The group was so large that it was broken into two rooms: one for helping with development/tech stuff and the other for content and planning.

Below are some images from the OakWiki workshop:



Raleigh, N.C.

The Raleigh City Council has begun to investigate how to, in its official capacity, best work with the TriangleWiki project! Here's a clip from their recent meeting where council member Bonner Gaylord asks city staff to come up with a report on how to collaborate with the project:

Tokyo area, Japan

Thanks to the internationalization work done by Pedro Lima and Nuno Maltez in Portugal, there's been an increasing amount of international interest in starting LocalWiki projects. A couple of weeks ago, Shu Higashi gave a demo of a LocalWiki to a group of open data activists in Japan. The best part? He demoed his Japanese translation of LocalWiki!

Black Rock City (Burning Man)


Some folks are starting up a LocalWiki project for Black Rock City/Burning Man!

It's literally just getting started, but it's such a cool idea we wanted to share it with you.

LocalWiki Organizers mailing list

Organizing a new LocalWiki project or wanting to get started? You should join the ultra-new LocalWiki-organizers mailing list! Be sure to send a little introduction to the list after you've joined.

Miscellaneous awesomeness

In no particular order, a few other interesting things that have happened over the past month:

Tallahassee photos courtesy Bob Howard. Oakland photos courtesy Eddie Tejeda.


Tired of Text Spam and Dropped Cell Phone Calls? You're Not Alone

Think you're the only one ready to throw your cell phone out the window the next time you have a dropped call or text spam? You're not alone, according to a new report from the Pew Research Center's Internet & American Life Project. The survey found that cell phone problems are a common reality for the 280 million users in the United States.

The report identified four major cell phone problems: 72 percent of all cell users experience dropped calls, 68 percent of all cell users receive unwanted sales or marketing calls, 69 percent of text messaging users in the U.S. receive unwanted spam or text messages, and 77 percent of those who use Internet on their cell phones experience slower than desirable download speeds.


The report also surveyed the frequency of all four mobile phone problems as experienced by smartphone owners. And in all four cases, smartphone owners reported higher incident rates. The largest margins are in spam and unwanted texts -- 29 percent of smartphone owners compared with 20 percent of other cell owners -- and slow download speeds -- 49 percent of smartphone owners compared with 31 percent of other cell owners.

Limited Solutions to Block Spammers

There are several ways people may attempt to remove cell phone nuisances from their daily lives. Step one is to contact your mobile carrier and request the available spam-blocking services.

University of New Hampshire student Feier Liu uses a non-smartphone and first called her mobile carrier to block a spam number about three years ago. The service was free, but only blocks individual numbers. Liu said she hasn't received a spam call since. She is certainly a lucky one.

Another service introduced back in March also counts on mobile users to vigilantly report spam text messages. North American mobile carriers have adopted a centralized spam-reporting service, which collects spam complaints into a shared database to help carriers identify and stop spammers. In practice, users forward spam texts to the shortcode 7726 (or SPAM), prompting the carrier to request the spam number.


Allin Resposo, a web designer and smartphone user, has been reporting every spam text to 7726 since the service was introduced. Resposo hasn't seen an obvious decrease in spam and said that the spam texts are never from the same number.

While smartphones experience more problems, they paradoxically enable more possible solutions. A search for "block spam" on Google Play brings up dozens of apps created to block spam calls and texts. Most of these apps have ratings of four stars or more and could be worthwhile efforts for Android users. However, because of Apple's restrictions on developers, similar apps are not available for the iPhone, which, according to a prior Pew report, is used by some 53 million people in the U.S.

Finding Digital Authenticity

The Pew report stated, "It is against the law in the U.S. to place unsolicited commercial calls to a mobile phone when the call is made by using an automated random-digit dialing generator or if the caller uses a pre-recorded message." Yet spam phone calls, like those offering free cruises to the Bahamas with a pre-recorded "[fog horn] This is your captain speaking" are as real as ever. Clearly, spammers are evolving faster than legislation.

In fact, they may be piggybacking on our mobile dependence. The report also noted that non-white cell owners experience all four of the common cell phone problems at higher weekly rates than white cell owners, possibly due to the fact that "African-Americans and Hispanics are more likely than whites to rely on their cell phones as their primary or exclusive phones for calling and for Internet access."

Does all this indicate that more mobile usage equals more problems?

In a world where there are 14 million spam accounts on Facebook and probably similarly disturbing figures on other social networks, it's not hard to imagine that spammers on these mobile-enabled networks will find a way to spam our mobile devices.

Jenny Xie is the PBS MediaShift editorial intern. Jenny is a rising senior at Massachusetts Institute of Technology studying architecture and management. She is a digital-media junkie fascinated by the intersection of media, design, and technology. Jenny can be found blogging for MIT Admissions, tweeting @canonind, and sharing her latest work and interests here.

This is a summary. Visit our site for the full post ».


Google Ventures invests $8.2m in electronic signature tech company DocuSign

The Next Web :: Electronic signature technology company DocuSign recently announced that it had raised $47.5 million in funding from Kleiner Perkins Caufield & Byers, Accel Partners, Comcast Ventures, SAP Ventures and an unnamed “large, global institutional investor”. Well-known KPCB partner Mary Meeker also joined the company’s board.

A report by Robin Wauters, thenextweb.com


Korea's SK Telecom claims world first with HD voice over LTE service

The Next Web :: Fresh from launching the world’s first multi-carrier mobile service last month, Korea’s SK Telecom has claimed another new landmark as the first operator to introduce an all-IP based HD voice over LTE service.

A report by Jon Russell, thenextweb.com

Tags: Technology

August 06 2012


An URL with ".book" costs $185,000 and you might have to wait a year

ReadWriteWeb :: The plan to create and sell new generic top-level domains, or gTLD, should have been a boon for the Internet and a gold mine for the Internet Corporation for Assigned Names and Numbers (ICANN). It’s not turning out that way. At nearly every turn, the process has been hamstrung by mistakes and a lack of foresight.

[Brian Proffitt:] ICANN sells these new domains - the letters that go after the dot in a ULR, like .com or .org - for $185,000 a pop - so you’d think that the agency would be motivated to get the process right.

A report by Brian Proffitt, www.readwriteweb.com

Tags: Technology

August 02 2012


Stable at Last, PANDA Reaches 1.0!

Eleven months ago, we began prototyping PANDA. The PANDA project aims to make basic data analysis quick and easy for news organizations, and make data sharing simple. I hacked for a month on an experimental version, verified that our technology choices worked, and then threw it out and started over. Since that time, development has proceeded in steady, week-long iterations, checkpointed by numerous releases and two-day long PANDA team-planning sessions. We've implemented every feature from our "must have" list, a large chunk of our "want" list, and even one or two off our "not likely" list (in response to user feedback).

Today, I'm pleased to announce that we have reached the end of our road map: PANDA Version 1.0 is ready!



If you've been taking a wait-and-see approach to getting PANDA in your newsroom, now is the time to see. Version 1.0 is the most polished release we've ever done. Among the highlights:

  • New user-oriented documentation at pandaproject.net.
  • No more default user accounts. A setup mode allows you to configure an admin user after installation.
  • Search for data within categories.
  • Additional metadata for datasets, including "related links."
  • Many, many, many bug fixes.

To get started with PANDA now, head over to our installation docs.

I have one month left to keep working full-time on PANDA. That means you have a month to get personalized help with any issues you encounter while setting up. If you get started now, I'll be answering your emails, tracking your bugs, and logging your future development requests. If you wait, you may have to get in line.

Still not persuaded? Check out an awesome presentation from Nolan Hicks, San Antonio Express-News reporter and PANDA beta tester.

Every newsroom can be a data-friendly newsroom. Get started with PANDA now.

August 01 2012


Can Google Maps + Fusion Tables Beat OpenBlock?

WRAL.com, North Carolina's most widely read online news site, recently published a tool that allows you to search concealed weapons permits down to the street level. It didn't use OpenBlock to do so. Why?


Or, if you're like many journalistically and technically savvy people I've spoken over the last few months, you could ask why would they? There's plenty of evidence out there to suggest the OpenBlock application is essentially a great experiment and proof of concept, but a dud as a useful tool for journalists. Many of the public records portions of Everyblock.com -- OpenBlock's commercial iteration -- are months if not years out of date. It can't be found anywhere on the public sites of the two news organizations in which the Knight Foundation invested $223,625. There are only three sites running the open-source code -- two of those are at universities and only one of which was created without funding from the Knight Foundation.

And, you, Thornburg. You don't have a site up and running yet, either.

All excellent points, dear friends. OpenBlock has its problems -- it doesn't work well in multi-city installations, some search functions don't work as you'd expect, there's no easy way to correct incorrect geocoding or even identify possible failures, among other obstacles that I'll describe in greater detail in a later blog post. But the alternatives also have shortcomings. And deciding whether to use OpenBlock depends on which shortcomings will be more tolerable to your journalists, advertisers and readers.


If you want to publish news from multiple cities or from unincorporated areas, or if you serve a rural community I'd hold off for now. If you visit our public repositories on GitHub you can see the good work the developers at Caktus have been doing to remove these limitations, and I'm proud to say that we have a private staging site that's up and running for our initial partner site. But until we make the set-up process easier, you're going to have to hire a Django developer (at anywhere from $48,000 a year to $150 an hour) to customize the site with your logo, your geographic data, and your news items.

The other limitation to OpenBlock right now is that it isn't going to be cheap to maintain once you do get it up and running. The next priority for me is to make the application scale better to multiple installations and therefore lower the maintenance costs. Within the small OpenBlock community, there's debate about how large of a server it requires. The very good developers at OpenPlans who did a lot of heavy lifting on the code between the time it was open sourced and the time that it should run nicely on a "micro" instance of Amazon's EC2 cloud hosting service -- about $180 a year.

But we and Tim Shedor, the University of Kansas student who built LarryvilleKU, find OpenBlock a little too memory intensive for the "micro" instance. We're on an Amazon Web Services "small" instance, and LarryvilleKU is on a similar sized virtual server at MediaTemple. That costs more like $720 a year. And if you add a staging server to make sure your code changes break in private instead of public, you're looking at hosting costs of nearly $1,500 a year.

And that's before your scrapers start breaking. Depending on how conservative you are, you'll want to set aside a budget for fixing each scraper somewhere between one and three times a year. Each fix might be an hour or maybe up to 12 hours of work for a Python programmer (or the good folks at ScraperWiki). If you have three data sources -- arrests, restaurant inspections and home sales, let's say -- then you may get away with a $300 annual scraper maintenance cost, or it may set you back as much as $15,000 a year.

I've got some ideas on how to reduce those scraper costs, too, but more on that later as well.

Of course, if you have someone on staff who does Python programming and whose done some work with public records and published a few Django sites and they've got time to spare, then your costs will go down significantly.

But just in case you don't have such a person on staff or aren't ready to make this kind of investment, what are your alternatives?


Using a Google Map on your news website is a little like playing the saxophone. It's probably the easiest instrument to learn how to play poorly, but pretty difficult to make it really sing. Anyone can create a Google Map of homicides or parking garages or whatever, but it's going to be a static map of only one schema, and it won't be searchable or sortable.


On the other hand, you can also use Google Maps and Fusion Tables to build some really amazing applications, like the ones you might see in The Guardian or on The Texas Tribune or WNYC or The Bay Citizen. You can do all this, but it also takes some coding effort and probably a bit more regular hand care and feeding to keep the site up-to-date.

I've taken a look at how you might use Google's data tools to replicate something like OpenBlock, although I've not actually done it. If you want to give it a whirl and report back, here's my recipe.


Step 1. Create one Google Docs spreadsheet for each schema, up to a maximum of four spreadsheets. And create one Google Fusion Table for each scheme, up to a maximum of four tables.

Step 2. If the data you want is in a CSV file that's been published to the web, you can populate it with a Google Docs function called ImportData. This function -- as well as its sister functions ImportHTML and ImportXML -- will only update 50 records a time. And I believe this function will pull in new data from the CSV about once an hour. I don't know whether it will append the new rows or overwrite them, or what it would do if only a few of the fields in a record change. If you're really lucky, the data would be in an RSS feed and you could use the ImportFeed function to get past this 50-record limit.

Of course, in the real world almost none of your data will be in these formats. None of mine are. And in that case, you'd have to either re-enter the data into Google Docs by hand or use something like ScraperWiki to scrape a datasource and present it as a CSV or a feed.

Step 3. Use a modification of this script to automatically pull the data -- including updates -- from the Google Docs spreadsheet into the corresponding Fusion table you created for that schema.

Step 4. Find the U.S. Census or local county shapefiles for any geographies you want -- such as ZIP codes or cities or school districts -- and convert them to KML.

Step 5. Upload that geographic information into another Fusion Table.

Step 6. Merge the the Fusion table from Step 3 with the Fusion table from Step 5.

Step 7. This is really a thousand little steps, each depending on which of OpenBlock's user interface features you'd like to replicate. And, really, it should be preceded by step 6a -- learn JavaScript, SQL, CSS and HTML. Once you've done that, you can build tools so that users can:

And there's even at least one prototype of using server-side scripting and Google's APIs to build a relatively full-functioning GIS-type web application: https://github.com/odi86/GFTPrototype

After all that, you will have some of the features of OpenBlock, but not others.

Some key OpenBlock features you can replicate with Google Maps and Fusion Tables:

  • Filter by date, street, city, ZIP code or any other field you choose. Fusion Tables is actually a much better interface for searching and filtering -- or doing any kind of reporting work -- than OpenBlock.
  • Show up to four different kinds of news items on one map (five if you don't include a geography layer).
  • Conduct proximity searches. "Show me crimes reported within 1 mile of a specific address."


The OpenBlock features you can't replicate with Google:

  • Use a data source that is anything other than an RSS feed, HTML table, CSV or TSV. That's right, no XLS files unless you manually import them.
  • Use a data source for which you need to combine two CSV files before import. This is the case with our property transactions and restaurant inspections.
  • Update more than 50 records at a time. Definitely a problem for police reports in all but the smallest towns.
  • Use a data source that doesn't store the entire address in a single field. That's a problem for all the records with which we're working.
  • Map more than 100,000 rows in any one Fusion table. In rural counties, this probably wouldn't be a concern. In Columbus County, N.C., there are only 45,000 parcels of land and 9,000 incidents and arrests a year.
  • Use data sources that are larger than 20MB or 400,000 cells. I don't anticipate this would be a problem for any dataset in any county we're working.
  • Plot more than 2,500 records a day on a map. Don't anticipate hitting this limit either, especially after the initial upload of data.
  • Parse text for an address -- so you can't map news articles, for example.
  • Filter to the block level. If Main Street runs for miles through several miles, you're not going to be able to narrow your search to anything relevant.
  • Create a custom RSS feed, or email alert.


And there's one final feature of OpenBlock that you can't replicate using Google tools without investing a good deal of manual, rote set-up work -- taking advantage of SEO or social media sharing by having a unique URL for a particular geography or news item type. Ideally, if someone searches for "home sales in 27514" I want them to come to my site. And if someone wants to post to Facebook a link to a particular restaurant that was scolded for having an employee with a finger-licking tendency (true story), I'd want them to be able to link directly to that specific inspection incident without forcing their friends to hunt through a bunch of irrelevant 100 scores.

To replicate OpenBlock's URL structure using Google Maps and Fusion Tables, you'd have to create a unique web page and a unique Google map for each city and ZIP code. The geography pages would display a polygon of the selected geography, whether it's a ZIP code or city or anything else, and all of the news items for that geography (up to four schemas, such as arrests, incidents, property sales, and restaurant inspections). That's 55 map pages.

Then you'd have to create a map and a page for each news item type. That's four pages, four Fusion tables, and four Google Docs spreadsheets.

Whew. I'm going to stick with our work in improving the flexibility and scalability of OpenBlock. But it's still worth looking at Google Maps and Fusion Tables for some small and static data use cases. Other tools such as Socrata's Open Data, Caspio and Tableau Public are also worth your time as you begin to think about publishing public data. Each of those have some maintenance costs and their own strengths and weaknesses, but the real trick for using all of these tools is public data that isn't in any usable format. We're looking hard at solving that problem with a combination of scraping and crowdsourcing, and I'll report what we've found in an upcoming post.

Ryan Thornburg researches and teaches online news writing, editing, producing and reporting as an assistant professor in the School of Journalism and Mass Communication at the University of North Carolina at Chapel Hill. He has helped news organizations on four continents develop digital editorial products and use new media to hold powerful people accountable, shine light in dark places and explain a complex world. Previously, Thornburg was managing editor of USNews.com, managing editor for Congressional Quarterly's website and national/international editor for washingtonpost.com. He has a master's degree from George Washington University's Graduate School of Political Management and a bachelor's from the University of North Carolina at Chapel Hill.

July 30 2012


Do We All Have iDisorders?

Raj Lal, a senior engineer for a mobile phone company, checked his iPhone at the dinner table before getting a searing look and some strong words from his wife in the middle of a romantic restaurant.

It was their 10th anniversary. Lal, 34, said he felt embarrassed about the scene, but more so that he didn't even think about it as he pulled out the smartphone.

Lal isn't the only one who can't escape the lure of his mobile device. Today, it's commonplace to compulsively check a smartphone or text friends on any, perhaps all, occasions, special or not.

This constant connectivity is so much a part of our culture that it's become a hot debate in the last few years -- so much so that two recently released books tackle the issue head on: Sherry Turkle's book, "Alone Together: Why We Expect More from Technology and Less from Each Other" and "iDisorder: Understanding Our Obsession with Technology and Overcoming its Hold on Us", in which psychologist Larry Rosen even coins a term for our problem: iDisorder.

The 'iDisorder'


"It's a mood disorder that's kind of a lifestyle," Rosen said. "I'm a tech devotee, but on another level, they are dangerous and encourage obsessive behaviors."

Rosen cites a study that shows that more than half of the iGeneration (born 1990 to 1998) and Net Generation (born 1980 to1989) respondents polled checked their text messages every 15 minutes or less. While the percentage goes down as the age goes up, Generation Xers (born 1965 to 1979) still make up 42 percent of frequent text-checkers, while 20 percent of Baby Boomers (born 1945 to 1964) seem to constantly check for calls. Rosen said it's easy for technology to become an addiction.

In Sherry Turkle's book, "Alone Together: Why We Expect More from Technology and Less from Each Other," she interviewed several teens on their mobile phone addiction. One refused to quit texting while driving, saying, "If I get a Facebook message or something similar posted on my wall... I have to see it. I have to." Others have chipped teeth or bruised themselves walking into furniture or objects while engrossed in their phones.

But similarly, older adults have also changed their expectations. In a chapter entitled "No Need to Call," Turkle interviewed several people who say an email or a text is sufficient and using the phone to speak to another person is just unnecessary. One 46-year-old architect explains it this way, "(A phone call) promises more than I'm willing to deliver."
Part of the reason, Turkle suggested, is that these people believe the human connection of a phone call is "asking too much, and they worry it will be received as demanding too much." Turkle even found herself not calling a close friend because it might be considered intrusive.


Part of the attraction is also the idea that by not using the phone, which can lead to an unscripted conversation that may portray them as awkward, email and texting lets people present themselves as they wish.

Infiltrating our most intimate moments

Technology is also taking a toll on our sex lives. "People are so addicted to that immediacy, that when they're up in the morning, before looking at their partner like they used to, they grab for the phone," said Mary Jo Rapini, a sex and relationship psychotherapist based in Houston. "Your smartphone isn't smart in bed."

It's also creating a generation of exhibitionists who are too shy to ask anyone out.

"What's intimate now is not what we used to consider intimate," Rapini said. "Teens are posting provocative pictures of themselves and sending nude pictures to each other."
Rapini said that sending nude photos is a powerful act for kids who feel pretty powerless. "They control it and feel good about it. But when you meet someone you have no control over it," she said. "Having a date and being judged, they don't have the skills to handle that."

The madness of multitasking

Rosen, a professor of psychology at CSU Dominguez Hills, said that human communication can't compete with technology's bells and whistles, likely the reason why some people often check their phones or messages while talking to another human being. "Technology overstimulates our brains with all the various sensory images we have," he said. "They're highly engaging. And what people aren't doing is taking time to let their minds calm down."

Instead of using your smartphone in the grocery line, Rosen advocates talking to someone else in line, looking at the magazines, or just taking time to decompress. "Go look at a flower, speak a foreign language or listen to music without ear plugs," he suggested.
Without this ability, people can't use metacognition, or the awareness of when to pay attention and determining when not to pay attention. "Instead they're thinking about their phone and what they're missing out on by not checking it," he said.

The idea of multitasking, which had become such a corporate buzzword, is also a bit of a parlor trick. More productivity is observed in "unitasking," but most multitaskers aren't convinced of that. "They're so interested in multitasking that when someone says, 'Let's talk face-to-face,' they can't do it," Rosen said. The same goes for students. "You can't listen to music and read a textbook at the same time."

Many in the psychological and medical fields have debunked the myth of multitasking as productive. Instead, the interruptions lead to more stress, attention difficulties, and poor decision-making.

The next generation

Another problem with our increasing Internet-laden society is that many children and young adults may believe all information found on the Internet is true. "There's no media literacy training," Rosen said. "You can't just assume your kids are good at parsing media."

The problem with teens and young adults is that their devices give them constant reinforcement and a squirt of dopamine, Rosen said. Worse, they feel they have to react to an incoming communication instantly or something bad will happen. That they will miss out on some conversation or news nugget that may be life-changing, he said. "That's compulsion," he said. "The Fear of Missing Out."

The biggest challenge and first step for those addicted is turning off the phone while they go to sleep, Rosen said.

And how do you know you're an addict? You start finding reasons to get up from the table so you can check your phone without anyone noticing. Manufacturing a reason is a bad sign. "You have to practice being strong," Rosen said.

So why are children allowed to become so dependent on a smartphone? Because for many parents, having a child quiet and not needing attention is considered a blessing to an overworked parent, he said.

Dinners Without Technology

Parents can help lessen the grip of the smartphone by not using their own smartphone as much and establishing "dinners without technology." Mainly he said he blamed many parents on what he termed "partial parenting," where parents continuously give children partial attention. "That's not how you should parent and the kids are not going to be OK," he said. "Learn to focus better and longer."

Mitchell Weiss, a business professor at the University of Hartford, told MediaShift that he worried about his students in the business world. "They avoid eye contact," he said. "You can see them tense when you get too close...The failure to make eye contact used to mean hiding something or being devious, but you can't think that way when dealing with these kids. Their social skills are not as developed."

Weiss said that many of the bad behaviors learned at home and school won't work at a job. "You can't be text-messaging back and forth with friends," he said. "Even having a conversation with an employee, their phone will make a noise, and I can see their eyes look over to the phone. And I can see they really want to pick up the phone but know they can't."

Despite the momentary lapse at his recent anniversary, Lal said that he tries to have more meaningful moments with friends and family. "The more you talk with people, the richer the experience," he said. "And that's more important than surfing the Web or talking to friends on Facebook."

Related Reading

> Special Series: Unplug 2012

> Hands-Free Parenting: How Much You Gain When You Unplug by Rachel Stafford

> 5 Tips to Prevent Digital Burnout and Maintain Good Mental Health by Sandra Ordonez

Barbara E. Hernandez is a native Californian who lives in the San Francisco Bay Area. She has more than a decade of experience as a professional journalist and college writing instructor. She also writes for Press:Here, NBC Bay Area's technology blog.

This is a summary. Visit our site for the full post ».

July 26 2012


May 04 2012


Jonathan Zittrain Takes the Stage at ROFLCon

Today with MIT Civic Media Center's Matt Stempeck and Stephen Suen, I'm live-blogging ROFLCon, a conference for things and people who are famous on the Internet. The livenote index is here.

Christina Xu, the event organizer, starts off ROFLCon to cheers. It's an amazingly packed venue. "One out of eight people in this room have done something crazy on the Internet," she says.


Zittrain on memes and society

Jonathan Zittrain is an Internet phenomenon. Emerging from humble beginnings as a longtime CompuServe forum sysop, he is now professor of law at Harvard Law School where he co-founded the Berkman Center for Internet and Society.

He starts by saying that fame can be tricky: "Just before the talk, someone came up to me and said, 'are you the huh guy? I thought you were the huh guy! I'm not that famous. I can aspire. In this room is the engine that makes the Internet sing ... Who's minding the store? Is this going to be a day without memes?"

"Where's Tron Guy?" asks Zittrain. Tron Guy, in full costume, raises his hand, and the room bursts into applause.

Zittrain says he isn't sure if he's one of these "Internet ROFL people" -- hence the tie. It's hard to explain what you're doing this weekend to friends and family who are not part of this tribe, he quips.

But he does have some background in the Internet. He shows us a picture of him using a Texas Instruments home computer with a 300-baud modem, with the obligatory model rockets, and the Webster's Collegiate Thesaurus -- just because you might run out of words.

Zittrain used to work for CompuServe and also got involved in politics. He threw his weight behind Mondale/Ferraro 1984. "At least I carried Minnesota," he says. "And the District of Columbia." When he wasn't doing those things, he was usually spending time stuffed inside a locker. "Whatever that does not stuff you so that you die, makes you stronger," he noted.

Zittrain thinks the image of a nerd stuffed in a locker helps us understand memes -- the dramatic moment of pathos.

"They're all crazy; I'm normal ... they're bad, and we're good. And here's to us for being good," he says. But that opens us up to the charge that this culture, the Internet, is not real life, and is rather a form of retreat. At the base of a lot of memes is some authentic, unguarded voluntary moment, Zittrain says. There's artifice around it, but there is often something authentic beneath it. That's not always the case -- consider Dramatic Hamster. Sometimes a hamster isn't a hamster. But there are other times that it's striking closer to a certain chord.

Wires can be crossed when this culture is commercialized. The nerds struck back against Hot Topic when they produced a T-shirt of Rage Guy.

unstaged authenticity

There's something about commercialization which is always at arm's length of Internet culture. Zittrain talks to us about the most recent Calgary Comic Con, where they invited the entire cast of Star Trek: The Next Generation. Going to the cons involves waiting in lines to get your photo with the cast. It has an Apple Commercial 1984 feel to it -- take a photo with the cast, you cannot touch the cast. He tells us about one of the least proud moments of The Oatmeal, a contest for advice features. There appears to be a negative attitude towards those who intentionally try to "engineer" a meme. People don't like being prompted -- it feels like trying too hard, feels inorganic.

We like unstaged authenticity, like Disaster Girl, who grins deviously as a house burns to the ground behind her. She rather enjoys the attention, and we are pleased to see her embrace her inadvertent success, but there are still lines that you can cross. The point at which you're running your own network and have a store-- maybe not.

Internet Fame is like winning the lottery -- it seems good until someone gets killed. What better example of this ambivalence than Star Wars Kid? So far as he knew, this was an exercise that would be completely private. He didn't realize that when he turned the camcorder in at school that it would be posted to YouTube. Jonathan shows us the video of the the Matrix Version of Star Wars Kid. In Wikipedia, there's a debate on the talk page on whether or not it is right for Wikipedia, the knowledge repository of record for humanity, to include his name in the page. Ultimately, they decided not to name him, despite the fact that the mainstream media has done it several times. And people on Wikipedia fell into line-- upholding the process with which they disagreed.

Can we build an infrastructure of meme propagation that respects people's preferences. He shows us one of the Awkward Family photo sites, with an image that says, "Image removed at request of owner." There are enough yuks to go around, so why not take down private content when someone asks us to?

Jonathan would love to see an infrastructure built native to the web which makes it possible for people to opt out of the celebrity of being a meme. This isn't DRM, but maybe something like robots.txt (a directive that tells web crawlers like Google which subdirectories not to index). Search companies respect robots.txt. No Internet organization created this. But people and companies respect it anyway-- a way to say, "Do you mind?" This is often used with court documents. How could we build this into our technology and our culture? One guy made a T-shirt that reads, "I do not agree to the publication of this photo."

In short, how can we enjoy the culture of Lulz which also respecting people's wishes?

A longer version of this post can be found on the MIT Center for Civic Media blog.

May 03 2012


How We Got Here: The Road to Public Lab's Map Project

Last week, Public Laboratory announced that public domain maps are now starting to show up on Google Earth and Google Maps. But how did the projects get there? Here's a timeline of a Public Laboratory map project.

Making a map

Public Laboratory projects take a community-based approach to making maps that differs depending on where you are and the reason you want to create a map. People map areas for a number of reasons, including because there's a need to monitor an area of environmental concern, a dynamic event is happening that there's a desire to capture, or you cannot find adequate aerial image data. Before going out to map, preparing for fieldwork starts with the Public Lab map tools page, where you can discover what type of equipment to use and how to safely use it. Multiple research notes on how to do things such as setting up a dual camera rig and stabilizing the camera with a picavet can help with specific problems, but there are also hundreds of people in the online Public Lab community of mapmakers, sharing tips and experiences on the site.

Upon return

After the mapping flight, the map making begins with backing up the images and sorting through the set, making a subset for map production. Depending on the time in the air, there will be hundreds and sometimes thousands of individual images. Depending on the area of interest, you can hone in on which images will be used in creating the map. Assuming the flight was at a steady altitude, the images that you want to select are the sharpest ones that are vertically oriented. If you have many images for the same area, pick the best one, but also pick overlapping images so that there is plenty of overlap among the different images in the next step.

mapmill.jpg Public Laboratory's MapMill.

Images can be sorted locally or online. Public Laboratory created an online tool where a group can do collaborative selection. MapMill.org is a web-based image sorting and ranking tool where multiple users can sort through a large dataset simultaneously.

Map production

With a smaller set of the best images on hand, the images can be dynamically placed on the map in a process known as georectification. After all the images have been added to the map, the project is exported. The MapKnitter export tool does all of the geographic information systems crunching behind the scenes with the geospatial data abstraction library (gdal.org) and produces a GeoTIFF map file. The GeoTIFF format is a public domain metadata standard that embeds geographic information into the image TIFF file. At this point, the map is now in an interchangeable format that can be easily distributed.

MapKnitter.jpg Public Laboratory MapKnitter web-based aerial image map production tool.

Public Laboratory Map Archive

Public Lab hosts its own map data archive for storing and sharing finished map projects. Each map in the archive has a "map details page" that hosts details such as: title, date, place, location, resolution, field map maker, field notes, cartographer, ground images, oblique images from the flight, and comments from website users. The map participants choose whether to publish the map as Public Domain, Creative Commons Attribution ShareAlike, Creative Commons Attribution, or Creative Commons Attribution Non-Commercial.

occupy-oakland.jpg Public Laboratory Occupy Oakland, November 2, 2011 -- General Strike map in Google Earth.

Maps are viewable on the archive itself, and you can subscribe to it as an RSS feed. However, it's also a place for distribution of the data. As we announced last week, Google Earth has started licensing our public domain maps. Google Earth plans to continue to publish public domain maps from the Public Lab Archive a few times a year.

It's quite exciting to see these Public Labs maps go online with a ubiquitous data provider such as Google. We look forward to more people participating in this activity, and more publishing of public domain data.

rifle.jpg Google published some of the maps to Google Maps as well as Google Earth, which makes those maps widely accessible in the web browser and on mobile applications that use Google Maps.

April 30 2012


How to Contribute to OpenStreetMap and Grow the Open Geodata Set

Hundreds of delegates from government, civil society, and business gathered in Brasilia recently for the first Open Government Partnership meetings since the inception of this initiative. Transparency, accountability, and open data as fundamental building blocks of a new, open form of government were the main issues debated. With the advent of these meetings, we took the opportunity to expand an open data set by adding street names to OpenStreetMap.

Getting ready to survey the Cruzeiro neighborhood in Brasilia.

OpenStreetMap, sometimes dubbed the "Wikipedia of maps," is an open geospatial database. Anyone can go to openstreetmap.org, create an account, and add to the world map. The accessibility of this form of contribution, paired with the openness of its common data repository, holds a powerful promise of commoditized geographic data.

As this data repository evolves, along with corresponding tools, many more people gain access to geospatial analysis and publishing -- which previously was limited to a select few.

When Steve Coast founded OpenStreetMap in 2004, the proposition to go out and crowdsource a map of the world must have sounded ludicrous to most. After pivotal growth in 2008 and the widely publicized rallying around mapping Haiti in 2010, the OpenStreetMap community has proven how incredibly powerful a free-floating network of contributors can be. There are more than 500,000 OpenStreetMap contributors today. About 3 percent (that's still a whopping 15,000 people) contribute a majority of the data, with roughly 1,300 contributors joining each week. Around the time when Foursquare switched to OpenStreetMap and Apple began using OpenStreetMap data in iPhoto, new contributors jumped to about 2,300 per month.

As the OpenGovernment Partnership meetings took place, we wanted to show people how easy it is to contribute to OpenStreetMap. So two days before the meetings kicked off, we invited attendees to join us for a mapping party, where we walked and drove around neighborhoods surveying street names and points of interest. This is just one technique for contributing to OpenStreetMap, one that is quite simple and fun.

Here's a rundown of the most common ways people add data to OpenStreetMap.

Getting started

It takes two minutes to get started with contributing to OpenStreetMap. First, create a user account on openstreetmap.org. You can then immediately zoom to your neighborhood, hit the edit button, and get to work. We recommend that you also download the JOSM editor, which is needed for more in-depth editing.

Once you start JOSM, you can download an area of OpenStreetMap data, edit it, and then upload it. Whatever you do, it's crucial to add a descriptive commit message when uploading -- this is very helpful for other contributors to out figure the intent and context of an edit. Common first edits are adding street names to unnamed roads, fixing typos, and adding points of interest like a hospital or a gas station. Keep in mind that any information you add to OpenStreetMap must be observed fact or taken from data in the public domain -- so, for instance, copying street names from Google is a big no-no.

Satellite tracing and GPS data

JOSM allows for quick tracing of satellite images. You can simply turn on a satellite layer and start drawing the outlines of features that can be found there such as streets, building foot prints, rivers, and forests. Using satellite imagery is a great way to create coverage fast. We've blogged before about how to do this. Here's a look at our progress tracing Brasilia in preparation for the OGP meetings:

Brasilia progress

OpenStreetMap contributions in Brasilia between April 5 and April 12.

In places where good satellite imagery isn't available, a GPS tracker goes a long way. OpenStreetMap offers a good comparison of GPS units. Whichever device you use, the basics are the same -- you track an area by driving or walking around and later load the data into JOSM, where you can clean it up, classify it, and upload it into OpenStreetMap.

Synchronizing your camera with your tracker

Synchronizing your camera with the GPS unit.

Walking papers

For our survey in Brasilia, we used walking papers, which are simple printouts of OpenStreetMap that let you jot down notes on paper. This is a great tool for on-the-ground surveys to gather street names and points of interest. It's as simple as you'd imagine. You walk or drive around a neighborhood and write up information that you see that's missing in OpenStreetMap. Check out our report of our efforts doing this in Brasilia on our blog.

Walking papers for Brasilia.

Further reading

For more details on how to contribute to OpenStreetMap, check out Learn OSM -- it's a great resource with step-by-step guides for the most common OpenStreetMap tasks. Also feel free to send us questions directly via @mapbox.


Ubiquity: LG Cloud to let users manage content on PCs, mobiles, smart TVs

siliconrepublic.com :: LG Electronics is launching its own cloud service, offering the beta opening on 1 May. LG Cloud will let users manage and consume content on Android smartphones, PCs and smart TVs.

Continue to read Laura O'Brien, www.siliconrepublic.com

Tags: Technology

April 29 2012


Going virtual: Technology doesn't (aim to) substitute "touch"

AdAge :: If the need arises, Dojo's team in San Francisco can have face time with clients in Europe at the drop of a hat. That's the beauty of the 2-year-old agency's extensive video-conference setup, said Managing Director Tiffany Coletti Titolo. "We can present to a client in Fremont, Calif., or in Switzerland without losing days to travel or wasting a client's money." In an industry once famous for three-martini lunches, virtual communication has become ubiquitous as more agencies and clients scale back on travel. The question is if something's lost when business is done virtually.

Continue to read Kunur Patel, adage.com

April 27 2012


At the International Journalism Festival: Can Data Journalism Save Newsrooms?

PERUGIA, Italy -- Here at the International Journalism Festival the launch of three large initiatives have generated a lot of the buzz around data journalism.

The School of Data Journalism, organized by the European Journalism Centre and the Open Knowledge Foundation, is composed of three panels and five workshops and dives into some of the key issues that media organizations are currently considering: "Is it worth my while starting out trying to do data journalism?", "Will data journalism make us money?", "How do you get data that you can search, filter and analyze with a computer?" and "How do I make data stories sexy?"


In addition, the 58 nominations for the Data Journalism Awards (DJA) were announced. DJA is the first international competition that recognizes and showcases the great work done in data journalism. Prizes are awarded for data-driven applications, investigations, and storytelling through visualizations. It's hoped that these awards will encourage more news organizations to embark on more ambitious data projects and alleviate the "loneliness in the newsroom" which some data journalists experience when their colleagues don't understand what they do. The six winners will be announced May 31.

And on Saturday, the Data Journalism Handbook will be launched. The handbook was born at the Mozilla Festival in November. It's a collection of tips, anecdotes and case studies from more than 70 leading data journalists and data wranglers, including contributions from The New York Times, Zeit Online, the BBC, the Guardian and many more. The book will be an open educational resource with key lessons a beginner data journalist should know. You can see a chapter overview of the handbook here and an excerpt from the first chapter here. A free version will be available online at datajournalismhandbook.org, and an e-book and print version will soon be published by O'Reilly Media.

So what is data journalism?

The School of Data Journalism, a series of panel discussions and workshops at the festival, was led by leading practitioners from all over the world and aimed to show participants what data journalists can do and why they should take the plunge and learn new skills.

The definition of data journalism varies depending on whom you ask. For some journalists, it's simply the courage to tackle sometimes huge and messy datasets. For others, it's being transparent and open about "showing the working" behind their conclusions, backing up their stories with facts and numbers where one might previously have only evidenced their point with "he said/she said." For others, it's a new way of presenting data through visualizations and interactive news applications; news is no longer simply static words on a page.

Increasingly, though, many are coming to realize that data journalism is a set of skills, involving new methods for acquiring, analyzing and working with data which simply weren't computationally feasible before. In an age that is positively drowning in data, we need more data journalists who typically have better storytelling skills than statisticians and can act as translators of complex datasets for the benefit of the public.

As activist and author Heather Brooke put it in the "Information wants to be free" workshop, data journalism is a misnomer -- one doesn't say "telephone journalism" if you contact your sources via telephone; journalists have to use data to do their job well.

Guerrilla Tactics: how to get started with Data Journalism

In the first panel of the school, "From Computer Assisted Reporting to Data Journalism," Pulitzer Prize winners Sarah Cohen and Steve Doig, highlighted their experiences working in the United States, where the notion of Computer Assisted Reporting (CAR) has been around for several decades -- far longer than the budding data journalism scene here in Europe.

They described their experiences learning how to use tools and techniques -- unfamiliar to journalists but popular in other disciplines such as social science and history -- to stay at the cutting edge of journalism. They also described the "guerrilla tactics" they initially had to use to get their work into print. "If you produce an amazing visualization, your editor is going to find a way to get it published," Cohen said, adding that it's far easier to show someone what data journalism is than to explain what it is.


Next up, Aron Pilhofer described his journey to data journalism at the New York Times. He said it came from a feeling of frustration with the inefficiency of working practices and tools. This sentiment resonated strongly with the other panelists -- a common complaint concerned individual journalists holding onto their data, producing datasets that only they could understand, instead of resources that could be built on and expanded by others on their teams.

On the same panel, Elisabetta Tola of formicablu and Simon Rogers of the Guardian gave a European perspective on data journalism. Rogers demonstrated how the Guardian Datablog's interactive maps of the U.K. riots helped disband false statements by the government that the "riots were not about poverty." Tola then explained some of the more basic problems facing wannabe data journalists in Italy, some of whom would be lucky to get data even on paper, as it's common for officials to simply dictate the numbers to journalists.

The second panel, "How can data journalism save your newsroom?", examined perspectives and business models for data journalism, and attempted to answer the question: "Is it worth it?"

Caelainn Barr of Citywire urged journalists not to consider data journalism as a fix-all that will save anything. She warned that editors are unlikely to be considerate and give you more time just because you're using complex data or working hard to present it better. Barr said journalists are constantly playing a game of catchup; advertisers are moving elsewhere; and journalists have less time to produce their stories and are struggling to keep up. All of this means journalists have to be more agile and learn to do things more efficiently.

To solve this problem, Pilhofer said, the New York Times has built resources that live on for future stories, allowing both journalists and the interactive news team to spring into action as soon as a related story breaks.

"What is the simplest thing you can do to start with data journalism?" ProPublica's Dan Nguyen asked rhetorically. "Keep your notes in a spreadsheet." He said often, the skills required to find stories involve sorting, grouping and averaging the data. With skills this simple, can newsrooms really afford not to teach them to their journalists?

The Future of Journalism is Bold

What does the future look like for data journalism? "Data journalism is just becoming journalism," said the Guardian's Rogers -- which was possibly the most encouraging statement from any of the panelists here.

Data journalism is no longer limited to only those who can afford to pay $900 for a piece of visualization software. Now incredibly powerful, open-source solutions are available. Organizations such as ProPublica encourage others to use their approaches in other stories to bring data journalism to local levels.

However, a change in culture will be needed to get more journalists into the fold. As Tola explained, collaboration is key, both journalist-journalist and journalist-coder collaboration. As Wired Italy's Guido Romeo put it, "Journalism is a one-man band. Data journalism is clearly not."

As technology develops, the ways of presenting this information just become more exciting. Could Italy be a land of opportunity for data journalism? The enthusiasm with which the workshops were met gave the impression that he who dares first will have a serious competitive advantage.

The workshops will continue over the next couple of days, and many have spaces open. Any budding data journalists? Join us! The Data Journalism Handbook will be launched at 6:30 p.m. PDT on April 28.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
No Soup for you

Don't be the product, buy the product!

YES, I want to SOUP ●UP for ...