Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

June 02 2011

18:53

New in the Campaign Finance API: Paper Filings

Today we're announcing the addition of paper campaign filings to our Campaign Finance API, which previously had only provided details of electronically filed reports.

March 04 2011

15:43

Councilpedia a Hit with New Yorkers, But Not Politicians

It's been a month since Gotham Gazette launched its Councilpedia project to monitor city elected officials and track money in local politics. (To read our earlier entry on Councilpedia, go here.)

In those weeks, we've learned a lot about what people like and don't like about the service. This information will help us improve what we think is an important tool for New Yorkers and an example other local news sites might want to follow.

Popular with People, not Politicians

councilpedia grab small.jpg

First, by and large, people like it. Even though most of the information -- but not all of it -- was already scattered about on Gotham Gazette and other sites, readers appreciate having all that data in one place. As someone who, like most editors, usually only hears from readers when they have complaints, I enjoyed getting emails with comments like "love it," "great new tool," "great addition to an already fine website," and so on.

We also received favorable coverage from a number of local news organizations. The New York Daily News ran a story about Councilpedia as did the local cable news channel and some political blogs. The New York Post even used it to call out a councilmember who seems not to have done much work during the last year.

Some of the city officials did not share that enthusiasm. In particular, they did not like the focus on campaign contributions. Our information on this is not original -- we took it from the city's Campaign Finance Board, which keeps track of such things and makes them public on its own. It's a very useful site. We did, though, sort all that information in an attempt to make it more user friendly and informative. So with Councilpedia, readers can, with two clicks, find out which unions gave money to Councilmember X and which lawyers helped Councilmember Y.

What some council members really, really do not like, apparently, is that we identify contributions from the real estate industry. Real estate -- developers, brokers, construction -- are probably the most important special interest in NYC politics. Many New York politicians rely on their support. They just hope people won't notice. Councilpedia makes it a bit harder to keep that secret.

There's a lesson in there somewhere.

What We've Learned

Some of the lessons are already clear to us. One is that, while people visit the site and explore it, they have been slow to post comments. Getting the public to share information and having a discussion about money and politics are key to Councilpedia, so we will try to ramp that up.

In the next week or so, we plan to add a tutorial explaining more fully how to foster user interactions. We also hope to offer some short information sessions on Councilpedia and how to use it. And we expect that fresh information on the site -- the list of earmarks for the next fiscal year, for example -- will spur more people to get involved.

We're eager for other suggestions and would appreciate hearing from anyone who has done crowdsourcing and had a good response.

The second lesson: People would like to see more of this. They wonder why we did not include the mayor (Michael Bloomberg is a billionaire, so he doesn't take campaign contributions). Other readers wanted to see the information on state legislators, judges, and possible mayoral candidates.

More Money, More Monitoring

So would we. The problem, alas, is resources.

Councilpedia has something like 31,000 pages. While some of the data was copied or downloaded, much of it required formatting and tweaking by our technical manager JaVon Rice. And every single campaign contribution to all of the 53 officials in Councilpedia had to be hand-coded by sector and location. This required a large number of interns and freelancers working under the supervision of our city government editor, Courtney Gross. Even with a generous grant from the Knight Foundation, this stretched our resources to the limit and probably beyond.

New York's state legislature, which has been termed one of the most dysfunctional in the country and is awash is questionable campaign finance shenanigans, represent a tempting target for this type of project. Now if only we had a million dollars to do it...

February 10 2011

19:11

Updates to the Campaign Finance API

Political campaigns don't have an off-season, but the brief lull between last November's general election and now has given us time to make some updates to our Campaign Finance API.

November 29 2010

22:23

Campaign Finance API Updates

We've made a few tweaks and additions to the Campaign Finance API.

October 07 2010

03:35

Introducing Version 3 of the Campaign Finance API

For developers looking to find information about political candidates and committees, friendly data formats can be hard to find. Version 3 of The Times's Campaign Finance API aims to start making that process a bit easier.

October 06 2010

14:50

Pushing the Limits of What a Wiki Can Do with Councilpedia

Barely two decades into the digital age, we take online media for granted. So much is so easy and convenient -- at our fingertips -- that we can forget technology can only do so much. Then we come up with a great idea that leaves us with the challenge of how to successfully push the limits.

This is what has confronted Gotham Gazette as we move into the final stages of creating our Councilpedia site. Councilpedia, a Knight News Challenge winner that I've blogged about here previously, will explore more fully the links between money and politics in New York City.

Councilpedia will enable visitors to the site to share what they know about politicians and their donors. It is to be powered by MediaWiki to let people flag something -- noting, for example, that one contributor to a candidate owns land she hopes to get rezoned for a Walmart. Gotham Gazette staff will then confirm -- or delete -- the comment.

Filtering Data

The core of Councilpedia is information already on Gotham Gazette, information from City Council (on earmarks, for example) and, above all, the massive records from the city Campaign Finance Board on giving and spending. The sheer magnitude of all this data has posed an array of problems.

The city data, while thorough and accessible, is inscrutable to most New Yorkers -- a list of largely meaningless names. To make it easier to search and understand, we set out to code the data (to indicate large donors, those from the city, unions, real estate industry etc.). With some candidates having thousands of contributors, this presented a massive task. Fortunately, we had some conscientious interns this summer who, between their other reporting responsibilities, dutifully researched and coded line after line of information under the supervision of our city government editor, Courtney Gross.

Readers will be able to examine this data in a number of ways. They can view by candidate. They can find out who else the contributor helped fund. They can look at intermediaries and determine whose money they bundled and then who it went to. And so on.

For the wiki, though, this mountain of information has been a bit much. When technical manager William JaVon Rice began uploading the data into spreadsheets he had created, the process took 36 hours and produced some 31,000 pages -- a sure indication no one would ever attempt this in print. The system balked, overwriting pages, for example, which required Rice to check every candidate's list of often hundreds of contributors to determine which ones had been overwritten. Then he had to undo the overwrite.

Pushing The Limits of MediaWiki

We're still planning to have this ready to show you in the next several weeks. And we think you'll be impressed. Not to boast, but the reporters, campaign finance aficionados and followers of city government who viewed our test felt that way.

But we do see a number of issues looming ahead. Councilpedia is intended as a living, breathing site, meaning data will continue to accumulate as officials collect more money, award more earmarks, pass more bills, and so on. The updating poses a challenge for a small non-profit like Gotham Gazette.

The magnitude of the new information -- added to the volumes we already have -- is likely to push the limits of MediaWiki even further.

With this in mind, we're looking for ways to automate the process more. And we hope someone -- any takers out there?-- will make MediaWiki more robust or create or an alternative.

As always, we appreciate your ideas, so feel free to share them in the comments below. And stay tuned for Councilpedia.

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl