Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 03 2012

13:53

How We Got Here: The Road to Public Lab's Map Project

Last week, Public Laboratory announced that public domain maps are now starting to show up on Google Earth and Google Maps. But how did the projects get there? Here's a timeline of a Public Laboratory map project.

Making a map

Public Laboratory projects take a community-based approach to making maps that differs depending on where you are and the reason you want to create a map. People map areas for a number of reasons, including because there's a need to monitor an area of environmental concern, a dynamic event is happening that there's a desire to capture, or you cannot find adequate aerial image data. Before going out to map, preparing for fieldwork starts with the Public Lab map tools page, where you can discover what type of equipment to use and how to safely use it. Multiple research notes on how to do things such as setting up a dual camera rig and stabilizing the camera with a picavet can help with specific problems, but there are also hundreds of people in the online Public Lab community of mapmakers, sharing tips and experiences on the site.

Upon return

After the mapping flight, the map making begins with backing up the images and sorting through the set, making a subset for map production. Depending on the time in the air, there will be hundreds and sometimes thousands of individual images. Depending on the area of interest, you can hone in on which images will be used in creating the map. Assuming the flight was at a steady altitude, the images that you want to select are the sharpest ones that are vertically oriented. If you have many images for the same area, pick the best one, but also pick overlapping images so that there is plenty of overlap among the different images in the next step.

mapmill.jpg Public Laboratory's MapMill.

Images can be sorted locally or online. Public Laboratory created an online tool where a group can do collaborative selection. MapMill.org is a web-based image sorting and ranking tool where multiple users can sort through a large dataset simultaneously.

Map production

With a smaller set of the best images on hand, the images can be dynamically placed on the map in a process known as georectification. After all the images have been added to the map, the project is exported. The MapKnitter export tool does all of the geographic information systems crunching behind the scenes with the geospatial data abstraction library (gdal.org) and produces a GeoTIFF map file. The GeoTIFF format is a public domain metadata standard that embeds geographic information into the image TIFF file. At this point, the map is now in an interchangeable format that can be easily distributed.

MapKnitter.jpg Public Laboratory MapKnitter web-based aerial image map production tool.

Public Laboratory Map Archive

Public Lab hosts its own map data archive for storing and sharing finished map projects. Each map in the archive has a "map details page" that hosts details such as: title, date, place, location, resolution, field map maker, field notes, cartographer, ground images, oblique images from the flight, and comments from website users. The map participants choose whether to publish the map as Public Domain, Creative Commons Attribution ShareAlike, Creative Commons Attribution, or Creative Commons Attribution Non-Commercial.

occupy-oakland.jpg Public Laboratory Occupy Oakland, November 2, 2011 -- General Strike map in Google Earth.

Maps are viewable on the archive itself, and you can subscribe to it as an RSS feed. However, it's also a place for distribution of the data. As we announced last week, Google Earth has started licensing our public domain maps. Google Earth plans to continue to publish public domain maps from the Public Lab Archive a few times a year.

It's quite exciting to see these Public Labs maps go online with a ubiquitous data provider such as Google. We look forward to more people participating in this activity, and more publishing of public domain data.

rifle.jpg Google published some of the maps to Google Maps as well as Google Earth, which makes those maps widely accessible in the web browser and on mobile applications that use Google Maps.

January 04 2012

15:20

Public Lab Produces Wetlands Maps From Balloon and Kite Flights

The Public Laboratory for Open Technology and Science (PLOTS) is an organization and membership community which develops and applies open-source tools to environmental exploration and investigation. Public Laboratory's mapping tools, openly available and easy to use, are putting the ability to do processes such as georectifying in the hands of people who may have never created a map. 

Using aerial mapping techniques, residents and volunteers of the Gulf Coast region began field mapping trips in 2010 to document the impact of the BP oil spill. Between May 2010 and April 2011, tens of thousands of images were collected and 50 regional maps created. Between May and October 2011, Public Laboratory partnered with Dr. Alex Kolker, from the Louisiana Universities Marine Consortium to begin a bi-monthly monitoring of select oil impacted and non-impacted sites in the Barataria Bay region. The intent of this phase of wetlands mapping was to monitor change over time with high-resolution aerial and ground imagery.

slong-image00.jpg
Wilkinson Bay in Louisiana.

MAKING MAPS WITH GEORECTIFICATION
The fieldwork that goes into collecting images is the first step in creating maps. On the back end, the next step involves georectification -- the process where balloon and kite aerial images are "published" into geographic data. Simply, it is the alignment of an aerial image with a map or other spatial data of the same area. This part of the process is where images become maps and are associated with geographic standardized formats so that other users and programs can exchange and experience them in the same context.

Public Laboratory map production has used some specific techniques in creating wetlands maps. The maps are made through a georectification process by which adjacent images from the flights are merged in overlay as they are aligned to existing mapping information. Different "base" data can be used in these types of projects. In this case, we're georectifying the imagery through examination of the new images with existing imagery.

slong-image02.jpg

A new unaligned image (right) about to be georectified with the other imagery. The base layer is visible in the background.

Distortion correction is applied to the new images, and they're moved around so that the same features of the base map are in perfect overlay with the new image on top. In the bayou setting where lots of change is occurring on the outer coastlines, features such as vegetation composition and interior waterways are used to match each overlay. Although waterways in marshland may shift quite rapidly, the center of a 3-way intersection is quite stable, as a rule.

Matching the imagery based on the interior features has proven effective in this simple map-making technique. While the outer coastline in our new images has changed since the time of the base data, the process of fitting the imagery with all of the interior ground control features allows us to discover where the coastline is with some measure of confidence. Or to put it differently -- when the alignment happens with the historic data in most of the image, then the new areas can be extrapolated with regularity.

slong-image01.jpg

The image is aligned in overlay with the base data during the georectify process.

Visit the Public Lab site for guides and discussion about the process and to view image sets and maps that have already been published in the map archive. The archive is a home and distribution channel for published open-source maps.

Visit MapKnitter.org to learn about our map-stitching tools and to view maps that are being created by people who are using aerial mapping techniques in new ways to document and monitor sites and events that are of importance to their community.

October 29 2010

14:55

Mapnik: The Coolest Mapping Software You've Never Heard Of

On the MapBox website we describe TileMill — the project we’re working on with our 2010 Knight News Challenge grant — as “a toolkit for rendering map tiles”. To be more specific, it’s essentially a “glue layer.” TileMill is built on top of a cocktail of other open source mapping software projects, and its biggest value is streamlining other more complex tools into a clean and easier workflow. For users to take advantage of TileMill, it can be useful to understand some of the underlying parts. Perhaps the most important part of that cocktail is a lesser known open source project called Mapnik. In this post I’ll talk a little about what Mapnik is and the important role it plays in helping users style their maps, as well as how it relates to TileMill.

The goal of the TileMill project is to make it easy for anyone with some basic web design familiarity to design their own custom maps. In past posts on this site we’ve introduced readers to the general reasons why we think custom online maps are valuable and have shared a couple examples for when custom maps have been particularly helpful on websites. Mapnik makes all this possible by providing the core technology to apply styles to GIS data and then render maps based on those styles.

Here’s the basic idea with styling maps: raw GIS data in the form of shapefiles contains information about various “features” — for instance, place names, points (e.g. center of a city), lines (e.g. roads), or polygons (e.g. state or country borders). If you have the data in its raw form, you’re only part of the way toward turning it into a map. Next you need to decide how to style each element.

Mapnik in action, styling maps of Kabul, Afghanistan Mapnik in action, styling maps of Kabul, Afghanistan

The style of each feature (or lack thereof) is why maps of the same location might look different from others. At a simple level, you might want your primary roads to be red versus orange. Compare MapQuest to OpenStreetMap for instance, at the exact same zoom level — note the difference in the styles for the same features.

Boulder, CO on MapQuest Screenshot of Boulder, CO on MapQuest

Boulder, CO on OpenStreetMap Screenshot of Boulder, CO on OpenStreetMap

Setting aside conversations about which features you decide to show on a map and assuming your data is accurate (both are huge factors), how you choose to style certain features might be the next most important part of map design. Getting styling right is essential for your users and is central to map design. If you over style or under style features, it has a direct impact on the readability and effectiveness of your maps.

This is where Mapnik comes in — it provides the framework for styling map data and then rendering new maps based on those styles. Mapnik is an open source project that is heavily used by the team at Cloudmade, who are involved in styling OpenStreetMap, and it’s been used by MapQuest, who have even released their Mapnik map style files for the public. Our team uses it heavily too, and AJ Ashton and Tom MacWright from the MapBox team were recently in London at Cloudmade’s offices with a group of core contributors, including Mapnik’s creator Artem Pavlenko, for the first ever Mapnik code sprint.

But where professional mappers are able to leverage Mapnik in complex ways, it has its downsides for the average would-be map designer. For starters, it’s not easy for noobs to even install it, before anyone worries about using it. This is part of why we’re working on TileMill — we want to make it easier for people to take advantage of these powerful tools. TileMill puts a wrapper around Mapnik that makes it simple to set up and leverage the powerful map styling capacity that it provides.

If you’re interested in more details about Mapnik, check out the Mapnik website or a recent Q&A with Mapnik developer Dane Springmeyer about Mapnik performance on Development Seed’s blog.

August 24 2010

17:05

Helping D.C. Drinkers and Bikers with Custom Maps

In my last post about TileMill, I outlined some of our general plans and the background for why we’re working on this project to help make it easier for people to design very custom maps online. One question that we get a lot from people who are new to the GIS space is, “When would I need this? How could I hope to improve on what (fill in the blank: Google/Bing/etc.) make available?”

The answer is that it’s all about the details of the specific communications goal you want to accomplish. In many cases, Google and Bing maps are great. In other situations, having additional control over map design is crucial to reach your goal (or at least improve your delivery). To get a sense of the kind of situations where custom designed maps really make a difference, I’ll share a story about some maps that we made for our hometown of Washington, D.C.

In 2008 the Washington, D.C. Office of the Chief Technology Officer (OCTO) released a large set of municipal datasets to the public to coincide with the original “Apps for __” contest, Apps for Democracy. Included in this data were ESRI shapefiles, a great format for GIS pros but hard to work with for anyone else. Using this same geodata, we created three very different custom maps, each for different use cases.

StumbleSafely

The first was for a website we called StumbleSafely. It was a bit tongue-in-cheek — the idea was that the site could help users see the latest crimes near their favorite bars so they could be aware of problem areas. Because we weren’t actually helping people map out navigation paths to get home, the real communications point we wanted to hit with the map was showing crime in proximity to bars and subway stations. Street names didn’t matter as much, and neither did highlighting any other kinds of businesses.

The D.C. police department was publishing crime data that we could scrape and add to the site to show crime locations, and the Alcoholic Beverage Regulation Administration was publishing liquor license data that we could use to visualize density points on a map that corresponded to areas with lots of bars. For the base map itself, though, we needed little else in the way of data to accomplish our purpose. We were able to take shapefiles for roads, parks, and river features in the city and do a very low key map design that fit the aesthetic of the site and contained no extraneous information. People familiar with the city could quickly look at it and see right away what the crime situation was like in their favorite areas, without any other distractions.

Stumble Safely

DC Bikes

Another example was a very similar map, but for a very different use case. DC Bikes was designed on the same platform as StumbleSafely to provide a resource to the D.C. cycling community. It showed bike thefts, bike lanes, and bike shops around the city.

For the map on that site, we again wanted to show crime data and were able to take the same basic approach to StumbleSafely for the base map design, but there was one additional feature we needed — bike lanes. These were made available as a shape file from the D.C. government as well, so we were able to quickly repurpose the map from Stumble Safely, tweak the colors to match a new design, and highlight the bike lanes in a new color so they stood out. For the cycling community, we were again able to show just the more relevant information on the map and omit any distractions.

DC Bikes

DC Nightvision

Finally, we wanted a much higher level of detail for a different project. Rather than omit details from our map in this case, we wanted to pack it full of details about public infrastructure. Not just buildings and roads, but even crosswalks, medians, and topography lines. The map we released for the public, “DC Nightvision,” includes all of these details, each of which are again published by the D.C. government as shapefiles.

DC Nightvision

With the increasing availability of shapefiles like the ones mentioned here, TileMill will make it easy for end users without a lot of GIS training to churn out custom maps that meet their unique communications needs. With the data in hand and user-friendly tools to work with it freely available, creativity will be the only limit for creating great custom maps.

17:05

Helping D.C. Drinkers and Bikers with Custom Maps

In my last post about TileMill, I outlined some of our general plans and the background for why we’re working on this project to help make it easier for people to design very custom maps online. One question that we get a lot from people who are new to the GIS space is, “When would I need this? How could I hope to improve on what (fill in the blank: Google/Bing/etc.) make available?”

The answer is that it’s all about the details of the specific communications goal you want to accomplish. In many cases, Google and Bing maps are great. In other situations, having additional control over map design is crucial to reach your goal (or at least improve your delivery). To get a sense of the kind of situations where custom designed maps really make a difference, I’ll share a story about some maps that we made for our hometown of Washington, D.C.

In 2008 the Washington, D.C. Office of the Chief Technology Officer (OCTO) released a large set of municipal datasets to the public to coincide with the original “Apps for __” contest, Apps for Democracy. Included in this data were ESRI shapefiles, a great format for GIS pros but hard to work with for anyone else. Using this same geodata, we created three very different custom maps, each for different use cases.

StumbleSafely

The first was for a website we called StumbleSafely. It was a bit tongue-in-cheek — the idea was that the site could help users see the latest crimes near their favorite bars so they could be aware of problem areas. Because we weren’t actually helping people map out navigation paths to get home, the real communications point we wanted to hit with the map was showing crime in proximity to bars and subway stations. Street names didn’t matter as much, and neither did highlighting any other kinds of businesses.

The D.C. police department was publishing crime data that we could scrape and add to the site to show crime locations, and the Alcoholic Beverage Regulation Administration was publishing liquor license data that we could use to visualize density points on a map that corresponded to areas with lots of bars. For the base map itself, though, we needed little else in the way of data to accomplish our purpose. We were able to take shapefiles for roads, parks, and river features in the city and do a very low key map design that fit the aesthetic of the site and contained no extraneous information. People familiar with the city could quickly look at it and see right away what the crime situation was like in their favorite areas, without any other distractions.

Stumble Safely

DC Bikes

Another example was a very similar map, but for a very different use case. DC Bikes was designed on the same platform as StumbleSafely to provide a resource to the D.C. cycling community. It showed bike thefts, bike lanes, and bike shops around the city.

For the map on that site, we again wanted to show crime data and were able to take the same basic approach to StumbleSafely for the base map design, but there was one additional feature we needed — bike lanes. These were made available as a shape file from the D.C. government as well, so we were able to quickly repurpose the map from Stumble Safely, tweak the colors to match a new design, and highlight the bike lanes in a new color so they stood out. For the cycling community, we were again able to show just the more relevant information on the map and omit any distractions.

DC Bikes

DC Nightvision

Finally, we wanted a much higher level of detail for a different project. Rather than omit details from our map in this case, we wanted to pack it full of details about public infrastructure. Not just buildings and roads, but even crosswalks, medians, and topography lines. The map we released for the public, “DC Nightvision,” includes all of these details, each of which are again published by the D.C. government as shapefiles.

DC Nightvision

With the increasing availability of shapefiles like the ones mentioned here, TileMill will make it easy for end users without a lot of GIS training to churn out custom maps that meet their unique communications needs. With the data in hand and user-friendly tools to work with it freely available, creativity will be the only limit for creating great custom maps.

August 05 2010

20:12

TileMill: Custom Maps to Help with Data Dumps, Hyper-Local

TileMill is an open source toolkit that helps you create beautiful custom maps in the cloud, built by Development Seed. We recently won a Knight News Challenge award (a.k.a. “Tilemapping”) to help us release a new version of TileMill that will make it even easier for people to design highly custom maps — using their own data or freely available public data — that they can then use anywhere online. Over the coming year, our team will be blogging on Idea Lab to share different pieces of our work and talk about our progress. In this post we want to introduce readers to what we’re up to and why.

Why TileMill

So why TileMill? There are a couple of trends happening right now that are leading civic and media organizations to want and need custom maps. One is the open data movement, which is leading to an onslaught of new data sets available for public use. As more data becomes open, access to information is no longer the barrier — you just need the tools to work with it.

There are plenty of simple-to-use and freely available tools for working with RSS or CSV files that are commonly released under open data initiatives, but this is not the case for GIS file formats. Even tech-savvy web users who run into these files on open data sites often don’t know how to use them. Just because data is freely available doesn’t mean it’s useful (yet). People also need the tools to work with the data.

Another relevant trend is the move toward “hyper-localism.” As the volume of information available to us continues to increase, one of the most certain factors to help people figure out which information is relevant to them is how much if effects their life in their local community. With everything from search engines to grocery stores touting their local relevance and credentials, there is a growing need for tools to show off highly detailed local information on maps.

People want and need to see details to make sense of local information, and large global map maintainers might not have any incentive to provide this data (classic example: poor road documentation by Google and Microsoft in Africa). Most organizations don’t have the resources to consider building custom maps to better highlight their local information.

We made TileMill to help solve these problems. If anyone can take available map data, highlight the details that matter to them, and generate their own custom maps without spending thousands of dollars, it will increase the quality of hyper-local content on websites and the value of many large public open data initiatives. Our hope is to dramatically reduce the barriers to making very custom maps online.

Making it Usable

At Development Seed, we’ve always been interested in building practical tools that help organizations nail the details surrounding their work. Over the past few years we’ve worked with international development organizations, domestic NGOs, media organizations, and government agencies who have all discovered a need for custom maps to help them better communicate the geographic details and context around their content or other key data. With so many groups wanting better maps, we started working on tools that would make map creation easier and more affordable. Last year we started work on a new suite of GIS tools at MapBox.com to provide accessible open source solutions to create custom maps, and TileMill is one of the projects that has come out of that initiative.

afghan election data.jpg

To get a sense of how TileMill works, users can bring their own GIS data or use publicly available data sets, add their own visual design styles to different map elements, and generate new maps to then load into a web browser and view online.

Instead of seeing generic publicly released maps like those from Google or OpenStreetMap, website visitors can see and browse custom maps that are designed to show off very specific geographic info or to match an organization’s branding and design aesthetic. This process has traditionally been very technical, involving a cocktail of different mapping software that can be hard to set up and that few people know how to use well. It’s also been expensive and resource intensive, often involving very large datasets that require considerable computing power to work with them well. TileMill makes this process simpler.

As for what we are up to next, we are really excited about the opportunities that will come from our Knight News Challenge award. This is key funding that will turn the current TileMill into totally revamped TileMill 2.0. Over the next few months we’ll be working to incorporate feedback and lessons we have learned from our first release into the 2.0 toolkit.

Our main focus will be on making TileMill more usable, reducing the learning curve so that users without development or mapping experience can get started. Our hope is that this work will make it possible for local bloggers, smaller NGOs, and other organizations without existing budgets for GIS teams to put very custom maps on their websites. We are ready to move fast with development. For full details on the improvements we’ll make, check out our plans in this blog post at developmentseed.org.

To see some examples of what kind of maps can be made with TileMill, check out the demos on MapBox.com and the custom maps we created for the Afghan presidential elections in 2009 in action at AfghanistanElectionData.org.

Have ideas for what would make mapping better in your world? We’d love to hear them in the comments, or on Twitter where you can follow our progress @mapbox.

January 23 2010

11:16
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!

Schweinderl