Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

April 21 2011


5 Great Media Literacy Programs and How to Assess Their Impact

Increasingly, Public Media 2.0 projects are moving not only beyond broadcast to social and mobile platforms, but into the realms of digital and media literacy training. Producers of such projects recognize that in order to participate fully in the new media world, children and adults need to be able to access, analyze, evaluate and communicate messages in a wide variety of forms.

Over the past two months, on the Center for Social Media's Public Media 2.0 Showcase, we profiled a series of such initiatives, examining in particular how project leaders evaluate their impact.

While there has been some controversy over semantics, for the purposes of this series, we used the term "digital and media literacy," which encompasses the foundations of traditional media literacy while emphasizing the importance of access to and informed use of digital tools. These types of programs help people to create their own media messages, participate in cross-platform civic dialogue, recognize and evaluate the messages implicit in media, assess the credibility of news and information sources, and understand the risks and responsibilities associated with social media and media production.

Strong, national support for digital and media literacy initiatives is currently lacking -- both in the public broadcasting and educational sectors. However, innovative programs are popping up across the country, sometimes in unexpected locations.

Snapshots from the Field

Our series examined initiatives from diverse sources, including public broadcasting stations, non-profit organizations, museums, schools and federal agencies, all designed to help users become fully engaged media consumers and producers. Each of the initiatives had a different focus (building students' journalism skills, recognizing hidden advertisements, bringing public media to underserved communities, etc.) They took place in person and online, in school and community-based settings, and in both kid- and adult-focused arenas. Five of the most interesting projects included:

  • The PBS NewsHour Student Reporting Labs: This program, which recently completed a successful pilot year, pairs high schools with public media professionals in order to create investigative video reports. The program combines digital and media literacy, media production, news and current events and journalism education and includes a flexible curriculum developed by Temple University's Media Education Lab.
  • Admongo: Last year, the Federal Trade Commission launched Admongo, an online gaming initiative aimed at helping 8- to 12-year-olds "become more discerning consumers of information." The centerpiece of the Admongo campaign is a single-player online game in which users navigate everyday settings, searching for hidden advertisements. The project includes an accompanying curriculum, developed by Scholastic. While Admongo provides a fun new way to look at advertising in the classroom, it is lacking in meaningful engagement, as it doesn't encourage students to critique or analyze advertisements so much as recognize them.
  • Common Sense Media: Common Sense Media recently released a new K-12 curriculum focused on digital citizenship. According to the Common Sense Media website, this curriculum aims to "teach students to be responsible, respectful, and safe digital citizens." The curriculum focuses primarily on digital ethics and responsibilities, using engaging classroom activities to tackle issues like privacy, cyber-bullying, online identities, and copyright/fair use.
  • City Voices, City Visions: City Voices, City Visions, a program from the Graduate School of Education at the University at Buffalo, provides summer professional development institutes for middle and high school teachers. These sessions educate teachers on how to incorporate digital video into their classrooms in both interdisciplinary and subject-specific settings. Teachers use handheld digital videocameras and basic editing software to turn academic concepts into familiar video formats and work with the City Voices, City Visions team to create appropriate classroom assignments, evaluation rubrics, and sample videos.

At the Center for Social Media, we are using our examinations of how these projects are assessing themselves to inform the evaluation of a project the Center has been incubating: the Public Media Corps (PMC), a public media and community engagement initiative from the National Black Programming Consortium. A service corps model, the PMC aims to increase both broadband adoption and public media creation/use in underserved communities. Last year, 15 fellows worked with Washington, DC, community organizations and public media stations to create a series of engagement models, which combined media production, media access and civic engagement. CSM will be releasing a report on the results in May.

Evaluating Media Literacy Projects

DigitalandMediaLit2.jpgAs with public media engagement projects, digital and media literacy initiatives face a challenge when it comes to evaluating success. There are currently no standard tools for assessing baseline digital and media literacy skills -- although in her white paper, Digital and Media Literacy: A Plan of Action, Dr. Renee Hobbs strongly advocates for their development. She notes that "there are so many dimensions of media and digital literacy that it will take many years to develop truly comprehensive measures that support the needs of students, educators, policymakers and other stakeholders."

Because the initiatives we looked at varied so much in scope and size, each took a slightly different approach toward evaluating programmatic success. Not every organization we profiled implemented a comprehensive evaluation plan. However, many of them did, and some key themes emerged:

1. Set clear and ambitious goals, and assess against them: It is important that digital and media literacy initiatives move beyond "raising awareness" and move instead toward
empowering users to make their own meaningful choices, critiques and content. For example, Admongo does not go far enough in allowing users to evaluate and analyze the game's advertisements, nor does it offer users much in the way of content creation. Successful digital and media literacy initiatives must set goals beyond awareness-raising, and evaluate their success based upon clearly-defined criteria.

2. Evaluate both media literacy and media production quality: One of the major tensions in evaluating youth and community media production initiatives is the extent to which media production values should be considered. Leah Clapman, director of the PBS NewsHour Student Reporting Labs, noted that, as the program progressed, program leaders moved away from evaluating the production values of student projects and towards measuring what students have learned in the process. City Voices, City Visions is able to negotiate this tension with a multi-pronged evaluation strategy. Students are judged in class primarily by how well they convey academic concepts through video, but an annual film festival showcases high quality student productions, as determined by external judges.

3. Evaluate both teachers and students: Staffers from almost every initiative we talked to expressed that feedback from both teachers and students is necessary in order to obtain a comprehensive understanding of how well a given project worked. Both Common Sense Media and City Visions, City Voices, for example, combined student assessments, teacher interviews and case studies. Dr. Suzanne Miller, director of City Voices, City Visions, stressed that evaluating teachers beyond the confines of teacher training institutes is key, as "not enough research follows teachers out of professional development institutes and into the classroom."

4. Examine a variety of data: While most of the data collected in these projects was qualitative (a potential problem for some funders), it took many forms, including case studies, teacher and student interviews, and student pre- and post-assessments. Some of the data was collected through less traditional methods: the teachers involved in the PBS Student Reporting Labs spent a day in Washington, DC to discuss and debate the program and analyze strengths and weaknesses with external evaluators. Most of the programs hired external evaluators at least for part of the analysis, which helped to ensure depth of analysis as well as objectivity.

PMCToolkit.jpg5. Share evaluation data with the field: Many of the programs are planning on publishing evaluation data in order to inform best practices. Common Sense Media plans on sharing video case studies on its blog. The Public Media Corps published a toolkit outlining the lessons learned from the program's pilot year. This toolkit is designed for use by public media stations looking to implement similar programs but can also be employed as a general guide for community-based media programs. The Center for Social Media will also be working with PMC leaders to release a more comprehensive evaluation next month.

It is this last point -- sharing information -- that may be the most crucial for measuring the success of digital and media literacy initiatives. Developing shared best (and worst!) practices and lessons learned through smaller-scale media literacy programs will help to ensure the development of the field and the success of future programs.

Katie Donnelly is Associate Research Director at the Center for Social Media at American University where she blogs about the future of public media.

This is a summary. Visit our site for the full post ».

November 23 2010


Innovative Projects at Public Media Camp 2010

CSM logo small.jpg

The Public Media 2.0 series on MediaShift is sponsored by American University's Center for Social Media (CSM) through a grant from the Ford Foundation. Learn more about CSM's research on emerging public media trends and standards at futureofpublicmedia.net.

Ira Glass. Gwen Ifill. Big Bird.

These are some of the public faces of public media, but behind the scenes lies a nationwide army of talented, passionate individuals whose efforts often go unrecognized. This weekend, around 300 of these workers -- producers, designers, strategists, engineers, and more -- gathered at American University in Washington, D.C. for an unconference called Public Media Camp, or "pubcamp," as the cool kids call it. A few game souls took a moment in between sessions to answer my question, "What's the coolest thing you're working on right now?"

With apologies for my rudimentary video skills, I hope you'll take a minute to take in the diversity of innovative projects happening throughout the public media ecosystem:

I also have to share a contribution that came in via Twitter from Annie Shreffler, a producer at the WGBH Lab who couldn't attend PubCamp in person, but followed the weekend's events closely via Twitter and a livestream. She said the coolest thing she's working on right now is building a community of public media audiences, video journalists and aspiring filmmakers to document today's gay rights movement. This is being done in conjunction with a documentary, "Stonewall Uprising," that's airing on American Experience this spring (watch a preview of the documentary here).

How about you? What's the coolest public media project you've come across lately? And if you work in public media and I didn't get you on camera - tell us, what's the coolest thing you're working on right now?

The former editorial director of PBS.org, Amanda Hirsch is a writer, improviser and digital media consultant. Learn more at amandahirsch.com and follow Amanda on Twitter at @amanda_hirsch.

CSM logo small.jpg

The Public Media 2.0 series on MediaShift is sponsored by American University's Center for Social Media (CSM) through a grant from the Ford Foundation. Learn more about CSM's research on emerging public media trends and standards at futureofpublicmedia.net.

This is a summary. Visit our site for the full post ».

September 15 2010


Public Media Corps Takes on Broadband Divide for Minorities

If there was a reality show about the Public Media Corps (PMC), the intro might sound something like this: "Here's the true story of how 15 fellows, five public media institutions, three high schools, three community organizations, a library and a museum collaborate to bridge the broadband divide."

Secretly, I wish there was a reality show about the project because I want to see how they're making it work. Many public media projects claim "community engagement" as a priority, but few make it the centerpiece of their work. For the Public Media Corps, that's never been an issue.

The PMC is a national service program that promotes and extends broadband adoption in underserved communities. It does so by placing technology, media production, and outreach fellows in residencies at underperforming high schools, public broadcast stations, and non-profit community organizations. The PMC evolved out of the New Media Institute, which was founded by National Black Programming Consortium (NBPC) in 2006 to train media makers of color in new media technology.

Cool Spots

The NBPC launched the beta of Public Media Corps in Washington, D.C., this June. Since then, the group has worked steadily to collect information from the community to build projects based on the needs of people living in Anacostia (Ward 7 and 8) and in Columbia Heights, Mount Pleasant and Shaw (Ward 1) -- neighborhoods that are home to predominantly African American, Latino and immigrant communities.

"The fellows are working in teams and using survey tools to gather more quantitative information about the ways in which people use the Internet and social media and what issues and information sources are important to them," said Jacquie Jones, executive director of the National Black Programming Consortium.

Collecting such data can be challenging in a community where residents have limited Internet access. To encourage wider participation, the fellows created "Cool Spots."


"Cool Spots are mobile Internet assessment hubs," said Jones. "The fellows set them up at block parties, festivals, and outdoor markets and events where the public uses netbooks onsite to complete the online surveys and learn more about the PMC. The Cool Spots are also 'hot spots' with free Wi-Fi to promote broadband use."

Starting this month, the PMC will begin using the data collected at Cool Spots to select three to four projects for the fellows and community partners to collaborate on and ultimately implement in communities for the final months of the D.C. beta.

Collaborating with a Cast of Dozens

Building trust and establishing communication channels are often the first challenges of any large-scale collaboration. With so many people, organizations and communities involved, I was curious to hear just how the PMC staff, fellows and partners were tackling it.

"PMC staff is in frequent communication with partners, stakeholders and the fellows through electronic means and meetings to discuss progress, performance, resources that can be shared and potential projects for the fall," said Kay Shaw, director of Public Media Corps. "The fellows prepare detailed weekly reports and meet as a cohort every Wednesday to discuss their activities, share insights and challenges, site needs, and how to build more collaboration between partners. The staff and the Fellows look forward to these meetings because of their vibrancy and the information and ideas that are shared."

The PMC experimented with several platforms for sharing information electronically. The two that stuck were Google Docs and Dropbox, a web-based file-hosting service that uses cloud computing to enable users to store and share files. The fellows use Google Docs to collaborate on writing projects. They post their weekly reports and media to Dropbox.

Ashley Mosley, a PMC fellow, video producer and community organizer, offered some perspective on the challenges and rewards of collaborating at this level.

"Collaborating with so many community groups has offered a broader perspective, because the personalities of the wards are so different," said Mosley. "The most challenging aspect of this project is building trust within my organization. However, the challenge has been rewarding. Community members have invited me into their personal circles, hangouts, and meetings and they now feel more comfortable discussing the disconnect that they feel with both public media and digital technology."

Measuring Success

Another hurdle collaborations face is measuring success. When working with multiple stakeholders and communities, priorities have a tendency to become malleable and impact ambiguous. To keep the focus on results, the PMC has placed a high priority on collecting data throughout the six-month beta.

"While there is no dearth of projects or project ideas, how many projects can we definitively say have been successful on a large scale in diverse communities?" said Shaw. "That's why we consciously decided to do the research and collect data first and let that inform the projects we would develop to ensure the best chance of gaining traction and making a significant impact in our focus communities."


The PMC is measuring levels of community engagement, use of public media resources, technological capacity of partner organizations and broadband adoption and patterns of use within the communities.

In addition to collecting community data, the PMC will seek feedback on the performance of its fellows from a team of technologists and public media stakeholders, who will conduct site visits and one-on-one meetings with the fellows.

"They will offer suggestions or adjustments to the project to improve impact and relevancy," said Jones. "American University's Center for Social Media is taking the lead in assessing the evaluation team's observations and recommendations, analyzing the data from the surveys, evaluating and assessing the impact of the projects, culling and documenting best practices, and producing a written report that will be distributed to the public in 2011."

Lessons Learned

In advance of the official 2011 report, I asked Jones and Shaw what they have learned so far from collaborating with the various community groups, fellows and the public on a project of this scale.

"On the one hand it is exciting and rewarding to work with organizations committed to and excited about the project and ready to learn how to expand their capacity to use and leverage public media assets," said Shaw. "On the other hand, because of the interest and need there are lots of demands on the project and we are constantly adjusting to accommodate new information to ensure relevancy and impact."

"One of the greatest challenges to innovation is the need to be constantly adaptive," said Shaw. "That's why we designed a process with maximum flexibility."

Jones' advice to organizations that want to increase collaboration and community engagement is succinct: "Be prepared to abandon your assumptions and what you think are your best ideas."

As for the fellows, they offered some advice of their own for future PMC participants.

"I would advise future fellows to make sure to engage not only their community organization, but also the surrounding communities," said Mosley. "It's important to establish trust. Oh... and of course, to have fun!"

Olivia Rubagumya, a PMC fellow working with PBS Interactive suggested that it's important to "be a good listener and observer. People's realities are often more complex than we can assume them to be, so remain open and attentive no matter the challenges -- the experience is a two-way street."

A public relations and social media consultant, Katie Kemple works with public media clients to build community, develop strategic partnerships, and create integrated public relations campaigns. Over the past ten years, she has held positions at WGBH, WETA, Capital News Connection, and Public Media's EconomyStory. You can find her every Monday at 8 p.m. ET on Twitter, as a co-host and organizer for #pubmedia chat.

This is a summary. Visit our site for the full post ».

Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.

Don't be the product, buy the product!