
Formative research with influentials and early adopters. Understanding impact starts before a digital journalism initiative is launched. A key method is to engage with influentials among a targeted set of users or groups, a form of “upstream engagement” that is often used in documentary film. This research can take the form of in-depth interviews and online surveys with influentials who are asked about the types of content and tools that would be of interest and use to their respective communities and affiliated groups. Then once a site is launched, these influentials would be asked to serve as active endorsers, promoters, and early adopters of the news project. They can also serve as bloggers or citizen journalists at the site.
.
Longitudinal panel surveys of users. Once a journalism project is launched, visitors and/or registered users to the site can be randomly sampled and interviewed at regular intervals over time. These surveys would track their usage patterns, their knowledge gain; their connections and interactions with other users, their feedback on features and content of the site; and their identification and/or support for the sponsoring organization.
Analysis of “quality of participation.” Tools such as traffic counters, google analytics, and third party vendors such as Nielson Online and comScore can deliver metrics on total user size for a digital media outlet, but the quality of engagement also matters. One of the benefits of having users register at a site, is that their “social recommendation” activity can be tracked, specifically monitoring whether users are spreading information and promoting the site through forwarded stories, recommending content to news aggregators, or linking by way of their blogs or social media applications. Content analysis techniques should also be used to track the quality of user engagement and contributions, examining depth of content, level of disagreement versus consensus and level of social support. This type of tracking will add insight as to whether a digital journalism project is indeed serving as a mechanism for engaging users, facilitating discussion, and participation.
.
Tracking media and policy impact. Beyond evaluating users directly, the influence on the agenda of other media and that of community groups or policymakers are also relevant dimensions of impact. These are easier to monitor through “google news” alerts that track mentions in other news media and blogs. Services such as Technorati and Wikio also track the “embeddedness” of online content based on the number and quality of links to the site.