This is how a metrics program has changed how The Virginian Pilot thinks about what they cover

By David Arkin, for Local Media Association

We caught up with Jeff Reece, senior editor at the Virginian Pilot to talk about their experience with integrating Metrics for News in their newsroom.

The Metrics for News program helps newsrooms understand what types of journalism users engage with and why and gives a blended metric that helps track audience engagement in a way that matches a news organization’s strategy.

Here’s our interview with Reece:

First, tell us a little about how the Metrics for News project works from the set up to running it on a daily basis?

At first, a handful of editors did all of the tagging manually. We have since automated most of the tagging. The parts we couldn’t automate we now tag on the front end through keywords. Auto tagging has taken a lot of the burden off of the few editors who were doing all of the work, but it has its drawbacks. The tagging is now inconsistent and there are certain areas we’re no longer tracking (like emotional impact).

I would say it would be a mistake to make decisions based on single story data points. Metrics for News is best used as data sets get large enough for you to see patterns. For instance, we tested how well “tips” stories would do, things like you’d see in the New York Times’ Smarter Living section. We did several stories and discovered that if it is something people can find on Google, they aren’t going to come to the Pilot for it. The only exception is if the tips have a strong local connection. Like when an outlet mall opened, we did a “how to shop at an outlet mall” tips sheet, and that did very well. However, when Toys R Us folded, we did “other places to shop for toys” list, and it bombed. By doing several of these types of stories we were able to see a pattern that influenced how we approach these kinds of stories now.

What is your role with the program?

I am the main editor trying to tie the data to coverage decisions at the moment. Before the sale of the paper, we were in the process of teaching reporters and editors how to use the data for themselves.

Why did The Pilot decide to get involved with it?

Early in 2015 when Erica Smith and I first arrived, there wasn’t much of an online strategy at the Pilot. They had been tracking page views and doing some social media work, though it wasn’t very sophisticated. Reporters and editors were still print-centric and the online team used what I’d call legacy news judgment when it came to deciding what to highlight on the homepage — anything they thought of as click bait, like crime briefs or stories with lots of drama.

In early 2017, we decided to pursue a strategy of building our online subscriber base. What we needed most to do this was data about how our stories engage with the local audience. We decided that page views alone wasn’t a sufficient measure. We wanted to know how engaged readers were. We wanted to add reading time, social shares, and social engagement. Metrics for News gave us a way to create an algorithm that brought all those elements into one number. It seemed like an elegant way to track how we were doing.

What are some of the biggest takeaways you all have gotten from the program?

This question could be answered a couple ways. One is the big picture “how to use data in the newsroom.” The other is “what decisions have we made based on Metrics for News.”

So the first example you may have heard already because the Table Stakes folks like to reference it a lot.

We have a huge military presence in this area of Virginia. The Navy employees probably somewhere in the neighborhood of 125,000 people in one way or the other in Hampton Roads, and probably twice that number has some kind of direct link to the Navy. We thought concentrating on the military would be the first thing we tackled in building an audience-focused team. But when we started looking at the numbers, the military didn’t really do that well.

The greatest area of opportunity we found – the lowest hanging fruit, so to speak – turned out to be consumer news. Not as in how to stretch a dollar, but as in where are the coolest places consumers can go to spend their money. Store and restaurant openings and closings, grocery store chains, major residential and entertainment developments, microbreweries – people really want to read about the places they can go to spend money. We created a “consumer” team, and we changed the focus of our food writing position from preparing food to covering the area’s restaurant and bar scene. The results have been phenomenal. Our engagement, page views, and time on site are all up. And consumer news has now eclipsed public safety as our No. 1 topic in reader engagement.

We learned other things as well.

  • Crime briefs don’t engage readers all that well. We have cut way back on those.
  • People don’t really care about routine council or board meetings. We no longer write meeting stories unless actual news comes out of them.
  • Watchdog news is highly engaging, and we have always done a good job in that area, but we have also tried to move away from routine daily coverage to more enterprise work.

Are you using other analytic software and what makes this different than the Google Analytics and Parse.ly’s of the world?

We use Google Analytics and Chartbeat. What Metrics for News does that Google Analytics can’t is give us the one simple number we can track. I think it will make it easier for reporters to grasp what works and what doesn’t. GA just gives you raw numbers, which mean nothing without context, and Chartbeat is nice for watching trends during the day, but doesn’t really help much in creating an overall strategy.

As you continue to evolve with Metrics for News what are some things on the horizon you want to do with it or that you think it will help you do?

We still have a ways to go in getting the whole newsroom using the data to help them make decisions about coverage. We also need to move on to the next topic we want to tackle. For instance, when it comes to military coverage, is the problem that there isn’t that much local interest in it, or there isn’t much interest in how we cover it? I think that is ripe for experimentation.

We aren’t anywhere close to where we’d like to be. But we have seen the fruits of our work from last year in the past four months. Our engagement continues to rise steadily and I feel like we were on the right track. In some ways I feel like we’ve barely scratched the surface of what’s possible with this, and yet we’ve already seen some great success.