A process for improving product roadmapping using Objectives and Key Results – for product managers

Today we are talking about roadmaps. Some product people love roadmaps, while a lot hate them. What can make them better? Our guest has had good experience creating roadmaps from objectives and key results (OKRs), and he is going to tell us how.

That guest is Michael Harrison. He is the Head of Product Management for Fleetio, a SaaS company that automates fleet operations to keep vehicles and equipment running smoothly.

Summary of some concepts discussed for product managers
[2:11] What are approaches to product roadmaps have you used?

As Fleetio has scaled, the needs of our roadmap have changed a lot. When we were small—seven employees—we operated on a project-based roadmap, a series of features with goals and timelines for when we hope to deliver those. That worked pretty well. When we were small, it kept us focused and we could afford major changes in direction. Now, as we’ve evolved, we need more of an outcome-based roadmap because it naturally keeps us more aligned as we get bigger.


[3:58] What led you to incorporate OKRs into your roadmaps?

The first change we made was switching to more theme-based roadmaps. Having narratives for product strategy is inspiring and allows the go-to-market side of the organization align with product more easily. We found if you just select roadmap themes and put them on a yearly cadence, it’s hard to judge relative priority among themes and choose a cap for how many themes you should have. If you’re a customer-led organization, almost every idea is a pretty good idea. The question is, what are the best ideas? For the last two quarters, we’ve been doing OKR roadmaps, and it forces us to make sure everything we’re building ties up directly to our company’s objective. It forces us to think about relative priority because we can only choose a couple of things to ladder up to the top company objectives.

You  need narratives, but you also need some cap on how those narratives tie to your strategy and how to measure which ones are making a difference.

[6:46] Have you found times when the strategic objective wasn’t the most helpful thing to guide OKRs?

Yes, how you are going to pick the right things is the principal challenge of roadmaps. Not all your ideas are going to work. Think of the product team more like the marketing half of your organization—half of your ad dollars will be wasted; you just don’t know which half yet. The question is which things will have the most impact and can you reduce the impact of the things you’re going to get wrong? The OKR structure has given us a way to measure the impact of things. We don’t just have a flurry of activity each quarter. We have a flurry of activity in specific categories that are measurable.

[8:16] What is your process for roadmapping with OKRs?

I think of our process as a four-layer ladder. We tie everything the company is doing up to five unchanging OKRs at the top level, for example our annually recurring revenue (ARR) is our North Star metric. We also have metrics for retention and expansion. That’s level one—company strategy.

Underneath that in level two, we think about what the product needs to do over the next year. What do we need to get better at? What kind of mission do we need to be on for the product to drive revenue, retention, and expansion?

Level three is what can we do this quarter? That’s where time bounds are very important because we’re planning where we want to be by the end of the quarter. That’s our outcome-based roadmap. What does the product need to achieve for users? What leading indicators can we measure that would tell us we’re driving the level-two product mission.

Level four is our roadmap. Those are the ideas, experiments, and features. These are loosely held ideas, hypotheses that can drive our convictions that ladder up to the top.

[16:05] What other benefits have you seen with creating the OKR project?

A lot of product leaders listening can probably empathize with the idea that we lose sleep over the opportunity cost of our decisions. When we get bigger and have bigger teams, it is such a frustrating feeling if you don’t feel in control or don’t know the main direction the organization is rowing in. Having this OKR structure gives me a way to feel like our activity is rowing in a direction I believe in and can trust. It’s not a series of disconnected teams I have to check in with to make sure they’re on course. It helps me sleep better at night knowing our activity is going in a consistent direction.

It helps me coach product managers, designers, and engineers on the autonomy of what they’re building. It gives us a way to know we’re only a couple of weeks away at most from measuring what we’re doing, and we can’t dig a hole too deep before we realize we weren’t supposed to go in that direction. It’s a safety net that allows us to test our hypotheses.

I’m in less emotional conversations about ideas. People feel like they have their beloved ideas that they really want to execute, but we can analyze an idea with a little bit more of a systematic approach because we know we’ll have a way to measure it, and our process is more important than our outcome.

[18:42] What metrics do you use?

We separate leading metrics and lagging metrics at the levels of the OKR structure. We have our North Star metrics like net revenue retention. Then at level two in our product mission, we almost always use lagging metrics. At level three, when we start getting into quarterly outcome, we need a leading metric that we are hypothesizing will move our lagging metric over time. At level three, you should be able to measure your metric every day and have it be meaningful.

When we choose a metric, we try to play out the game. I call them pre-mortems. We write what would happen if we hit this metric or what would happen if we didn’t hit this metric. If it just leads to more questions, we’ve probably chosen a poor metric.

[22:00] Tell us more about your pre-mortem.

We have a written product culture. Something I believe in is that you build your convictions through writing, and I want product managers to be compelling persuasive writers. One of the template items in our product briefs that our product managers write is, What are the key beliefs? What are you operating within? What is your hypothesis? One of the reasons we write that is so we can be systematic to say there are ways this could fail. There are things we might be overlooking. There is a belief we’re operating within, and we’re not 100% certain this will go right.

I try to probe teams to explore what could go wrong. What are the scenarios in which you would know you’re on the wrong path? It’s extremely important to know those things ahead of time because once you’re in the middle of executing your idea, you have a subjective bias because you’ve put sunk cost and emotional cost into what you’re building. We try to think of those things long before we’re building.

[24:30] What challenges are you still encountering with product roadmaps?

Two things happen when you’re very metrics-based: You pick bad metrics sometimes and you may have a tendency to make fewer major swings, to be less ambitious as a product team. If you’re outcome-based, you want as many green metrics as possible, so you try to make something more achievable. The I’m trying to push against that is using one of my favorite principles from game theory: What do you win when you win? I want design metrics so that if we actually hit the metric, we’re thrilled. Sometimes that would be a metric we aren’t likely to hit, but if you have a 10% chance of hitting a 50x outcome, that’s a bet you should take. Something I’d love to see us get better at as we design metrics is sometimes we need to take really ambitious swings because the payoff is worth the effort.

Action Guide: Put the information Michael shared into action now. Click here to download the Action Guide.

Useful links:

• Learn more about Fleetio
  
• Connect with Michael on LinkedIn
  

Innovation Quote

“As our island of knowledge grows, so does the shore of our ignorance.” – John Wheeler  

Thanks!

Thank you for taking the journey to product mastery and learning with me from the successes and failures of product innovators, managers, and developers. If you enjoyed the discussion, help out a fellow product manager by sharing it using the social media buttons you see below.