Outcome Vectors OKR Consultancy Logo

Author: Robin Hyman

  • Goal Setting – Careful What You Measure

    Goal Setting – Careful What You Measure

    Why Goals and Measures Matter

    Given the power of goals to direct and focus our attention and motivation, it’s probably a good thing to invest at least some effort into being deliberate about how we choose them, be they personal or business, micro or macro. 

    In enterprise settings where incentives are involved, people will bring their own perspectives on what matters most, consciously or otherwise. The challenge in that context, beyond inspiring action, is to avoid conflicting priorities, and that is easier said than done.

    Goal-setting frameworks like OKR are useful in helping to structure, communicate and track goals, but they don’t on their own solve the problem of choosing the right objectives and the right measures. Thoughtfulness and care are necessary if you want to maximise the chances of driving the right behaviours and avoid some of the pitfalls around measurement. Frameworks help, but they can’t do the thinking for you.

    Used carelessly, goals and incentives have strong potential for unintended consequences, which can be very expensive and even take you completely in the wrong direction.

    When Goals Go Wrong

    Take the story of the Great Hanoi Rat Massacre. In 1902, French colonial administrators in Vietnam had built sewers under the part of Hanoi where they lived, and unfortunately these sewers became a perfect breeding ground for rats. This concerned them enormously because there was an outbreak of bubonic plague in China. This was at a time when scientists had recently linked the spread of plagues to rodents, so they feared it being carried by rats on ships and trains that came from there.

    Their solution was to enlist the locals to help them kill rats. They offered a bounty of 1 cent per rat tail.

    • Goal: Prevent the spread of bubonic plague by reducing the rat population of Hanoi.
    • Success measure: Number of rat tails. 
    • Outcome: Oh. More rats. Damn!

    So what went wrong? It seemed to go well at first; plenty of rat tails and bounties. But gradually, officials started noticing more and more live rats without tails. Worse than that, rat farms sprang up around Hanoi. The enterprising locals had gamed the system. The system, in fact, had asked for it by creating a rat economy.

    The well-intentioned incentive had put the organisational goal at odds with those of the people doing the work. Success meant fewer future income opportunities, making rat tails a very bad proxy for reduction in the rat population.

    The administrators understandably cancelled the rat tail bounty, a move unpopular with the locals, as I’m sure you can imagine. Which, incidentally, brings us to one reason to be careful working with incentives. Giving something and then taking it away is worse than not having given it in the first place. People have memories, so once you introduce an incentive, you change the system. Not getting it right first time can make it harder to get it right next time, because now you may have introduced resentment and resistance. For this reason, it is better to start small with a trial if you can, to limit the blast radius if things go wrong.

    Fixing Metrics vs Fixing Intent

    Anyway, it’s easy with hindsight to see that rat tails were a bad proxy measure, but that’s another feature of complex systems – causes often seem obvious in retrospect, but can be difficult to predict looking forward.

    What could our unfortunate French administrators have done instead? What lessons can be learned from stories like these to help us avoid similar mistakes?

    How about a bounty of whole rat bodies instead of rat tails? Well, that might remove the incentive to free rats to breed after chopping off their tails, but it wouldn’t disincentivise the rat farming operations. A partial solution at best, but still open to abuse. Adjusting the metric without revisiting the intent simply moves the problem elsewhere.

    So, let’s take a step back and consider the ultimate intent behind the initiative – to prevent or minimise bubonic plague infections. The main success measure, the lagging indicator they really care about, is the infection rate. Framing success in those terms doesn’t pre-suppose a solution. It leaves open the possibility for initiatives other than killing rats in the sewers of Hanoi. It opens up the solution space to allow us to consider other dimensions like prevention. Infection-carrying rats arriving on ships were known to be a key vector of infections, so they could look at measures to control the entry points, stricter port controls, mandatory periods at anchor before docking, inspections before unloading, and all sorts of other initiatives. 

    The OKR framework wasn’t a thing in those days, but if it had been, the administrators could have used it to approach the problem with a balanced set of leading and lagging indicators, working in short cycles, and closely monitoring for impact to catch unintended consequences early.

    Let’s try to write an OKR for our French colonial administrators (in reality, an endeavour like this would use a set of related OKRs focused on different aspects of the problem, but I want to keep this simple just to illustrate the point).

    An OKR could have looked something like this (don’t get hung up on the numbers. I’m just guessing here. Really they would be whatever was ambitious but doable in the time period covered by the OKR):

    Objective: Prevent the introduction and spread of bubonic plague in the city

    KR 1: Increase the percentage of arriving ships held at anchor at least 500 feet offshore and inspected before docking from 0 to 80%
    KR 2: Increase the percentage of arriving ships to have implemented rat prevention measures before docking from 10% to 90%
    KR 3: Increase the percentage of sewer access points in port adjacent districts that are sealed or trapped from 0% to 75%
    KR 4: Increase percentage of new infections reported within 7 days from 20% to 80%
    KR 5: Decrease 4-week rolling window infection rates near ports from 60 to 20

    Hopefully, you can see from this quick example that being clear about the core intent in the objective statement instead of jumping straight to a solution opens things out. I’ve been a bit lazy here because ideally an objective should be focused on one thing, whereas this one chases two. We could consider splitting this into two prevention focused OKRs, one each for introduction and spread. Still, even with this purist-offending, slightly lazy OKR, notice how we have some key results that are leading indicators that readily suggest initiatives, one that improves data-gathering and another lagging indicator that measures for overall impact. 

    Success for the objective is defined as meaningful progress in all these dimensions, not just one single metric. This is a core strength of the OKR framework – it encourages a holistic approach to defining what must be true for an objective to be achieved. The requirement  to come up with multiple key results and balance them out makes you think harder.

    It’s not a silver bullet and you can still get things wrong, but the rigour involved improves your chances. Importantly, defining success at the objective level rather than focusing on a single metric reduces the chances of driving unintended behaviours.

    The Corrupting Impact of Measures

    The potential for unintended consequences driven by measures has been observed by many and well studied. Most often cited are Goodhart’s law and Campbell’s law.

    Goodhart’s law states:

    When a measure becomes a target, it ceases to be a good measure

    Campbell’s law states:

    The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

    Both apply to our rat-tail example, but Campbell’s is probably the best fit because it captures the corrupting effect. Of course, Goodhart’s is true too – number of rat tails did indeed cease to be a good measure, if it ever was one, but in this case the attachment of an incentive led to deliberate corrupt behaviour and an unintended outcome.

    We see this effect in education when a focus on test results leads to the behaviour of teaching to the test. We see it in software engineering when a focus on lines of code leads to a bloated and unmaintainable code-base. Once measures are introduced, behaviour adapts (not always deliberately or consciously) and this effect is magnified when you throw incentives or penalties into the mix. It’s human nature.

    Measurement is more than just observation, it’s intervention too. Once you introduce a measure, you change the system, especially if you attach incentives. If you’re not careful, you can invite gaming and leave yourself wide open to unintended outcomes. Balance your metrics, monitor closely and favour short cycles. And remember, goal-setting frameworks are more forcing function than silver bullet, serving to help you maintain discipline around how you do goal setting. If anything, they should make you think more, not less.

  • OKR – More Wood Behind Fewer Arrows

    OKR – More Wood Behind Fewer Arrows

    Priority for All is Priority for None

    Easily the most recurrent theme I encounter in my work with clients is that their people feel that there is too much going on at once. They report feeling overburdened and confused by the sheer volume of initiatives. Everyone is busy but little seems to be getting finished and that’s a situation that creates stress, contention for resources, conflict and burnout. 

    More work going on at once means longer queues and more complexity. Shoving more work into a system where demand already exceeds capacity can only further slow everything down. That might seem like a statement of the obvious, but still we see exactly this type of self-inflicted gridlock over and over.

    It’s an easy trap to fall into, even with the knowledge that multitasking dilutes or even destroys focus. It can be difficult to resist the pressure to start immediately and too easy to fall for the seductive illusion of speed this creates. Most of us instinctively know that it would be better to say “not yet” and wait until existing commitments have been delivered, yet we can be our own worst enemies sometimes.

    “It can be difficult to resist the pressure to start immediately and too easy to fall for the seductive illusion of speed this creates.”

    When you have stakeholders to satisfy, finite political capital, commercial pressures and more, saying “yes” can feel like the easiest way to get through the day, even when you know that you and/or your teams are already stretched.

    Prioritisation as a Strategic Discipline

    This is where OKRs can help. They are a goal-setting framework used to translate strategy into focused and measurable action. The framework encourages deliberate focus on fewer things; ruthless prioritisation based on business value. The framework is based on the belief that tightly focused attention and energy is more impactful than diffuse, scattered activity.

    Google co-founder Larry Page’s mantra captures the idea succinctly; “more wood behind fewer arrows”.

    OKR stands for Objectives and Key Results. Objectives state what we want to achieve, and key results state how we will measure progress. The enabling constraint that helps with focus is to limit both the number of objectives and the measures that collectively define success.

    Done well, OKRs are a simple way to concentrate the attention and resources of the business on what matters most right now, and to get fast feedback on what’s working and what isn’t. In practice, they operate in cycles of create, track, review and adapt, typically yearly at company level and quarterly for departments and teams.

    The leadership team shares their top-level strategic OKRs and then the departments and teams use those collaboratively to derive their own contributing OKRs. Initiatives come last. They are subservient to the key results and exist only to drive them. Initiatives that don’t drive any key results should be deprioritised. The net result should be a set of initiatives that collectively form a strategically coherent and transparent programme of work that provides a clear view of how it connects to the bigger picture and gets everybody pulling in the same direction.

    The idea is to keep strategy central to the day-to-day decision-making, leveraging the motivating and uniting power of a clear sense of shared purpose.

    That’s the idea, anyway! You’ll notice I said ‘done well’. There are plenty of ways to go wrong with OKRs; vague strategy, too many objectives, weak leadership examples, or poor discipline around tracking, to name just a few.

    Crucially, discipline around prioritisation starts from the top. The OKR framework is about more than just how the objectives are communicated and formatted. It’s about behaviour change, and leaders need to lead by example. The framework won’t help much if leaders still insist on umpteen top-level priorities. If a culture of “we want it all and we want it all now” dominates, then simply throwing OKR theatre into the mix will likely have zero or even negative impact.

    Slowing Down to Speed Up

    It can be difficult for some, especially those under pressure, to accept that striking a better balance between multitasking and focused work actually means getting more done. Slowing down to speed up can seem paradoxical, but it’s not a new idea; less haste, more speed; slow is smooth, smooth is fast. Still, it often seems to get ignored or forgotten in business contexts.

    Jeff Bezos learned this when an employee told him “Jeff, you have enough ideas to destroy Amazon”. It made him realise that constantly bombarding the team with new initiatives was causing distractions and slowing things down. “I started prioritizing ideas better, keeping lists of them, keeping them to myself until the organisation was ready”. 

    When we talk about doing less concurrently, we mean getting more done over time, not less. We are advocating for increasing system throughput, not reducing effort. We mean enabling focus to improve speed, quality, impact & sustainability. And again, it has to start from the top.

    Put simply, if you have many initiatives going on at once, and progress is painfully slow, pausing several and focusing only on the few most important will often result in more getting finished. Then, as capacity frees, you can pull paused work from the queue. It’s how you get everything sooner, prioritised by value. OKRs can facilitate this behaviour by introducing rigour around defining the desired outcomes and success measures, and encouraging shorter cycles.

    It’s not easy. Changing habits takes effort. Prioritisation is hard and it can sometimes take difficult conversations. These things can be tackled with or without OKRs, but the framework provides a tried and tested method that has benefitted many very successful businesses. 

    So, whilst it might seem contradictory or self-defeating to add a change initiative like adopting OKRs into an organisation that already feels maxed out, the truth is that it’s more like the proverbial lumberjack stopping to sharpen his saw; yes, progress slows briefly, but over the course of a day, far more gets done.