Ron, the shoemaker

Ron is the manager of a small shoe factory. And he’s quite good at it, I must say. An excellent small shoe factory manager.

Over time, Ron came to the conclusion that he could measure the factory productivity. It’s merely the number of shoes produced by a unit of time.

If you would make 10 shoes a day and now you produce 15, well, you go faster.

You’re more efficient.

More productive.

And that’s it: you are always faster if you end up with 15 shoes a day instead of 10.

Picture by Rosan Harmens

Ron can measure its system productivity by looking its outputs (i.e., the shoes) given a set of inputs (i.e., time, raw materials, people).

He knows the stuff. So he’s happy. He sleeps well at night. He gets back home every day, kisses his children goodnight, plays at Counter Strike for a bit, and watches Netflix. All good.

But Ron has a sister, Luna. And Luna isn’t that lucky.

Because…

Luna, the software builder

(Aaah! Software! Screams and shouts all over the place. Software!).

Boomer.

— Hey, Ron — says Luna — how come you’re so happy? You sleep like a baby. And I can’t even… Dunno, can’t figure it out at work. I don’t know how to measure how much work my team’s doing, so I can’t sleep. And I suck at Counter Strike.

— OK, one step at a time. Do you make shoes? No, you don’t. You don’t work in a factory. You don’t sell a physical product, do you? Then you cannot measure things the way I do.

— Yes, but…

— No buts, Luna. Mmm… Say you make movies. One output is its length, right? Think about it. Does it make any sense to evaluate your team’s performance based on the total amount of minutes?

— Yes, but…

— No buts, Luna. You could measure, for instance, how many tickets you sell. Or the average review score. Those could be easily interpreted as results, and it is quite different from measuring outputs.

— Yes, but…

— I’m 100% positive that the metrics you were working with were flawed or could be tempered easily. You said you track Story Points, right? lol.

— Of course, I do! It’s my way of knowing how much work we deliver. They help me establish our velocity. The faster, the better.

— Ok. Notice that you are now measuring how long the movie is. I mean, you’re measuring the output of the team. Lines of code, features, wireframes, story points… team outputs. I’d go the extra mile: facing the same results, the lesser outputs, the better. You’ll see…

An example

Let’s say you have this web form. It performs poorly — that is, it has a low conversion rate. You have two development teams, so you ask them to fix it up.

After a sprint, Team #1 comes to you with results. They say: “We’ve redesigned the form while introducing an analytics system so that we can track user behavior and put in place some A/B testing. We then segmented users depending on some metrics, and it worked. Conversions are now on track. The resulting code is pretty well-organized. We’ve delivered 45 story points.

So, Team #2: “Hey, we noticed the content around the form wasn’t quite clear and was poorly-written. We talked to a couple of stakeholders and agreed upon better copies. Then we logged in the CMS, changed the copies, and so far, so good. I guess we delivered 0 story points?“.

Outcomes instead of outputs

— You see that, Luna? Both teams achieved the same result, the same outcome: they boosted conversions. But each team had different velocities. Which one would you pick?

— Yeah, OK, I see… they managed to get the same result while creating less output. So they generated less stuff, too…

— Notice that the second team has an astounding velocity of 0. Is it a failure?

— Well, no, but…

— In your software world, you have this saying about simplicity, right? Let me quote it for you:

Simplicity — the art of maximizing the amount of work not done — is essential.

— OK, let me see. Are you saying a team isn’t any “better” because it goes faster because we’re not a shoe factory?

— This is it.

— …because even if we go faster, we might end up with the same outcome?

— Yep.

— …and that measuring outcome instead of output (such as velocity) is quite more interesting…

— Sure.

— …because in my world, more outputs may lead to more bugs, more complexity, more defects… more costs? Are you saying that code is a liability?

— Sure. Hard to say it is an asset, given this point of view. Right?

— …so, if I got it right: our goal is to cut the total work done (outputs), while maximizing the results?

— I couldn’t have said it better myself. Fancy a Counter Strike?