Let's have a little talk about tweetle beetles. What do you know about tweetle beetles? Well...
Dr. Seuss, Fox In Socks
Someone clever once said engineering is doing for a dime what any fool can do for a dollar. That speaks to the over-constrained nature of almost all business ventures. Another way to state this is that we generally are struggling even to come close to building a desired good for a cost that can barely make the business objective viable. Even that is optimistic, since it implies we really did deliver the good at some known cost – when the likely reality is that the actual cost is a gift that will keep on giving. I’m referring, of course, to my favorite subject – technical debt. It is the responsibility of engineers to communicate cost and risk as effectively as possible so that stakeholders can make reasonable decisions. We project cost all the time – sometimes we even do it well. But tech debt is different because we are trying to project the cost of something we didn’t do. There is a hypothetical aspect to the cost of tech debt that makes our attempt to communicate it sound, well… lame. We come across as if we were saying “we have to use a titanium encased lead-core hammer because someday we might run into a nail we won’t be able to drive properly”. Statements like this very properly have zero credibility and it’s not enough to try to figure out how to dress them up with the mere appearance of credibility. So how are we to do our job and give stakeholders what they need to do their job? For this discussion I need to introduce two unsavory acquaintances of mine…
Being Hittites (a dead civilization), their insalubrious nature won’t hurt anyone’s feelings. So, say hello to my two Hittite friends, Sven and Ole. Now, Sven and Ole deal in debt instruments, to put the best possible face on it. These are the kind of debt instruments where collection enforcement doesn’t so much involve lawyers as brass knuckles. We’ll come back to them shortly. Now let’s meet some respectable accountant types.
Accountants have given us a tool for thinking about the present value of a stream of future cash flows called net present value. You can play around with the NPV function in Excel if you like – because that’s your idea of a good time, right? You’ve encountered the idea if you’ve ever been given a choice between accepting a lump sum payment now, or a stream of payments over time. And of course, our ability to weigh that choice depends entirely on our ability to predict what value we could extract from that lump sum if we had it now. I know – it seems like we’re a long way from tweetle beetles but stay with me here. Of course, the cash flows can be negative as well as positive. So, we can also think about the tradeoff between paying a lump sum cost now, or paying it as a series of negative cashflows over time. This is what we are doing every time we decide to defer quality, hence the term “technical debt”. So far, I have not said anything about whether this can be a legitimate trade. I happen to think it can be. But I also happen to think we rarely understand the cost well enough for it to be a responsible trade. We tend to be hyper-aware of the risk of gold-plating. I really wish gold-plating was the ditch we mostly fell into. It’s so much easier to deal with. Rust-plating is far more insidious.
To see this let’s look at the cash-flows that are in play with tech debt. On the negative side of the balance sheet we have:
- NPV of increases in operational maintenance costs attributed to the thing you didn’t do (e.g. – automation)
- NPV of increases in cost of future enhancements because you made the software harder to change
- NPV of opportunity costs associated with delays implementing future enhancements because you made the software harder to change
On the positive side of the balance sheet we have:
- Lump-sum cost savings up front
- NPV of all gains attributed to increased brand value due to earlier market impact
- NPV of accelerated revenues due to earlier time to market
Can we put a dollar sign on that? We should be so lucky. It should be pretty clear by now that these are not easy things to attribute concretely. So are we any further for’arder? Yes. This shows us the result has the potential to go either positive or negative under the right circumstances. We will be coming back to that. So – on one level it’s just math. But honestly, if it were that easy, we wouldn’t be talking about it because those respectable accountants eat this stuff for lunch… if only we could give them the incremental cash flows. A really dedicated enterprise could start tracking actuals and try to correlate them with various decisions – and that would be fascinating and useful. But this is just not a weapon that exists in the arsenal for most of us. As much as I dearly want engineers to be able to quantify the cost of quality, I would also be remiss if I didn’t point out that business analysts don’t exactly cover themselves in glory with their ability to project upside gains. We live in a soup of uncertainty both ways. So, what to do? Let’s dig a little deeper.
The kind of debt I suspect competent accountants prefer comes in a sharp three-piece suit with well-defined payment schedules, defined terms of default, collateral, and legal recourse if things go south. The vast majority of technical debt, however, is brokered by our friends Sven and Ole. This is the kind of debt that no business would dream of incurring if it were financial debt. It comes in a greasy cheap pin-striped suit with dandruff on the collar, a couple of heavies packing brass knuckles (if you’re lucky), and rather ominous terms regarding the consequences of default. Now, the ability to incur debt is a kind of asset. Er…ok that’s a lie. But… If you have good credit, you can incur certain amounts of debt, and that can be a valuable business tool. The same is true of technical debt. A well-controlled dose of technical debt could be like water in a dry and thirsty land. The key phrase is “well-controlled” and it implies you have good credit standing. If you don’t have good credit, you end up dealing with our friends Sven and Ole. And just to clarify in case there is any doubt… in the absence of hard evidence to the contrary you can just assume your credit stinks.
Is there a qualitative way to understand what good credit looks like? Broadly – I would suggest it looks like a confirmed history of managing technical debt as evidenced by observable phenomena. What are these observable phenomena? One would be that the incremental cost of change you experience on your projects is similar to the cost you experience in a green-field code base (or at least not painfully and manifestly different). Put another way, the incremental cost of change does not increase over time due to code rot. While we’re here, this backs us into a pretty good definition of legacy code – code that has become more costly to change than the business model can afford. A common Agile formulation is “code that is unsafe to change”, but I want to make it a bit pointier to suit this discussion. Notice that there is no “age” component here. If your green-field code is more expensive to change than the business model can afford… congratulations on writing green-field legacy code.
What else? I would expect the engineering organization to have high credibility – because they are delivering the goods… repeatedly over time. I would expect relatively high job satisfaction (inversely proportional to the number of dead stares you get from everyone responsible for squeezing the next increment of revenue out of the tech stack). I would expect a business that is financially successful – not because I think low tech debt guarantees this, but because I think low tech debt requires heavy investment. If you have it, it wasn’t free. I would expect a steady but moderate rate of technology refresh as manifest by not being on the bleeding edge but also not constantly plagued by being hopelessly behind in the ability to leverage innovation. All of these things are contingent on the ability to change and the ability to change is what tech debt corrodes. If your enterprise has these attributes, you totally don’t care about this discussion because you are living the dream. But if it doesn’t, I think it’s extremely unlikely that you can claim low tech debt. And if you don’t have low tech debt… what does the future hold? What is the zero point at which tech debt brings the enterprise to its knees?
We’ve just spent time walking over the landscape and things don’t look good. We haven’t yet come up with a quantitative way to assess tech debt, and from a qualitative assessment most enterprises would struggle to make any claim of low tech debt. This raises two questions to me.
- Is there no way to say anything quantitative about tech debt?
- If tech debt is as corrosive as we think, and few if any enterprises are managing it intentionally, how can any company thrive? And. They. Do. Thrive. Wassup wid dat?
But you said there would be tweetle beetles. Um… you will just have to wait for Part II