Managing Your Software Mortgage

Technical Debt

Every developer has at one time or another faced the situation where the client asks for a “minor change” to the stated requirements.  Often this might be wording, layout or styling.  Other times it could be a functional change that appears to be a minor tweak but has far-reaching impacts that cause your household appliances to eat your pets.

The difficulty (at least for me) arises when the client says something like

It’s just a small change, isn’t it?  It should only take you an hour at most

This always makes me uncomfortable because I want to do what the client’s asking (I like to please people) and from their point of view, they’re quite right that the change they’re asking for appears to be a very simple one.  However, I also know from experience that since they’re asking for this change during our release phase (i.e. we’ve done testing, UAT and pre-production) and it Just Can’t Wait it’s extremely risky and means that the testing I’ve done will effectively be rendered invalid and my version control will no longer reflect what’s being released into production.

So what are some good counter-arguments to clients who simply have to get that one last JavaScript animation on that text block?

Your Software Mortgage

Your Software Mortgage is the technical debt you incur whenever you have to “just get it done” at the expense of code quality or attention to process.  This might mean you don’t comment anything and just throw it together quick and dirty in time to meet your deadline or you don’t document this part for your handover & support notes.  The upshot is of course that you’ll have to go back and do it right at some point.  The cost of having to do this instead of concentrating on your next iteration is your technical debt.

It’s not a 1:1 relationship either.  To get your code back to an acceptable standard (“paying your mortgage off”), you have to forego spending time on your next set of deliverables.  This means that not only is the client paying for deliverables that can’t be started yet, they’re also paying again for work that has already been delivered.  In order to make your debt repayments, you either need to deliver the next set of requirements early (ideally without incurring more debt) or negotiate a reduction in scope for the next release.

In order to evaluate the cost of technical debt against cost of lost business opportunities, the client usually benefits from seeing the impacts in dollar terms.  This is would be easy if it was actual money being discussed, but technical debt is a more abstract concept and therefore difficult to quantify.

Quantifying Your Liquidity

Liquidity determines your ability to pay your bills

Being able to put an actual figure on your liquidity gives you more leverage to negotiate paying off your existing debt or avoid taking on new debt.

If your liquidity ratio is high (i.e. you have a high capacity to take on new debt), borrowing against your software quality will likely be less of a risk and you may feel more comfortable taking a quick and dirty approach in order to meet a deadline.  However if you have low liquidity (i.e. you’ve already taken on a lot of debt) there is considerably more risk in falling further behind on your mortgage payments.  As already mentioned, your code base must be maintained to keep a minimum standard of quality and supportability or you risk Software Decay:

[Software Decay is] …code decayed to the point in which fixing anything in a hazardous proposition – every fix is likely to break something else.

So how do you calculate your liquidity?  Accounting has a number of different approaches, but most are more complex than the abstract idea of software management allows.  However, the current ratio, which is the simplest measure and is calculated by dividing the total current assets by the total current liabilities can be loosely applied to code.

Depending on how granular you wanted to get with your reporting, you could either measure Lines of Code (LoC) or discrete functional blocks for a broader view.  Let’s assume you have exactly 100 separate methods that made up your application.  If your latest release required that 10 of these methods were rushed through and needed work, your current ratio would be 100 ÷ 10 which (obviously) equals 10; or – to put it another way – a ratio of 9:1 (i.e. for every code block needing refactored, you have 9 others that are considered optimal).

Deciding what constitutes an acceptable ratio depends a lot on your application’s complexity and size, but a general guide might be that a ratio of 3:1 (i.e. 25% of your code needs refactored) should be considered to be the maximum sustainable leverage your code base can afford and anything over 2:1 should be setting off alarm bells.

In short, keeping your unintended debt down gives you more room to intentionally take on debt when it’s useful to do so.

Risk Ratio

The later through the development / deployment process a change is submitted, the greater the risk associated with it becomes and the higher the cost to fix the consequences of that risk is.  If the client is asking for a change to be made after all testing phases are complete and the release is being deployed, it stands to reason that there is a much higher risk of introducing new problems than if the change had been done in the design phase.  This Risk Ratio is the proportional increase in the amount of work required to fix something relative to when it goes wrong.

The Risk Ratio is an extremely valuable bargaining chip that quantifies the potential cost to the client of their requested change.  It forces them to consider the financial impacts of the change from a cost/benefit perspective.  An example Risk Matrix that has risk weightings from a scale of 1 – 10 looks like this:

	Requirements Analysis	Requirements Review	Design	Build	Functional Testing	User Acceptance Testing	Deployment Testing	Release
Typographic (spelling / punctuation)	1	1	2	2	3	4	5	6
Copy (text / wording)	1	2	2	3	4	5	6	7
Cosmetic (styles / layout)	2	2	3	4	5	6	7	8
Functional (client-side)	2	3	4	5	6	7	8	9
Functional (server-side)	3	4	5	6	7	8	9	10

This matrix provides a quantifiable measure of how serious a change is based on the iteration’s progress.  You could use this as the basis of a calculation to determine the potential cost in dollar terms for any adverse impacts resulting from an unplanned change.  For example, if the client requested an existing style to be changed on your web application during your UAT phase and that style affected the layout for three pages, then you are tell the client that based on experience, there is a 60% chance that introducing a change at this point will cause a problem further down the line.  Given those odds of just under 1 in 3, there’s an extremely good chance that one of the three pages affected will suffer.

Alternatively, you can use this matrix to argue that any change being made at this point will require 6 times more effort to fix than if it had been raised during the outset of the project.  This is not an outrageous claim when you consider that the requirement might need to be formally documented, approved, developed, unit tested and acceptance tested again.

Technical Inflation

Technical Inflation could be viewed as the ground lost when the current level of technology surpasses that of the foundation of your product to the extent that it begins losing compatibility with the industry. Examples of this would be falling behind in versions of a language to the point where your code is no longer compatible with main stream compilers.

Managing your technical debt and inflation requires you to have a very clear understanding of how your debt is distributed.  Documenting candidates for refactoring is the best way to get a clear idea of the scope of work required to keep your application well-maintained.  This might be done using a source control system such as Microsoft TFS, where tasks, bugs and other work items can be recorded against a particular change or section of code; or using third-party tools such as AgileZen.  Think of this as your Application Balance Sheet where your assets and liabilities can be easily reported on.

Technical Inflation is a direct product of sustained low liquidity and software decay.  The longer you leave something before fixing it, the more expensive it becomes to fix.

Ensure you have a clear idea of how much debt your application currently has so that you can make informed and reasoned arguments against adding any more work to an otherwise hard-to-see pile.  If you need to push back on a client’s demands so that you can ensure the sustainability of the client’s application, you will need evidence of why.  Having your books in order is a critical piece.

Further Reading

Don’t Enron Your Software Project

Monetizing the Technical Debt

No Excuse for No Requirements

Dilbert on Agile

It’s scary how common this situation seems to be.

So often as a developer, I find choosing to acquiesce in the client’s insistence that their enhancement or change is vitally important and, while they’re sorry it’s such late notice, needs to go out right away.

Rather than demand that I am supplied with at least some semblance of a requirements document (thereby incurring the Outraged Stare of Daggers), I reluctantly smile pleasantly and get on with adding the feature.

Problems can snowball

It should therefore come as no surprise to me (or anyone else) when, after the fifth “last change” the client comes back for, I find myself wishing I’d saved everyone the cost, effort and frustration of having things explicitly written down from the outset.  Yet this situation is just so very common that it makes me wonder why it’s such a hard lesson to learn.

The difficulty in these sorts of situations arises typically when the change being requested is “just a small 5-minute job” or when “the colouring-in department won’t approve this feature unless we change these items”.  Each concession on a development iteration creates a precedent that becomes the expectation for all current and future iterations.  In other words, each unmanaged change that is allowed through without written requirements carries an exponentially proportional degree of risk the further through the delivery process you move.

Why is this so difficult? Well, the client (or their appointed representative) has a great deal of personal ownership in getting the feature exactly right.  If they’ve forgotten or omitted to include a detail that – while arguably insignificant – they absolutely must have, they will plead, cajole, argue and assert their case to the point where it’s often easier to simply comply than to appear pig-headedly stubborn.  The downside here is that you create – or at least contribute to – a sort of Parent / Child relationship where one party is always trying to bend the rules or circumvent established processes and the other is always resisting the temptation to concede to those demands.  It’s unhealthy for your team, it’s unhealthy for your process and it’s ultimately unhealthy for your client because that risk adds up: either as technical / design debt, increased development and testing cost, decreased auditability of design, or any combination of these.

So what’s the solution?  To always say “No” every time to teach your stakeholders a lesson?  Say “Yes” every time and make sure you document the discussion somehow?  Neither of those options are realistic or sustainable.  The best approach is to ensure you have communicated a clear and consistent process for handling this sort of situation up front before the issue actually arises.  I’m a recent convert to the Lean Software Methodology because I like the freedom it offers for requirements changes as part of a defined process.  The specifics I’ve outlined for my stakeholders work like this:

• The development team works in 3-week development cycles with an additional week for requirements review and workshops (workshops MUST include people from development, business analysis and the client)
• Changes can be made at any time so long as those changes have absolutely zero impact on delivery timeframes or test effort
• Any changes can be added to the requirements but will only be commenced once development is complete on the originally-scoped requirements being used as the basis for the current iteration
• Changes that absolutely must be included in the current iteration can be included as long as all parties accept the delivery impacts and risks

To help make the Production Line run as smoothly as possible, there are some basic rules that will help grease the wheels:

1. Always have clear lines of ownership. Know exactly what you are supposed to do and ensure others know your domain of responsibility as well as theirs.
2. Know your environment. You can’t know everything, but what you can know try to know it well.
3. Always factor in at least 10% contingency.  Try to build a culture of “under-promise, over-deliver”.
4. Credibility is king. Don’t make commitments based on uncertain knowledge; always deliver on commitments.
5. If you find yourself in a hole the first thing to do is stop digging. Old cowboy saying.

All these strategies are well and good as long as the business have defined clear and thorough requirements up front, but I can’t stress enough the importance of rule number 1, above.  Make sure your production line process has clear expectations defined for not only what your outputs are, but what your inputs need to be.  The business has an implicit obligation to provide as full and complete requirements to you sufficiently ahead of time so that you can review them, refine them and respond with accurate estimates.  If you regularly find yourself in a situation where you’re being asked to “just sneak this one little change in”, something is broken. You owe it to yourself and your broader team to get it fixed. Fast.

The Psychology of Poor Process

Empty Production Line

You would think after 18 months of forming and stabilising my development team that we’d have a pretty strong and settled process in place.  But the development team is only one player in a larger ecosystem and how that team is fed depends heavily on the requirements that they are given.