How can open source projects deliver high quality software without dedicated testers?

I recently received the following email from a WatirMelon reader Kiran, and was about to reply with my answer when instead I asked to reply via a blog post as I think it’s an interesting topic.

“I see most of the Open source projects do not have a dedicated manual QA team to perform any kind of testing. But every Organization has dedicated manual QA teams to validate their products before release, yet they fail to meet quality standards.

How does these open source projects manage to deliver stuff with great quality without manual testers? (One reason i can think of is, developers of these projects have great technical skills and commitment than developers in Organizations).

Few things I know about open source projects is that they all have Unit tests and some automated tests which they run regularly.But still I can’t imagine delivering something without manual testing…Is it possible?”

I’ll start by stating that not all organizations have dedicated manual QA teams to validate their products before release. I used the example of Facebook in my book, and I presently work in an organization where there isn’t a dedicated testing team. But generally speakingI agree that most medium to large organizations have testers of some form, whereas most open source projects do not.

I think the quality of open source comes down to two key factors which are essential to high quality software: peer reviews and automated tests.

Open source projects by their very nature need to be open to contribution from various people. This brings about great benefit, as you get diversity of input and skills, and are able to utilize a global pool of talent, but with this comes the need for a safety net to ensure quality of the software is maintained.

Open source projects typically work on a fork/pull request model where all work is done in small increments in ‘forks’ which are provided as pull requests to be merged into the main repository. Distributed version control systems allow this to happen very easily and facilitate a code review system of pull requests before they are merged into the main repository.

Whilst peer reviews are good, these aren’t a replacement for testing, and this is where open source projects need to be self-tested via automated tests. Modern continuous integration systems like CircleCI and TravisCI allow automatic testing of all new pull requests to an open source project before they are even considered to be merged.

How TravisCI Pull Requests Work

From TravisCI

If you have a look at most open source project pages you will most likely see prominent real time ‘build status’ badges to indicate the realtime quality of the software.

Bootstrap's Github Page

Bootstrap’s Github Page

Peer reviews and automated tests cover contributions and regression testing, but how does an open source project test new features?

Most open source projects test new changes in the wild through dogfooding (open source projects often exist to fill a need and open source developers are often consumers of their own products), and pre-release testing like alpha and beta distributions. For example, the Chromium project has multiple channels (canary, dev, beta, stable) where anyone can test upcoming Chromium/Chrome features before they are released to the general public (this isn’t limited to open source software: Apple does the same with OSX and iOS releases).

By using a combination of peer reviews, extensive automated regression testing, dogfooding and making pre-release candidates available I believe open source projects can release very high quality software without having dedicated testers.

If an organization would like to move away from having a dedicated, separate test team to smaller self-sustaining delivery teams responsible for quality into production (which my present organization does), they would need to follow these practices such as peer reviews and maintaining a very high level of automated test coverage. I still believe there’s a role for a tester on such a team in advocating quality, making sure that new features/changes are appropriately tested, and that the automated regression test coverage is sufficient.

Notes from the 2015 ANZTB Conference in Auckland

I was lucky enough to make my first trans-Tasman journey to Auckland last week to attend the 2015 ANZTB Conference. The conference was enjoyable and there were some memorable talks I really enjoyed (I personally like single-stream events). Here’s some favorites:

Secure by Design – Laura Bell – slides

I loved the essence of this talk which was basically (in my own words) ‘take security testing off the pedestal’. Laura shared five simple tools and techniques to make security more accessible for developers and testers alike. One key takeaway for me was to focus on getting the language right: ‘security vulnerabilities hide behind acronyms, jargon and assumptions‘. For example, most people understand the different between authentication (providing identity) and authorization (access rights), but both these terms are commonly shortened to ‘auth’ which most people use interchangeably (and confusingly). A great talk.

Innovation through Collaboration – Wil McLellan

This was probably my favorite talk of the day, as it was a well told story about building a collaborative co-working space called ‘EPIC’ for IT start-ups in Christchurch following the 2011 earth quake. The theme was how collaboration encourages innovation, and even companies in competition benefit through collaboration. My key takeaway was how designing a space you can encourage collaboration, for example, in EPIC there’s only a single kitchen for the whole building, and each tenancy doesn’t has it’s own water. So, if someone wants a drink or something to eat they need to visit a communal area. Doing this enough times means you start interacting with others in the building you wouldn’t normally do so in your day to day work.

Through a different lens – Sarah Pulis – slides

Sarah is the Head of Accessibility Services at PwC in Sydney and she shared some good background information about why accessibility is important and some of the key resources to analyse/evaluate and improve accessibility of systems. Whilst I knew most of the resources she mentioned, I thought here talk was very well put together.

Well done to the team that organized the conference.

Auckland was a beautiful city BTW, here’s a couple of pics I took:

A tale of working from trunk

Let me share with you a story about how we went from long lived feature/release branches to trunk based development, why it was really hard and whether this is something I would recommend you try.

Background

I’m familiar with three main approaches to code branching for a shared code-base:

  1. Long lived feature/release branches
  2. Short lived feature branches
  3. Trunk based development

Long lived feature/release branches

Most teams will start out using long lived feature/release branches. This is where each new project or feature branches from trunk and at a point where the branch is feature ready/stable then these changes are merged into trunk and released. The benefits of this approach is that changes are contained within a branch, so there’s little risk of non-finished changes inadvertently leaking into the main trunk, which is what is used for releases to production. The biggest downside to this approach, and why many teams move away from it, is the merging that has to happen, as each long lived feature branch needs to ultimately combine its changes with every other long lived feature branch and into trunk, and the longer the branch exists, the more it can diverge and the harder this becomes. Some people call this ‘merge hell’.

Short lived feature/release branches

Another version of feature branching is to have short lived feature branches which exist to introduce a change or feature and are merged (often automatically) into the trunk as soon as the change is reviewed and tested. This is typically done using a distributed version control system (such as GIT) and by using a pull request system. Since branches are ad-hoc/short-lived, you need a continuous integration system that supports running against all branches for this approach to work (ours doesn’t), otherwise it doesn’t work as you’d need to create a new build config every time you created a short lived feature branch.

Trunk Based Development

This is probably the simplest (and most risky) approach in that everyone works from and commits directly to trunk. This avoids the needs for merges but also means that trunk should be production ready at any point in time.

A story of moving from long lived feature/release branches to trunk based development

We have anywhere from 2-5 concurrent projects (each with a team of 8 or so developers) working off the same code base that is released to production anywhere from once to half a dozen times per week.

These project teams started out using long-lived feature/release branches specific to projects, but the teams increasingly found merging/divergence difficult – and issues would arise where a merge wasn’t done correctly, so a regression would be inadvertently released. The teams also found there would be manual effort involved in setting up our CI server to run against a new feature/release branch when it was created, and removing it when the feature/release branch was finished.

Since we don’t use a workflow based/distributed version control system, and our CI tools don’t support running against every branch, we couldn’t move to using short lived feature branches, so we decided to move to trunk-based development.

Stage One – Trunk Based Development without a Release Branch

Initially we had pure trunk based development. Everyone committed to trunk. Our CI build ran against trunk, and each build from trunk could be promoted right through to production.

Trunk Based Development without a Release Branch(1)

Almost immediately two main problems arose with our approach:

  1. Feature leakage: people would commit code that wasn’t behind a feature toggle which was inadvertently released to production. This happened a number of times no matter how many times I would tell people ‘use toggles!’.
  2. Hotfix changes using trunk: since we could only deploy from trunk, each hotfix would have to be done via trunk, and this meant the hotfix would include every change made between it and the last release (so, in the above diagram if we wanted to hotfix revision T4 and there were another three revisions, we would have to release T7 and everything else it contained). Trying to get a suitable build would often be a case of one step forward/two steps back with other unintended changes in the mix. This was very stressful for the team and often led to temporarily ‘code freezes’ whilst someone committed a hotfix into trunk and got it ready.

Stage Two – Trunk Based Development with a Release Branch

Pure trunk based development wasn’t working, so we needed some strategies to address our two biggest problems.

  1. Feature leakage: whilst this was more of a cultural/mindset change for the team learning and knowing that every commit would have to be production suitable, one great idea we did implement was TDC: test driven configuration. Since tests act as a safety net against unintended code changes (similar to double entry book-keeping), why not apply the same thinking to config? Basically we wrote unit tests against configuration settings so if a toggle was turned on without having a test that expected it to be on, it would fail the build and couldn’t be promoted to production.
  2. Hotfixing changes from trunk: whilst we wanted to develop and deploy from a constantly verified trunk, we needed a way to quickly provide a hotfix without including every other change in trunk. We decided to create a release branch, but not to release a new feature per say, but purely for production releases. A release would therefore involve deleting and recreating a release branch from trunk to avoid having any divergence. If an hotfix was needed, this could be applied directly to the release branch and the change would be merged into trunk (or the other way around), knowing that the next release would delete the release branch and start again from trunk. This alone has made the entire release process much less stressful as if a last minute change is needed for a release, or a hotfix is required, it’s now much quicker and simpler than releasing a whole new version from trunk, although that is still an option. I would say that nine out of ten of our releases are done by taking a whole new cut, whereas one or so out of ten is done via a change to the release branch.

Trunk Based Development with Release Branch(2)

Lessons Learned

It’s certainly been a ride, but I definitely feel more comfortable with our approach now we’ve ironed out a lot of the kinks.

So, the big question is whether I would recommend team/s to do trunk based development? Well, it depends.

I believe you should only consider working from trunk if:

  • you have very disciplined teams who see every-single-commit as production ready code that could be in production in a hour;
  • you have a release branch that you recreate for each release and can uses for hotfixes;
  • your teams constantly check the build monitors and don’t commit on a red build –  broken commits pile upon broken commits;
  • your teams put every new/non-complete feature/change behind a feature toggle that is toggled off by default, and tested that it is so; and
  • you have comprehensive regression test suite that can tell you immediately if any regressions have been introduced into every build.

Then, and only then, should you work all off trunk.

What have your experiences been with branching?

Microservices: a real world story

Everywhere I turn I hear people talking about microservice architectures: it definitely feels like the latest, over-hyped, fad in software development. According to Martin Fowler:

“…the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.”

[link]

But what does this mean for software testing? And how does it work in the real world?

Well, my small team is responsible for maintaining/supporting a system that was developed from scratch using a microservices architecture. I must highlight I wasn’t involved in the initial development of system but I am responsible for maintaining/expanding/keeping the system running.

The system consists of 30-40 REST microservices each with it’s own code-base, git repository, database schema and deployment mechanism. A single page web application (build in AngularJS) provides a user interface to these microservices.

Whilst there are already many microservices evangelists on board the monolith hate-train; my personal experience with this architectural style has less than pleasant for a number of reasons:

  • There is a much, much greater overhead (efficiency tax) involved in automating the integration, versioning and dependency management of so many moving parts.
  • Since each microservice has its own codebase, each microservice needs appropriate infrastructure to automatically build, version, test, deploy, run and monitor it.
  • Whilst its easy to write tests that test a particular microservice, these individual tests don’t find problems between the services or from a user experience point of view, particularly as they will often use fake service endpoints.
  • Microservices are meant to be fault tolerant as they are essentially distributed systems that are naturally erratic however since they are micro, there’s lots of them which means the overhead of testing various combinations of volatility of each microservice is too high (n factorial)
  • Monolithic applications, especially written in strongly typed/static programming languages, generally have a higher level of application/database integrity at compile time. Since microservices are independent units, this integrity can’t be verified until run time. This means more testing in later development/test environments, which I am not keen on.
  • Since a lot of problems can’t be found in testing, microservices put a huge amount of emphasis on monitoring over testing. I’d personally much rather have confidence in testing something rather than relying on constant monitoring/fixing in production. Firefighting in production by development teams isn’t sustainable and leads to impacted efficiency on future enhancements.

I can understand some of the reasoning behind breaking applications down into smaller, manageable chunks but I personally believe that microservices, like any evangelist driven approach, has taken this way too far.

I’ll finish by giving a real world metric that shows just how much overhead and maintenance is involved in maintaining our microservices architected system.

A change that would typically take us 2 hours to patch/test/deploy on our ‘monolithic’ strongly typed/static programming language system typically takes 2 days to patch/test/deploy on our microservices built system. And even then I am much less confident that the change will actually work when it gets to production.

Don’t believe the hype.

Addendum: Martin Fowler seems to have had a change of heart in his recently published ‘Microservice Premium’ article about when to use microservices:

“…my primary guideline would be don’t even consider microservices unless you have a system that’s too complex to manage as a monolith. The majority of software systems should be built as a single monolithic application. Do pay attention to good modularity within that monolith, but don’t try to separate it into separate services.”

[link]

Waiting for AJAX calls in WebDriver C#

I was trying to work out how to wait for AJAX calls to complete in C# WebDriver before continuing a test.

Whilst I believe that your UI should visually indicate that AJAX activity is occurring (such as a spinner) and in this case you should be able to wait until such an indicator changes, if you don’t have a visual indicator and you use JQuery for your AJAX calls, you can use a JavaScript call to jQuery.active to determine if there are any active AJAX requests, and wait until this value is zero.

I wrapped this into a WebDriver extension method on Driver, so you can call it like this:

Driver.FindElement(By.Id("name")).Set("Alister");
Driver.WaitForAjax();
Driver.FindElement(By.Id("next")).Click();

The actual extension method looks like this:

public static void WaitForAjax(this IWebDriver driver, int timeoutSecs = 10, bool throwException=false)
{
  for (var i = 0; i < timeoutSecs; i++)
  {
    var ajaxIsComplete = (bool)(driver as IJavaScriptExecutor).ExecuteScript("return jQuery.active == 0");
    if (ajaxIsComplete) return;
    Thread.Sleep(1000);
  }
  if (throwException)
  {
    throw new Exception("WebDriver timed out waiting for AJAX call to complete");
  }
}

I hope you find this helpful if you’re ever in the same situation.

Extensive post release testing is sign of an unhealthy testing process

Does your organization conduct extensive post-release testing in production environments?

If you do, then it shows you probably have an unhealthy testing process, and you’ve fallen into the “let’s just test it in production” trap.

If testing in non-production environments was reflective of production behaviour, there would be no need to do production testing at all. But often testing isn’t reflective of real production behaviour, so we test in production to mitigate the risk of things going wrong.

It’s also the case that often issues are found in a QA environment don’t appear in a local development environment.

But it makes much more sense to test in an environment as close to where the code was written as possible: it’s much cheaper, easier and more efficient to find and fix bugs early.

For example, say you were testing a feature and how it behaves across numerous times of day across numerous time zones. As you progress through different test environments this becomes increasingly difficult to test:

In a local development environment: you could fake the time and timezone to see how your application behaves.
In a CI or QA environment: you could change a single server time and restart your application to see how your application behaves under various time scenarios: not as easy as ‘faking’ the time locally but still fairly easy to do.
In a pre-production environment: you’ll probably have clustered web servers so you’ll be looking at changing something like 6 or 8 server times to test this feature. Plus it will effect anyone else utilizing this system.
In a production environment: you’ll need to wait until the actual time to test the feature as you won’t be able to change the server times in production.

Clearly it’s cheaper, easier and more efficient to test changing times in an environment closer to where the code was written.

You should aim to conduct as much testing as you can in earlier test environments and taper this off so by the time you can a change into production you’ll be confident that it’s been tested comprehensively. This probably requires some change to your testing process though.

Tests Performed per Environment

How to Remedy A ‘Test in Production’ Culture

As soon as you find an issue in a later environment, ask why wasn’t this found in an earlier environment? Ultimately ask: why can’t we reproduce this in a local environment?

Some Hypothetical Examples

Example One: our tests fail in CI because of JavaScript errors that don’t reproduce on a local development environment. Looking into this we realize this is because the JavaScript is minified in CI but not in a local development environment. We make a change to enable local development environments to run tests in minified mode which reproduces these issues.

Example Two: our tests failed in pre-production that didn’t fail in QA because pre-production has a regular back up of the production database whereas QA often gets very out of date. We schedule a task to periodically restore the QA database from a production snapshot to ensure the data is reflective.

Example Three: our tests failed in production that didn’t fail in pre-production as email wasn’t being sent in production and we couldn’t test it in pre-production/QA as we didn’t want to accidentally send real emails. We configure our QA environments to send emails, but only to a white-list of specified email addresses we use for testing to stop accidental emails. We can be confident that changes to emails are tested in QA.

Summary

It’s easy to fall into a trap of just testing things in production even though it’s much more difficult and risky: things often go wrong with real data, the consequences are more severe and it’s generally more difficult to comprehensively test in production as you can’t change or fake things as easily.

Instead of just accepting “we’ll test it in production”, try instead to ask, “how can we test this much earlier whilst being confident our changes are reflective of actual behaviour?”

You’ll be much less stressed, your testing will be much more efficient and effective, and you’ll have a healthier testing process.

My useless websites

We recently had a competition at work where you had to create a ‘useless website’. There weren’t many rules to the contest (make it publicly accessible, SSFW, enter as many times as you like), so I decided to hedge my bets and create/submit half a dozen simple sites all using the same concept of randomly generating something.

It was a good example of disposable software as I could churn out an entire site in 10 or 15 minutes including publishing it live on github and didn’t have to worry about tests/technical debt or any such thing. It was really fun.

I ended up with a runner’s up award for the sloth site, the winner was my very talented colleague James and his ‘Potato Simulator 2015‘.

Here’s the six sites I created in a week:

pizzagenerator

Pizza Generator: randomly generate a succulent pizza (with bonus exotic mode)

Drink Tea Every Day: Australian Tea Tally

Drink Tea Every Day: Australian Tea Tally

Are you faster than a sloth?

Are you faster than a sloth?

Quote of the Day

Quote of the Day

Are you taller than a giraffe?

Are you taller than a giraffe?

Ralph says...

Ralph says…

Testing beyond requirements? How much is enough?

At the Brisbane Software Testers Meetup last week there was a group discussion about the requirement to test beyond requirements/acceptance criteria and if you’re doing so, how much is enough? Where do you draw the line? It came from an attendee who had a manager pull him up for a production bug that wasn’t found in testing but wasn’t in the requirements. If it wasn’t in the requirements, how could he test it?

In my opinion, testing purely against requirements or acceptance criteria is never enough. Here’s why.

Imagine you have a set of perfectly formed requirements/acceptance criteria, we’ll represent as this blue blob.

Requirements

Then you have a perfectly formed software system your team has built represented by this yellow blob

System

In a perfect, yet non-existent, world, all the requirements/acceptance criteria are covered perfectly by the system, and the system exists of only the requirements/acceptance criteria.

Requirements - System

But in the real world there’s never a perfect overlap. There’s requirements/acceptance criteria that are either missed by the system (part A), or met by the system (part B). These can both be easily verified by requirements or acceptance criteria based testing. But most importantly, there are things in your system that are not specified by any requirements or acceptance criteria (part C).

Requirements - System(1)

These things in part C often exist of requirements that have been made up (assumptions), as well as implicit and unknown requirements.

The biggest flaw about testing against requirements is that you won’t discover these things in part C as they’re not requirements! But, as shown by the example from the tester meetup, even though something may not be specified as a requirement, the business can think they’re a requirement when it effects usage.

Software development should aim to have as few assumptions, implicit and unknown requirements in a system as reasonably possible. Different businesses, systems and software have different tolerances for how much effort is spent on reducing the size of these unknowns, so there’s no one size fits all answer to how much is enough.

But there are two activities that a tester can perform and champion on a team which can drastically reduce the size of these unknown unknowns.

1 – User Story Kick-Offs: I have only worked on agile software development teams over the last number of years so all functionality that I test is developed in the form of a user story. I have found the best way to reduce the number of unknown requirements in a system is to make sure every user story is kicked-off with a BA, tester and developer (often called The Three Amigos) all present and each acceptance criterion is read aloud and understood by the three. At this point, as a tester, I like to raise items that haven’t been thought of so that these can be specified as acceptance criteria and are unlikely to either make it or not make it into the system by other means or assumptions.

2 – Exploratory Testing: As a tester on an agile team I make time to not only test the acceptance criteria and specific user stories, but to explore the system and understand how the stories fit together and to think of scenarios above and beyond what has been specified. Whilst user stories are good at capturing vertical slices of functionality, their weakness, in my opinion, is they are just a ‘slice’ of functionality and often cross-story requirements may be missed or implied. This is where exploratory testing is great for testing these assumptions and raising any issues that may arise across the system.

Summary

I don’t believe there’s a clear answer to how much testing above and beyond requirements/acceptance criteria is enough. There will always be things in a system that weren’t in the requirements and as a team we should strive to reduce the things that fall into that category as much as possible given the resources and time available. It isn’t just the testers role to either just test requirements or be solely responsible/accountable for requirements that aren’t specified, the team should own this risk.

Test your web apps in production? Stylebot can help.

I test in production way too much for my liking (more details in an upcoming blog post).

testinprod

Testing in production is risky, especially because I test in a lot of different environments and they all look the same. I found the only way I could tell which environment I was in was by looking closely at the URL. This was problematic as it led to doing things in a production environment thinking I was using a pre-production or test environment – oops.

I initially thought about putting some environment specific code/CSS into our apps that made the background colour different for each environment, but the solution was complex and it still couldn’t tell me I was using production from a glance.

I recently found the Stylebot extension for Chrome that allows you to locally tweak styles on any websites you visit. I loaded this extension and added our production sites with the background colour set to bright red, so now I immediately know I am using production as it’s bright red, be extra careful.

Stylebot Example

I’ve also set some other environments to be contrasting bright colours (purple, yellow etc.) so I am know from a quick glance what environment I am using.

I like this solution as I haven’t had to change any of our apps at all and it works in all environments: which is just what I needed.

Do you do something similar? Leave a comment below.

Intentionally Disposable Software

Update 11 Feb: Sorry, somehow comments were disabled for this post. This has been resolved now.

There’s a series of code retreats that take place each year where a group of programmers get together to work in groups to solve a problem (kata). They do this in iterations over and over again, and most importantly they delete their entire code at the end of every iteration (typically 45 minutes).

“It’s much easier to walk a new road when the old road is out of sight”

~ Drew Miller

Programmers don’t delete enough production code. Which is funny because I’ve met heaps of programmers, including myself, who love deleting code; it’s a strangely satisfying, cleansing ritual.

What if we could replicate what we do when doing these katas and delete the entire source code of a system every 6 months or a year and start again? Does that sound crazy? Can we do this?

We couldn’t do this with how we currently develop software. It’s because we build software systems that are way too complex, have features that no-one uses and are built to last way too long. We expect our software systems to last 10+ years, and we’re so anxious about long-lasting technology impacts we make overly cautious or conservative decisions that come back to bite us constantly in the years to come. We build technical debt into legacy systems that no one wants to work on. We obsess about re-usability, longevity and salvageability of every piece of code we write. We build massive overly complex regression test suites because we expect the system to work for so long and be maintained for so long we expect it to eventually deteriorate and we want a regression test suite as a ‘safety net’ when it inevitably does.

Maintaining legacy software is like painting the Sydney Harbour Bridge. You start painting it on one side and by the time you get to ‘finish’ it on the other side it’s taken you so long you now need to start repainting the original side again. Wouldn’t it be easier to just build a new bridge?

What we need is Intentionally Disposable Software. We need to build software only designed to last 6 months, a year max. As soon as we deploy it we start immediately on a replacement for it. We put bare minimum effort into maintenance as we’ll just replace what we have in Production as soon as we can: why wash up when you can use fresh paper plates for every meal? As soon as the replacement is ready, we deploy that and completely blow away the old software system. We rinse and repeat.

It’s somewhat similar to planned obsolescence but we don’t do it to annoy our customers and attempt to generate repeat purchases, we do it to refine our software system without any legacy.

We use analytics to tell us exactly what features of the system in production are used and most importantly, what features are little or never used. We don’t build those features into the replacement systems ever again so each system we build is leaner, more focused on the important things it is meant to do and does them better each time. We don’t waste any time on building or supporting unimportant things.

We don’t have time to build up technical debt. We aren’t anxious about choosing a wrong technology. Did we use AngularJS and now hate it? Never fear, we start work immediately on our new system replacement and use ReactJS (or whatever the latest/coolest framework is).

Developer happiness skyrockets! No legacy code! No technical debt! Everyone can work on the latest/best technology to get the job done and want to stick around in our organization to do just that. It’s like being a consultant without being a consultant. Because everyone has already implemented the same thing before, everyone is aware of the gotchas, so whilst people are constantly learning new technology, they’re efficient because they know what they’re actually trying to achieve. And because we’re building such a lean system we’re lean in our approach.

We do need to make sure we use open standards and have an easy way to export/import/migrate data in fast, clean ways – which is good.

The same applies to integration points, we need to be modular enough and use open standards and protocols exclusively to be able to drop out one system and replace it with another that integrates easily.

So what about testing?

If we expect a system to last six months to a year, we need just enough testing. We need just enough testing to make sure the system is built right (doesn’t break), but not too much requirements based testing around building the right system, because we know we won’t build the right system, we’ll be building a ‘more right’ replacement as soon as this one is built.

We need testing that focuses on intention of the system over implementation, because the implementation will constantly change each time we rewrite it. If we write our automated tests in a declarative style devoid of implementation detail we’ll be much better off.

Think


Scenario: Overseas customers must provide ID for expensive purchases
Given I am a non-Australian customer
When I order more than $100 of tea
Then I need to provide additional ID on checkout

Not

Scenario: Overseas customers must provide ID for expensive purchases
Given I go to the Beautiful Tea Home Page
When I click Sign InAnd I enter my username and password
And I click OK
...

Think of Disposable Software like building a house.

You want somewhere to live so you decide to build a house. You design and build a new ‘dream home’ with the best intentions but soon you realize there’s a big difference between what you actually need and what you thought you needed. There’s also some fundamental design flaws you didn’t even realize until it was built and you’re living in it, like mold in the bathroom as it has not enough air-flow and the bedrooms face the wrong direction and are constantly too hot to sleep at night. Plus life has since thrown a new baby into the mix, so 12 months later you find yourself with a house with a lot of design flaws that doesn’t meet your now much clearer, and since expanded, requirements.

So what do you do? Since you’ve invested a lot (financially and emotionally) into your house and you expected it to last ten years of more, you renovate. But the problem with renovating is that you’ve got to work around all the original design flaws, and since the house is already built it’s much more difficult and expensive to make changes to it, and also since you’re living in it, any renovation comes with risk of disruption/displacement to the occupants including an overly sensitive newborn. You find since it’s mainly custom work that the renovations you’re planning will cost nearly as much as the original house.

This is like legacy software. You can’t renovate it easily as it’s already in use, you’re constantly working around its design flaws so it’s much more difficult and costly to make changes to it. Plus it’s really hard to remove the unnecessary parts of your house by renovation.

But what’s the alternative? What if you built the house knowing that come 12 months time you could knock it down, recycle it, and build a new house knowing exactly what you want this time around. You’ll know not to face the bedrooms West. You’ll know the make the bathroom has plenty of air-flow. You’ll even include a bedroom suitable for the baby. But you don’t get too caught up in getting this house ‘perfect’ because come 12 months time you can do it all again. The house could be prefabricated so it’s much cheaper to build off-site in a construction friendly environment, and the migration involves temporarily moving some furniture from the old structure, placing the new house in place with the furniture and recycling the old. You own less objects (clutter) as you know this happens and are prepared for it. As you kids grow up their needs change so instead of doing ‘extensions’ and ‘renovations’ you simply redesign your new house which will be delivered in 12 months time.

This is like Intentionally Disposable Software.

As Alex Bundardzic wrote almost ten years ago in his fantastic Disposable Software post:

“In reality, a good software developer should be able to… deliver a disposable, short lived product that addresses only the pressing needs of the moment, and then simply disappears. No maintenance, no enhancements, no song-and-dance, nothing.”

What do you think?