Introduction to Software Quality Challenges

This week's blog comes from our guest writer, Renee Mineart. 

I am often (far too often if you ask me) heard saying to my colleagues the phrase, ‘should is my favourite IT word’. 

I normally say this right after someone in development or on a technical team says to me, ‘we’re going to apply this patch and everything should be fine. The customers should not experience any issues.’ or when saying ‘once we’ve installed this new software, the users will just need to reboot and everything should work better than before.’

When I hear the word “should”, it screams out to me that something in the process, however small, hasn’t been tested and the team are relying upon past experience (with a sugar-coated sprinkling of luck on top) for everything to go smoothly. That tends to worry me.

I’m not talking about theoretical concepts or what ITIL and agile says we should do. I’m speaking from experience, often my own experience, when software quality challenges have gone horribly wrong despite initially having high confidence that they were going to go as planned. 

If every single step in the process or if every contingency and back out plan has been thoroughly tested, then there should be no “should”. Just confidence, knowing it will work. 

Yes, one can argue that you can’t possibly test everything and that you can’t plan for every potential occurrence of an error. That’s very true. That’s why we have back out plans and why we test along the way with a nice UAT towards the end. All cumulating into a last-minute Go/No Go meeting.

However, when you are close to a project that you’ve been working on for weeks, it’s incredibly easy to miss the small things. It’s often these small things that come back and bite us hard when we least expect it. This is why sandboxes, dev and test systems exist. So, we can hammer out the “shoulds” and replace them with test results, or at the very least a list of bug fixes before the customer experiences a failure first-hand. 

“Should” says to me that there may be an accumulation of small things that are easily overlooked, but when combined, can cause a failure. 

I got my private pilot’s license during my second year of university in a small town in southern Arkansas. There was one particular lesson that has always stuck with me and is part of the reason why I don’t trust “should”. He told us a story that went something like this. 

"Imagine you are planning a flight tomorrow. It’s a private charter to take a paying passenger from point A to point B. You’re going to fly back alone in the afternoon. It’s a flight you’ve done a hundred times before and you could almost do it with your eyes closed. 

But you didn’t sleep very well and hit the snooze a few too many times that morning. As a result, you don’t have time for breakfast. This didn't bother you as you’ll just grab a sandwich at the airport and eat it mid-flight. You’ve done it before, no big deal. 

Then your partner reminds you that you have dinner plans that evening. She wanted to make sure that you will get back in time as you didn't previously. She had to cancel her plans before which she hates doing. “Yeah, yeah, no problem,” you say as you head out the door.

You get to your car and it has a flat tyre. Swearing, you quickly throw the spare on before heading off. Halfway to the airport, you hit thick traffic due to some road works you had forgotten about. Now you’re hungry and stressed because your partner is on your case, angry about the flat tyre and screaming at the traffic. 

You get to the airport and your passenger is waiting, anxious to get to their destination on time for a meeting. So, you check the weather, submit your flight plan from the last time you did this route and start your pre-flight checks. All looks good apart from the oil being a bit low. Not dangerously low, but you’ll want to remember to top it up when you get back. The plane is due for an oil change soon anyway.

Passenger; check. Fuel; check. Oil pressure; check. All looks good and you lead the plane to the runway. Cleared for departure, full throttle, nose up and you quickly climb to 8,000 feet, level out and set your course. Then the engine starts to sputter but you don’t respond immediately because you’re not fully awake. It also doesn't help that you are already frustrated that your day hasn't started off well. It takes a few moments to focus your thoughts and take action. But by then the engine has stopped and now you have real problems!"

The thing is, any single one of these disruptions to your routine wouldn’t be a problem and had never been a problem before. However, when combined and multiple small things pile up, that’s when disaster strikes. We get lulled into a false sense of security because we’ve gotten by on 98% perfect in the past.

When I hear a software developer, tester or technician say “should”, my mind instantly leaps to this scenario and I begin to wonder how many small things may have been overlooked in the rush to go live on time?

My instructor’s solution to this problem was straightforward. If  you’re not in the right state of mind to fly and if the plane isn’t in perfect condition, then don’t fly. Get a different pilot, get a different plane, or reschedule. 

Too often, we are so focused on the goal of delivering a product that we fail to step back and take an honest look at the situation. Have we tested everything we can test? Have we thought about the user and how they are going to use the software? I promise, it is far better to deliver a working product a little late than to crash the plane right after take-off.