My Thoughts on Increasing Test Maturity and What it Means to Me - Part 4

In my last blog post, I shared the steps I go through to understand what an organisation is trying to achieve through testing and how their current testing model is being implemented. In this, the final blog in the series on increasing test maturity, I want to discuss how I get to the end goal of a mature test model that does what it needs to do.

It’s important to recognise that the term “test maturity” should mean many different things, depending on your perspective. I’ve worked on projects that have immediate implications to life and death such as air traffic control systems and NHS communication platforms, to those with massive financial implications such as core banking platforms for investment banks, to those with less obvious risk such as local radio playback management system. These clients have a very different risk profile and have a very different perspective on what constitutes testing maturity (although the two factors didn’t always correlate as you might expect).

I’m assuming you’ve read the last blog post and have identified what you’re trying to achieve and what your current testing model looks like. This model will, in some form or other, cover the people involved, the processes that have been adopted (not always the same as those that have been documented) and the tools and technology that are being used. It’s now time to look to the future.

What is the “TO BE” model?

Having identified where the problems lie and formed an opinion of what testing is needed, the next stage is to create the Target Operating Model. This will need to be specific and detailed; covering roles and responsibilities, the test policy, the test approach and elements that will be needed to support the test activity. It will cover the people, processes, tools and technology in the same way the 'AS IS model' did but should be the ideal operating model for the system under test and the organisation. It should have ideal practices – not industry best practices – it should be personal, realistic, affordable and have the support of your stakeholders. It will cover test phases that are needed, environments and data that will support the execution of the phases and the skills and team members that will be needed. It should cover the test practices and processes that will fix the incumbent problems and the steps needed to remove obstacles and implement good behaviour.

As well as creating a Target Operating Model to fix the specific problems my client is facing, my role is to advise on implementing test practices that are aligned with the technology, risk and objectives of my client. This is where I draw upon my experience as a consultant and knowledge of what market leaders are doing, what is happening in different sectors, pros and cons of different tools and automation approaches. I look at the skills within the test organisation, the technology and architecture strategy of the organisation as a whole and make recommendations accordingly.

One of my biggest bugbears is individual testers, often contractors, being tasked with defining an automation approach and framework. Invariably, this results in the said person doing the best they can and drawing on their own experience. However, this experience is typically relatively limited. As such complex, far reaching, strategic decisions are based on poor advice. I’ve seen this scenario time and time again, wasting hundreds and thousands of pounds. It’s important to seek advice from experienced professionals with the scars on their back from many different projects and tool sets.

Once I’ve created a straw man Target Operating Model, it’s time to share with the stakeholders in workshops and presentations. This is an important part of the process as I have described in my second post. Gathering feedback and input from the various stakeholders is critical to getting their buy-in and creating the foundations for a successful change.

The Roadmap

Once the Target Operating Model is agreed, it’s possible to perform the gap analysis between where an organisation is now and where it wants to be. The gaps will likely be in all areas - the skills, roles, processes, environments and tools. This often takes my clients by surprise as they typically hope/assume that a few changes here and there will fix their broken model that has been slowly and quietly degrading over years. The gaps can be filled but it takes time, effort and money and typically requires the creation of a roadmap.

I usually break it down into short, medium and long-term roadmaps as there’s way too much to do in a time period of less than a year. These roadmaps will need to be supported with project plans and budget estimates. It’s important to recognise that there needs to be a sustained effort led by motivated, dedicated people to realise the required change in maturity. It’s never an easy undertaking. Some things can certainly be adopted and implemented relatively quickly and without cost. These should go into the short-term roadmap. Getting some quick wins is really helpful to demonstrate benefit, create momentum and win support. Implementing the roadmap requires the change management and leadership skills I mentioned in my first post in this series. It’s challenging and frustrating but extremely rewarding when you get it right.