Understanding the Impact of QA Risks on DX Initiatives

I often find many of the high-level risks have repeated themselves on the projects I’ve worked on. However, the risk profile, risk appetite, detail of the risk, how it gets managed, mitigated and controlled varies massively.

As such, in order to generalise about some of the risks that need to be considered for a particular project, we should really talk about those risk categories. When it comes to DX, I’m making some assumptions for the sake of this blog that the Digital Transformation is digitising business processes and providing services over new, or different, digital channels.

1. Security Risks
Opening up back-end systems to new, public facing user interfaces massively increases the risk that someone naughty will have the means to access your servers and data.

What the hackers then choose to do with your data and/or servers is anyone’s bet but whatever it is, you and your customers won’t like it. What your business does, how high your profile is and who your customers are, will collectively determine how valuable these commodities are to hackers. This in turn affects your reputational and financial risk but not necessarily the likelihood that it will happen.

Hackers will, and do, target any potential vulnerability and generally assume smaller businesses will be less secure and more vulnerable to attack. Your QA process, regardless of your business size and profile, therefore needs to ensure that the appropriate security best practises are adopted, there is rigour in the design and development and that security has been built into the architecture.
Like all testing, security testing is obviously key to any QA process – but it can’t just be a box ticking exercise.

Have We Done the Security Test?
To properly assure the quality of the system’s security and mitigate the security risks, the quality of the test(s) scope and frequency all need assurance.

I’m amazed at how many projects simply plan 5 – 10 days for a single security test at the end of the project. The scope is invariably limited by budget and time and those responsible do the same old thing they’ve always done. It just seems to be accepted that only the “Highs” and “Mediums” will get fixed. There is also little consideration for alternative solutions such as automated security testing built into the deployment pipeline. A good and rigorous QA process should be looking at what is best practise and asking, “has it been followed?”.

2. Performance Risks
Digital Transformation will inevitably increase network and server traffic, so that QA process needs to ensure that this is considered by the project team. Having controls in place to ensure this has been thought about and the risk is mitigated is the QA’s job. Ensuring that non-functional requirements are defined and the architecture is designed to deliver them. Transactional performance is critical to the success of DX projects. The QA process must therefore make sure that adequate thought has been given to ensuring the transactional performance of the delivered system is high enough. Transactional performance means response time and throughput.

Transaction Response Time
How quickly can a user complete an end to end transaction. The end to end transaction is made up of many steps, calls and individual system level transactions. These all need to be thought about and included in any design and test. For many years, testing the transaction response time started and stopped at the server level. How long it takes to render on the GUI didn’t feature in the test plan, let alone how long it takes for a voice response to be heard. New technology requires new testing models and so if your QA process doesn’t ensure that the testing process is testing the right things, the QA process needs to step up.

Transaction Throughput
Unless there are lots of transactions going through our newly transformed digitised process, it’s unlikely the business case will stack up. This means that the delivered system must be able to support many concurrent transactions. The QA process needs to ensure high volume throughput is considered at the database, server, network and integration layers. You need to look for this in the design and again in the testing. I think a good Quality Assurance process should have the responsibility of looking at this level of detail.

3. User Experience Risks
User experience is critical to the success on many Digital Transformation projects. I’d like to see much more done by project teams and therefore the QA teams in the UX space. It’s the QA’s job to make sure UX is not an afterthought and that there has been proper consideration to getting the design right. Again, and apologies if I’m sounding like a broken record by now, but detail matters. There are many risks and they are specific – so the assurance needs to be specific too. This means looking at the why, what and how of UX design.

• Why is the user engaging and what is their motivation?
• What is the user trying to achieve?
  What functionality does the user want?
• What functionality is needed to complete the transaction?
• What features will make things easier and/or better for the user?
• How can the system be accessed?
• How should it look?
• How can it be easier?

We need to ask the question whether expert advice and consultancy has been sought in order for the delivered solution to meet users’ varied needs and desires. This includes all users, different personas, varying levels of experience, physical and mental ability. 

Summary
A Digital Transformation project is likely to be at risk from gaps in security, poor performance and poor user experience. The QA process can help mitigate these risks by getting into the detail of each and taking responsibility for assuring the practices, processes and deliverables meet high standards.