Benefits of effective 'User Acceptance Testing' - nFocus Blog

The role of UAT and how it can be an important driver for quality and continuous improvement. How UAT is often misused or misrepresented - its usefulness and its limitations.

Firstly, what is UAT and who is involved?

UAT is a type of testing that stands for User Acceptance Testing. It’s the way in which we verify if a solution (new or improved) is operating in a manner suited to real-world circumstances and usage. Other types of acceptance testing are alpha testing and beta testing. 

UAT is one of the most critical phases of software development and change implementation. It should be run after unit testing so that development teams are satisfied that their code works as expected and its meets the acceptance criteria.

This verification is carried out by the people who are going to be working with the new system on a daily basis. Essentially checking that they can do their jobs on it.

Why is it important?

Prior to it being moved to the production environment, UAT testing can illustrate that the developed system effectively supports the business’s day-to-day operations. It can help to identify behaviours within the system and confirm that it functions properly and meets the defined business conditions.

Longer term, running effect UAT can help improve software quality and overall acceptance of newly implemented software (or iterations).

What can it achieve?

Some of the questions an effective UAT will answer are:

• • Is the software suited to the end-users’ needs?
  • Does the system behave and show expected results?
  • Do users encounter problems using it?

The benefits an effective UAT can bring include:

• • Provide further opportunities to identify and repair broken features/usability issues
  • Provide end-users with a vision of the new system
  • Increase software robustness and usability
  • Increase end-user happiness
  • Validate whether business requirements are met as defined in the user stories
  • Reduces the risk of finding defects post-production

Sounds simple, right?! So, how can there be such variances in how well it works? Well, for a start, UAT can mean many things to many people. To some, it's the full extent of functional testing on a project or programme: ‘Just chuck it at the users and they can tell us if it works or not’. To others, it's almost a forgotten part of the puzzle: ‘Oh yeah, we'll let the users have a play on it to keep them sweet. Tick a box!’

Neither of these positions is ideal. Like most things in life, the truth and value of something lies in-between the two extremes of what it could be.

At its best, UAT can be a conduit for company-wide confidence in a new system or process. A real aide to a smooth roll-out. A window into the roles required for the system to work at its best for the company. A means by which to get the best processes in place and a whole company take-up. The best way to share system & process knowledge to ensure there are no single points of failure in terms of knowledge.

At its worst, it can foster resentment against the new system or processes. It can be an irrelevant box-ticking exercise, staffed by the wrong people, looking at the wrong thing. It can make bad processes into the new solution. It can lead development down the wrong path. It can over-promote obscure scenarios. It can over-hype minor issues and can create unnecessary frustration.

So how do we ensure our UAT is on the right side of the coin? Let’s break this down into some key factors of what makes an effective UAT:

• • What should our UAT be ?
  • When should we do our UAT?
  • Who should be involved in our UAT ?
  • What does our UAT need ?

What should our UAT be?

Full UAT is the final phase of functional testing prior to the system being rolled out to production or live. Where business users logged in as ‘real’ system roles, enact the to-be processes or user journey’s on the full solution to determine if the new or enhanced system can do the job that is required. That’s it! 

What it's not...

We cannot maximise UAT’s role. UAT is not a substitute for system or integration testing. It cannot be the only form of functional testing done. It cannot be based on 'as is' processes. It cannot be about doing your old/existing job on the new system. It cannot be a design or re-design forum.

Similarly, we cannot minimise its role either. UAT should not be an afterthought. Users not engaged properly, not made to feel an important part of the process. We should not treat it as a box ticking exercise and look to sideline the results.

When do we do our UAT?

By the time UAT kicks off, the programme should be 100% confident that the full solution works from end-to-end. We should have a full set of reviewed user journeys, written and approved. All major functional issues have been identified and resolved. Key business users have been identified and had basic training on the new system.

When a dedicated fully operational test environment is ready, it performs UAT alongside but it must be separate from operational acceptance testing (OAT) and any non-functional testing such as performance,  load & stress testing.

When not...

• • Prior to the system being functionally ready
  • Prior to processes’ being signed off
  • In place of any other functional testing
  • With the wrong people available
  • With major defects outstanding
  • In a shared environment overlapping other test phases
  • With compromised data
  • Prior to user journeys being written

Who does our UAT?

• • Key business end users
  • People who are flexible to change
  • People who understand that it is a test system and won't panic if something goes wrong
  • People who don't have pre-conceived (usually negative) ideas about new system tests
  • Ideally, but not limited to, people who have worked on projects/implementations/releases before
  • Supported by testing professionals, who can help manage and record progress and ensure the results are analysed correctly

What does our UAT need?

All prior phases of functional testing completed and signed off, with no major defects outstanding: 

• • Fully developed Application code, applied to a fully functioning, dedicated ‘as live’ UAT environment containing ‘clean’ test data
  • Knowledgeable business users available for the duration of the phase
  • All supporting documentation (UAT Plan, Schedule, Test Scripts & Test Cases) in place
  • Project support structure in place

The most important thing to remember is that you usually only get one chance with UAT. The initial perception of it will be the lasting perception of it, no matter what is done after that. But if we approach it properly, UAT can help improve software quality and overall acceptance of newly implemented software (or iterations). Who doesn’t want that?!