The Possibilities & The Impact of UAT on Agile Performance
The Manifesto for Agile Software Development is based on twelve principles:
- Customer satisfaction by early and continuous delivery of valuable software
- Welcome changing requirements, even in late development
- Deliver working software frequently (weeks rather than months)
- Close, daily cooperation between business people and developers
- Projects are built around motivated individuals, who should be trusted
- Face-to-face conversation is the best form of communication (co-location)
- Working software is the primary measure of progress
- Sustainable development, able to maintain a constant pace
- Continuous attention to technical excellence and good design
- Simplicity - the art of maximising the amount of work not done - is essential
- Best architectures, requirements, and designs emerge from self-organising teams
- Regularly, the team reflects on how to become more effective, and adjusts accordingly
It’s interesting that despite the first principle being; ‘Customer satisfaction by early and continuous delivery of valuable software.’ UAT, as we have traditionally known it, doesn’t fit well into an Agile delivery model. Many agile teams dispense with UAT and rely more heavily on the 'show and tell' session to get customer sign off. In scrum, this is done in the sprint review meeting and involves a demonstration of the user stories that have been delivered (according to the 'definition of done') in the sprint. One of the objectives is to elicit stakeholder's feedback.
This is good practice, fosters collaboration and creates a high-level of discipline while also meeting the objectives of the first principle. Product demonstrations should be interactive so that stakeholders have the chance to provide feedback however, I often wonder “Is this enough?”. Participants in the ‘show and tell’ and the sprint review meeting should include, amongst others, the product owner, stakeholder, sponsors and customers. However, in practise, I tend to find two problems:
- Not all the relevant and required user stakeholders can regularly make the meeting that takes typically 1.5 to 4 hours
- The show and tell doesn’t drill into the same level of detail that a thorough UAT does
There’s a third problem that not only affects the value of a product demonstration at the end of a sprint but also affects how effective an “in-sprint” UAT phase can be. Delivery of working software doesn’t mean that an end-to-end business process is delivered within a single sprint. As such it can be difficult to test business processes in a sprint making it more challenging to meet the objectives of UAT. The Wikipedia definition of UAT fits what I believe most people think of as UAT.
“User acceptance testing (UAT) consists of a process of verifying that a solution works for the user. It is not system testing (ensuring software does not crash and meets documented requirements), but rather ensures that the solution will work for the user (i.e. tests that the user accepts the solution); software vendors often refer to this as "Beta testing". This testing should be undertaken by a subject-matter expert (SME), preferably the owner or client of the solution under test, and provide a summary of the findings for confirmation to proceed after trial or review. In software development, UAT as one of the final stages of a project often occurs before a client or customer accepts the new system. Users of the system perform tests in line with what would occur in real-life scenarios.”
Because, as in Waterfall, locking down users and getting enough of their time to do thorough testing is difficult, trying to do this in a specific testing window in each sprint has its challenges. Also, unless the build and deploy processes are automated or at least slick; deploying into a separate UAT environment also creates additional work which, if needed every sprint can become an overhead.
Whether user acceptance testing is done in-sprint or has its own dedicated sprint, similar good practice will increase the efficiency and benefit of this valuable test. It’s worth noting that these are similar for waterfall projects too.
Some ideas that might help your UAT:
- Set expectations of the desired outcomes for UAT
- Be clear to differentiate between a bug and a change request at the start – and every time some raises a bug that is a change (...it will happen)
- Involve someone from the test team to assist the users – this includes system access, training (if needed), following test steps and capturing bugs
- Identify the user stories to be tested and stick to them. Spreadsheets are fine but a test management tool is more efficient
UAT should be done by the user and from their perspective. There are pros and cons to doing UAT in-sprint as part of the DOD or in a separate, dedicated sprint. To decide how best to implement UAT in Agile, we must therefore consider:
- Who are the users that will be deciding to accept or not the delivered product?
- How can we best engage and get the most from the users we need to execute the tests?
- The length of the sprint and completeness of business processes at the end of each sprint
- What do users want to see in UAT?
- What decisions will be made based on UAT results
I hope this helps you decide if, when and how to do UAT in an agile delivery model. Good luck and as always, let us know if you have any questions!
.png)
