<img alt="" src="https://secure.wauk1care.com/164394.png" style="display:none;">

How UAT Might Fit Into Agile

Posted by Scott Summers on 12/12/2019
The Manifesto for Agile Software Development is based on twelve principles:
  1. Customer satisfaction by early and continuous delivery of valuable software
  2. Welcome changing requirements, even in late development
  3. Deliver working software frequently (weeks rather than months)
  4. Close, daily cooperation between business people and developers
  5. Projects are built around motivated individuals, who should be trusted
  6. Face-to-face conversation is the best form of communication (co-location)
  7. Working software is the primary measure of progress
  8. Sustainable development, able to maintain a constant pace
  9. Continuous attention to technical excellence and good design
  10. Simplicity—the art of maximising the amount of work not done—is essential
  11. Best architectures, requirements, and designs emerge from self-organising teams
  12. Regularly, the team reflects on how to become more effective, and adjusts accordingly

iStock-1163339325

It’s interesting that despite the first principle being; ‘Customer satisfaction by early and continuous delivery of valuable software.’ UAT, as we have traditionally known it, doesn’t fit well into an Agile delivery model. Many Agile teams dispense with UAT and rely more heavily on the Show and Tell session to get customer sign off. In Scrum, this is done in the Sprint Review Meeting and involves a demonstration of the user stories that have been delivered (according to the Definition of Done) in the sprint. One of the objectives is to elicit stakeholder feedback.

This is good practice, fosters collaboration and creates a high level of discipline while also meeting the objectives of the first principle. Product demonstrations should be interactive so that stakeholders have the chance to provide feedback however, I often wonder “Is this enough?”. Participants in the ‘Show and Tell’ and the Sprint Review Meeting should include, amongst others, the Product Owner, Stakeholder and Sponsors and Customers. However, in practise, I tend to find two problems;
  1. Not all the relevant and required user stakeholders can regularly make the, typically 1.5 to 4 hours, meeting; and
  2. The show and tell doesn’t drill into the same level of detail that a thorough UAT does

There’s a third problem that not only affects the value of a product demonstration at the end of a Sprint but also affects how effective an “in-sprint” UAT phase can be. Delivery of working software doesn’t mean that an end to end business process is delivered within a single Sprint. As such it can be difficult to test business processes in a Sprint making it more challenging to meet the objectives of UAT. The Wikipedia definition of UAT fits what I believe most people think of as UAT.

“User acceptance testing (UAT) consists of a process of verifying that a solution works for the user. It is not system testing (ensuring software does not crash and meets documented requirements), but rather ensures that the solution will work for the user (i.e. tests that the user accepts the solution); software vendors often refer to this as "Beta testing".
This testing should be undertaken by a subject-matter expert (SME), preferably the owner or client of the solution under test, and provide a summary of the findings for confirmation to proceed after trial or review. In software development, UAT as one of the final stages of a project often occurs before a client or customer accepts the new system. Users of the system perform tests in line with what would occur in real-life scenarios.”

Because, as in Waterfall, locking down users and getting enough of their time to do thorough testing is difficult, trying to do this in a specific testing window in each Sprint has its challenges. Also, unless the build and deploy processes are automated, or at least slick, deploying into a separate UAT environment, also creates additional work which, if needed every sprint, can become an overhead.

One solution can be the inclusion of a dedicated sprint to perform a UAT after multiple (say 3 or 4) regular sprints. It’s worth noting that I often find a similar solution is necessary for non-functional testing but let’s stay focused on User Acceptance Testing for now. An argument for this approach is where users need to validate the entire system before acceptance. However, the inherent risk in this approach is that delaying UAT until a later sprint in the lifecycle, breaks the good practise of testing early (Shift Left) and increases the impact of any bugs. This is the same issue with the placement of the UAT (and non-functional test) at the end of a Waterfall project.

Whether User Acceptance Testing is done in-sprint or has its own dedicated sprint, similar good practice will increase the efficiency and benefit of this valuable test. It’s worth noting that these are similar for Waterfall projects too.

Some ideas that might help your UAT:
  • Set expectations of the desired outcomes for UAT
  • Be clear to differentiate between a bug and a change request at the start – and every time some raises a bug that is a change (it will happen)
  • Involve someone from the test team to assist the users – this includes system access, training (if needed), following test steps and capturing bugs
  • Identify the user stories to be tested and stick to them. Spreadsheets are fine but a test management tool is more efficient
When it comes to tools to support UAT in agile, I like the Test & feedback Extension for Microsoft Visual Studio/Azure DevOps/VSO. The Microsoft feedback tool keeps track of, and records, everything the user is doing. It also lets the user add additional information about the things he/she encounters. This can be a video or a screenshot. A nice feature is that a screenshot can be edited, have text added, bits highlighted, just like Microsoft Paint and these can be attached to a bug work item. I haven’t fully explored the dozens of other tools out there and some may have similar features, but in my travels, I haven’t yet come across one that I like as much as the Microsoft tool. Please let me know (politely) if you have been using a similar feedback feature in a different test management tool.

UAT should be done by the User and from their perspective. There are pros and cons to doing UAT in-sprint as part of the DOD or in a separate, dedicated sprint. To decide how best to implement UAT in Agile, we must therefore consider.
  • Who are the users that will be deciding to accept or not the delivered product?
  • How can we best engage and get the most from the users we need to execute the tests?
  • The length of the sprint and completeness of business processes at the end of each sprint
  • What do users want to see in UAT?
  • What decisions will be made based on UAT results
I hope this helps you decide if, when and how to do UAT in an Agile delivery model. Good luck, and as always, let me know if you have any questions.
Mobile and UAT Case Study

Topics: Agile,, Software Testing, User Acceptance Testing, UAT

nFocus Blog

Welcome to the nFocus software testing blog. As thought leaders and technical innovators, we created this blog to distil our thoughts, ideas and musings on improving software quality.

Fill out the form below to receive future communications from nFocus including our latest:

  • Blog articles
  • White Papers
  • and plenty more!

Follow us

Follow us on LinkedIn to see our latest content!

Subscribe Here!

Recent Posts

Posts by Topic

see all