Best Practices for Virtual UAT

Today we welcome back our popular guest author, Renée Elizabeth Mineart.

If I were asked to encapsulate what makes humans unique in a single sentence, I’d probably say: “...our ability to achieve the impossible.” This trait is often best demonstrated when we are faced with adversity - a planet-wide pandemic, for example.

What amazes and fascinates me about our “new normal” is how quickly companies around the world have adapted to working from home. Ask managers back in 2019 if they would allow their team to work from home and many would say that it was impossible; that their team had to be on-site in order to carry out their work.

But now, one pandemic later, we are most of us working from home, quite happily and rather successfully, I might add. In order to achieve this monumental and previously impossible feat, we have all learned to do things differently. One of the areas in which my team are working differently is when it comes to User Acceptance Testing (UAT).

In the “old reality” we would have regular meetings with our user team, discuss upcoming changes and what we required in regard to UAT. We also provided a shared spreadsheet where everyone could log the bugs that they found during testing and we’d review the spreadsheet during meetings. This spreadsheet would then be analysed by the Scrum Master, line by line with each item being prioritised and scheduled for fixing.

Even though we could have just imported that model into a remote working environment, we have taken this opportunity to rethink how we engage with our testers. Our new approach looks something like this:

User Meetings: We still have regular meetings with our user group. The user group, by the way, is made up of at least one representative from each team that uses the software currently in development. But instead of us telling them what we are doing, what they need to test and what we want them to disseminate to their team, the meetings are about us listening to them.

• What do they want to see in the software?
• What do they consider to be a priority?
• What are their teams asking for most?

Comms: We have redesigned our communication plans. Actually, we’ve properly designed a comms plan for the first time, from scratch. We rely heavily on Teams now and have a dedicated channel for each project where anyone in the department can go for the latest updates and status of the project. We deliver more urgent news in our department-wide virtual meetings and send out emails as we near the date for a change or upgrade. Comms is very carefully thought about, planned and executed so the right people get what they need to know at the right time.

UAT Test Document: We have completely revamped how we engage with our testers when it comes to User Acceptance Testing. As a Sprint reaches the point of UAT, the Scrum Master produces a document that summarises the changes we need testing. The cover page of the document lists the nitty gritty details: Sprint number, team members, testers, stakeholders, testing start & end dates, etc. It even has a box for the service owner to sign off on to say they are satisfied the testing is complete.

The document then details what parts of the software have been under development, what we are asking our testers to test as well as some step-by-step instructions to carry out testing. Testers are encouraged to go outside these instructions and try to break it but we only want them to test very specific parts of the software. If we’re developing a new form that is only ready for testing in the client version of the software, we don’t want them to try and test that form in the web version. It’s not ready yet, we know it’s not ready and we don’t want people wasting their time and ours testing it.

Webinar Sessions: With every Sprint, we run a 1-hour webinar session with the testers. This session is run via Teams and is recorded so that if someone can’t attend, they can watch the video later. The Webinar is a chance for us to demo the changes and answer any questions our testers might have.

Tester Videos: Lastly, we also publish a series of very short 2-3 minute videos that walk our testers through what needs to be tested. This is just a video version of the step-by-step instructions written into the UAT document. It is short, sweet and doesn’t have any added splash screens, intros, music, etc. Just a step-by-step of what to test.

You might be asking to yourself, ‘...aren’t you just saying the same thing in different ways?’ I would answer with a resounding ‘YES! That’s the whole point.’ I have testers that know the software intimately and all they want is a 3-minute read and quick instructions to point them in the right general direction. I have testers that are dyslexic and really struggle reading a document, even a short one, on the computer screen. For them, a short 2-minute video is perfect. Some like to ask questions and interact directly with developers, others don't.
By delivering well defined UAT requirements in different ways we are finding a greater uptake in testing and getting better feedback on the product. Especially with people working from home. You can’t just pop around the corner, stick you head into the Scrum Master’s office and ask a question. So, we make sure we provide the answers to our tester’s questions before they ask, in a format that works for them.

Bug Reports: We have also done away with the large, complex spreadsheet previously used for reporting. This spreadsheet forced our testers to review each other’s comments, so feedback wasn’t duplicated. It was a very large spreadsheet with many small boxes in which to enter large amounts of text. It was cumbersome and time-consuming for both testers and the sprint team to use. At present, we have replaced the spreadsheet with a simple one-page word template, with room to add screen grabs on subsequent pages. The template and documents are stored in the Teams channel for the project and we collate the information and add it to our backlog.

Eventually, this form will be incorporated into our ticket tracking software in one of our upcoming Sprints. But the goal is the same, to make it as easy as possible for people to report bugs.

Success or failure often comes down to how readily we can adapt, or not adapt, to an ever-changing world. Sometimes, a simple thing like a new approach to an old problem is all that is needed to achieve the impossible.