Understanding Test Automation In DevOps

Ever heard the expression ‘More speed, less haste’? Acting too quickly and without due diligence, focus and attention to detail will result in avoidable mistakes and thus require even more time to complete the task satisfactorily.

Having a test automation approach is a great way to reduce time in the iteration or sprint but so many organisations get it wrong and spend unnecessary time and money. In today’s world of Continuous Delivery and DevOps it can take too long to get to a point whereby automation is paying you back sufficiently for the effort you have put in.

Firstly, you should take advantage of virtualisation and the cloud to automate your application deployment in your test environments. Making your test environments easy to build and set up is a must but you should also make sure you can run a test automatically in that environment with the application deployed and configured before writing additional tests.

Once you have a proven environment to test upon, then set about creating an automation framework that provides reusable and expandable test cases for automation. These first tests should be built to cover the core application functionality to ensure you iron out debugging issues from the beginning. Start with fewer tests at first, so your initial automation is solid and gives the payoff later.

Now you have a robust framework in place it is worth ensuring that every code change is committed with a test and that test is then added to the automation framework. Within a short period of time you’ll have an expansive set of robust tests.

Six automation issues to watch out for:
1. Be wary of record and playback – or ‘automation light’ can be brittle and awfully time consuming to maintain. We often see this as a starting point for automation where an organisation has few technical testers. There are better ways of dealing with this challenge.

2. Automated tests failing to handle errors correctly will lead to a lot of wasted time – it can be difficult to find the root cause of a problem but solving these early will save time in the long run.

3. Automation can be time consuming - some approaches mean automating tests within the sprint takes too long and ends up being done outside of the sprint. This will cause issues further down the line!

4. Test Automation tooling often only targets the GUI – this can be an inefficient way to test the full system functionality.

5. Custom automation frameworks are often inflexible, brittle and do not scale. We often see organisations employ contract resource to create an automation framework but when the contract resource leaves, the existing team are unable to maintain the framework and thus falls into disrepair.

6. Automation is often left to the developers. Typically, developers have different skills from testers and do not test in the same way. As a result, who tests the automated tests?

Six top automation tips:
1. Automate at the GUI, Services and Database Layers – we always recommend using a common test framework for the GUI, Web Service and Database layers. This means that you can automate much more functionality in a common and robust manner.

2. Be Tool Agnostic and Future Proof Your Test Architecture. It is not un-common for a test team to change their strategic test tool after a number of years. As the technology under tests changes, in many cases so much the tool. To future proof the solution, try and keep it execution tool agnostic.

3. Automate Early – in times gone by, automating testing would begin at the end of the first release when change was likely to be limited. With the adoption of Agile and DevOps this can no longer be the case, with automated tests being created at the same time as the development team writing automated code. To overcome challenges here, we recommend building a common Object Map that is used by developers and testers alike, automated tests can be written by the testers in parallel with application code being written by the developers.

4. Get Testers Doing the Automated Testing - testers have a unique skill set and over the years I’ve observed a much higher return on investment from automated tests created by testers over developers. There are solutions available to put the automated tests in the hands of manual testers who do not have strong coding skills.

5. Create Build Verification Tests and Regression Tests – this will help you get as much benefit from automation as possible. These tests should be added into the continuous integration process which will in turn enforce a much more robust build process, providing instant feedback to the dev team when code is built and deployed.

6. Manage Test Data Well - To avoid false positives, defining, creating and managing test data is just as important as defining, creating and managing your tests. It goes without saying, but real user personal data needs to be anonymised in the context of GDPR legislation.

If you make automated testing an integral and invaluable part of your DevOps process, the code you deploy will be of much higher quality and your applications are much more likely to work as designed first time without the unnecessary rework and retesting.

If all else fails, ask an expert. You will find many test consultancy organisations will provide some kind of test automation consultation. We developed a test automation health check and readiness assessment for just this scenario. For more info please get in touch.