Risk Assessment and Analysis

As testers we are inclined to try to test everything, however factors such as timescales, available resources, technical complexities and costs can prevent this from happening.

We need to use the resources we have as effectively as possible, we need to test the most important things as a priority.

This is where we apply risk-based analysis to determine the areas that carry the most risk and prioritise testing these areas first. Risk is the probability of the asset under test not working as required and the impact if it doesn’t work as required. This approach is known as risk based testing.

When assessing the probability of the risk occurring, you could consider if the asset is new, is it complex, have there been recent failures and does history indicate it will be poor quality?

For impact, consider how critical the asset is, this could mean critical as in human life or critical to the organisation operation, will the organisations reputation suffer how many users rely on it, what is the importance of these users?

When thinking about risk, remember non-functional testing, is sensitive data involved, is it secure, is it recoverable, usable, how many users are expected etc.

You may have the information readily available, you may need to seek information from colleagues or use incident reports.

You can use a matrix that will allocate different areas and tests risk values. This makes assigning the risk values easy and provides clear information on which areas need to be tested as a priority. The complexity of the matrix used will depend upon what works best for you and your organisation but it will usually follow this calculation:

Risk Score = Impact + Probability or Risk Score = Impact x Probability

Once you have allocated and agreed the risk scores with your stakeholders you can agree what needs to be tested as a minimum. Risk scoring provides information to make decisions on which tests need to be run as a minimum and which test can be sacrificed.

For an organisation we’re working with, we wanted to provide an easy mechanism to provide a quick response on the level of testing required for a change. Using the risk based testing approach, we agreed the criteria for higher risk changes.

Once we agreed our criteria, we turned these into simple yes/no questions and using SharePoint created a simple set of questions with an automated decisioning flow. Our colleagues can quickly receive an automated response on whether their change is considered a low enough risk to follow the BAU change process or whether it needs to be accessed in more detail by the test team.