Higher or Lower? Tactics for Mastering Software Testing Estimations

Continuing my series on career growth and learning experiences, here’s another blog about another challenge I’ve faced in my career: planning and estimating testing efforts.

Maybe not in the early days, but certainly as your career in testing progresses, one thing you will be asked to do on a regular basis is estimate a testing effort. Now, this could be for a phase of testing (e.g. a UAT), or for an entire project testing effort.

Either way, this was something that used to be daunting for me and often led to a lot of frustration. I was forever doing a "Brucie" and either going ‘higher, higher’ or ‘lower, lower’ than I needed to be! Experience has helped, but there are a few things I wished I’d known a few years ago.

Understanding Software Test Estimation

Software test estimation is the process of predicting and calculating the time, effort, resources, and budget required to complete testing activities for testing phase, or project. It involves assessing various factors to provide an educated estimate of the testing effort needed to ensure the quality and reliability of the software being developed.

Often overlooked, estimation is a crucial phase in the life cycle of software development:

• It aids stakeholders and project managers in allocating suitable resources and establishing them.
• It helps team members, clients, and other stakeholders communicate effectively.
• It plays a pivotal role in fostering realistic expectations by ensuring that reasonable deadlines are set, accounting for the complexities of testing. 

There are some important factors that a person estimating testing for a phase or a project must take into account:

• The time and materials needed to perform the testing tasks, including test design, execution, and reporting.
• The breadth and depth of the testing operations (scope).
• The features and complexity of the software.
• The potential risks.

Common Challenges

• Incomplete or Ambiguous Requirements: It might be difficult to estimate the testing effort when project requirements are ambiguous, lacking details, or are frequently modified.
• Increase in Human Error: Squeezing the test window puts a much greater dependency upon accuracy of the prioritisation of tests and related business impacts.
• Time Constraints and Pressure: Testing is frequently put under pressure to complete within strict deadlines. This can affect the quality, by forcing testing to be less comprehensive, or by shaving off time by adopting a risk approach and removing low priority tests from scope.
• Complexity of the System: Complex systems require more time and effort to create and execute test scenarios, because there is more to analyse, test and more integration points that are at risk.
• Lack of Suitable Resource: A lack of capable or experienced testers could result in a testing estimate being incorrect because the tests will take much longer to prepare and execute.

But there are some techniques that will help:

• Work Phase Breakdown Technique: Apply approximate estimates to all phases (max value) and sum.
• Test Estimation Example: Establish the number of test cases; apply a complexity weighting (by calculating the number of possible outcomes) and sum.
• Use Case Estimate: Calculate the number of decisions in a test, apply a weighting (based on the number of decisions) and sum.
• Add Caveats: Always caveat estimates and add explanatory notes referencing any assumptions and dependencies such as dependency on 3rd party teams who may have different geographical locations, national holidays and time zones.  

Planning & Estimating a Testing Effort: Lessons Learned

So, there are techniques, challenges, and key items, but what are the common sense, experienced based shortcuts I can give you? Here are some practical tips based on my experience:

1. Be Realistic: First up, do not overstretch yourself or your team with an unrealistic workload. Overloading key resource is a very bad idea. For example, I once worked for a client where an experienced PM overruled my testing estimate and submitted a revised (shorter) one based on a key project support technician working 7 days a week. A person who was also the sole support resource for a very flaky system in live (a system that regularly chucked out P1 errors). I’ll give you two guesses how that panned out.

Be realistic as to what a person or team of people can achieve in a day, week or month. That way, you’ve got the option to crank it up if the project starts to fall behind. Base the timeline on working full pelt, and where do you go?

Another reason that estimates are better, being realistic, is that sticking to them is very important in building a good relationship with a client. It makes you look professional and inspires confidence that you know what you are talking about.

What can help you get a realistic estimate?

• Look at previous phases/projects for clues as to how long previous testing took.
• Use all available documentation to understand scope of testing.
• Talk to any resource that has been involved in previous projects.
• Be clear on roles and responsibilities of all people involved in testing – and their availability.
• Assume failure and ensure you add some contingency to your estimate. Things go wrong on projects such as: 
  • Code doesn’t work.
  • Resource isn’t available.
  • Data gets refreshed.
  • Access to test environments get pulled.
  • Functionality changes.
  • Requirements change
  • Servers and system access might differ on weekends so consider planned maintenance and scheduled outages after regular 9-5 office hours, if you might need to factor in evening or weekend testing.
  Try to calculate the likelihood of these things happening and ensure you allow for the potential effect on your timelines.
• Allow for reviews and generation of output as well - especially if your estimate is for an Automation project. Depending on the availability of resource, this could add a decent chunk of time to your testing effort.
• Next, consider the variables that could affect how long a test phase may take to complete:
  • Set-up (environment; resource)
  • Static/reference data
  • System roles and permissions
  • Support requirements.
• Try and mirror the layout/format of the project plan and align your dates with those in that plan (if possible). Project and programme managers appreciate streamlined plans that come together effortlessly.
• Always revisit estimates during the test phase – preparation and execution. Things change and an estimate may need to change with them. Also, it’s essential that this (and anything else to do with the estimate) is communicated effectively, especially in terms of restrictions on the estimate.

Conclusion

In the context of planning and estimating a testing effort, the establishment of a structured testing process is paramount for project success. A well-defined test strategy, encompassing factors such as point estimation and function point analysis, forms the cornerstone of effective test management.

This entails assessing the time, effort, resources, and budget required to execute testing activities accurately, aligning with the project's overall goals. Furthermore, crafting a meticulous work breakdown structure (WBS) aids in systematically organising testing tasks, ensuring comprehensive coverage of the test scope.

Techniques like work phase breakdown, test estimation examples, and use case estimates offer practical methodologies to derive accurate estimates despite common challenges such as incomplete requirements, change requests, unplanned absence, lack of available resource/skills, fluctuating workloads and time constraints.

By incorporating realistic expectations, contingency planning, and consideration of variables into the estimation process, testers can mitigate risks and foster effective communication among stakeholders, thereby enhancing the likelihood of project success.

Test estimation is a critical process in software development that involves predicting the time, effort, and resources needed for testing activities. It helps in efficient resource allocation, realistic project planning, and effective communication among stakeholders.

Various techniques will help you. Challenges like incomplete requirements, time pressure, system complexity, and resource availability will cause you headaches.

By keeping these key points in mind, you pave the way for smoother project execution and more precise testing effort estimates.

• Be realistic
• Consider all variables
• Ensure you build in contingency
• Factor in the bug fix cycle / defect management cycle
• Ensure that the availability of resource is taken into account
• Communicate
• Playback your estimate to stakeholders with supporting data to signpost your calculation workings