Introduction

Hi, I’m Jane Kelly, I have worked in the IT industry focusing mostly as a manual test analyst and consultant. I've embodied various roles akin to the many hats worn by a manual tester, adapting to the evolving needs of each project.

One of the critical roles within a project involves reviewing and analysing requirements and other documents. This process includes breaking them down, designing, and writing test cases or scenarios that can be executed in a test environment to demonstrate whether the requirements have been met. This is the first area where a tester, can help a project reduce costs, by identifying poorly written, ambiguous, or misleading requirements.

In this blog, I’ll discuss where a tester involved from design to implementation, helps to improve customer satisfaction and reduce issues being released in the real-world and can have a positive impact on project costs.

Let’s Talk About Costs

Getting the balance right between quality assurance and delivery costs within a project is challenging within the fast-paced IT industry. Development expenses can escalate when project structures and processes are not aligned or understood. Despite these challenges, when on a project, my goal is always to support my projects and clients to achieve their objectives and demonstrate that investment in testing provides a valuable return on investment (ROI) - a release that meets expectations for quality and time to market.

Crucial Factors Influencing Testing Costs

Typically, I have found that issues consistently arise in five key areas:

• Requirements
• Ownership
• Resources
• Test data
• Lessons learned

Here is an example about ambiguity in requirements costing more money...

Example of Ambiguity in Requirements

Original Requirement: User must be able to login.

On one project, I was given a requirement to test written just like that. At first glance, the requirement seems straightforward and easy to write test cases for. However, with my testing hat on, I’m able to delve further, questioning details and intentions behind the requirement. I liaise closely with the developer, solution designers and business analyst to identify areas of risks and functionality that could be inadvertently affected.

If a change impacts login functionality, I need to know: 'What type of users must be able to login?' This particular project had a variety of versions of software across global geographical locations. Some of the staff had multiple live, test, and training accounts. I had to consider impacts to cloud and the on-premises elements.

Typically, project personnel are not all aligned on the correct environment to be tested. I asked for details about different user hierarchy and restricted accounts, such as HR, managers, superusers, guest users, and training personnel. There are many other conditions to check for impacts, depending on what is changing.

Other considerations include: 

• limits on password resets
• expiration on temporary passwords
• password format complexity
• number of successive logins permitted
• multifactor authentication
• variety of devices used

On a humorous and personal note, as a tester, I want to sleep easily at night so, I will always aim to mitigate relevant risks as much as I can to protect the client and their systems.

Remember, anything not tested, isn’t guaranteed to work in your live production environment. Automated testing can help ensure thorough coverage. Testing, like all services costs money; however, the implications and costs associated with the issues in your operational space can far outweigh preventative costs.

The reason is that costs can increase is that because ambiguous requirements need reviewing again, reassessment for impact and re-sign off. Alternatively, there can be queries causing delays in testing or queries and ambiguity are missed, they can cause issues in live production.

My advice is to keep testing relevant to the changes in scope and evaluate relevant risk areas thoroughly.

Establishing Clear Ownership

Another area where clients gain ROI is establishing clear ownership. Often, I see project elements drift and are not able to control them directly, as they sit within the client, or external partner space. I step up during times like this, by escalation to call these out, ask questions and suggesting potential resolutions. Meanwhile, impacts to testing activities are mitigated by restructuring activity.

For instance, I worked on a project where the ownership was initially ambiguous due to an internal organisational restructure, a newly recruited trainee, a hospital operation, and busy holiday season.

To change this challenge into an opportunity, I escalated the matter through my test manager and organised a call with the project manager (PM). During this call, we reviewed the context, discussed potential solutions, and accessed impacts, making them visible to the executive team for efficient management. By enabling temporary alternative resources, we quickly and effectively resolved the issue.

Meanwhile, I redirected focus to other unaffected testing areas, ensuring that we still met the project milestone. This proactive and solution-oriented approach ensures that projects stay on track and clients experience a seamless and efficient process, ultimately enhancing their ROI.

Resource Management

Resource management issues often arise, whether they involve human resources or technical assets, particularly shared ones.

During a project, I faced resource challenges when business resources were scheduled for User Acceptance Testing (UAT) within an already packed schedule. Unexpectedly, some team members had pre-booked annual leave dates that clashed with the testing timeline, which could have caused delivery delays and increased costs.

We worked collaboratively to restructure the testing sequence and reallocate tasks, ensuring that the work continued smoothly without affecting the project milestones. By quickly and effectively adapting to the situation, we maintained our project timeline and controlled costs, turning a potential setback into a success.

Test Data Challenges

Test data is another inherently challenging task and is very time-consuming. Data needs to be a relevant, up-to-date 'live-like' replica of the production systems. By addressing the need upfront, it allows for investigation and an opportunity to format the test data, cleansing and desensitising data where required. 

On one project, creating test subject data manually limited me to about 15 test samples per day. I decided it was worthwhile to invest time using automation to generate test sample subject data. With the support of my nFocus engineers, an automated programme was set to run, which automatically generated 100 test samples overnight. This significantly improved efficiency, ensuring we had ample and accurate test data to facilitate comprehensive testing.

Lessons Learned

Lessons learned, along with post-deployment monitoring and post-implementation review (PIR), are all frequently left off the agenda, once delivery is completed and the project team are pulled onto other areas. However, these are crucial opportunities to reinvest in the process, recognise areas to improve future delivery and save money. It's best to conduct timely post-implementation reviews and maintaining a regular communication between the live helpdesk, the business, the development, and test teams. This data can be valuable in improving the quality of future projects.

The Value of Testing

In addition to those common pitfalls across the industry, the value of testing needs to be understood. The importance of testing on a project is so often undermined and undervalued; remember that testing protects your business, operational systems, brand, and customers.

Every week, in the news, I still see regular big-name corporations hitting the headlines for all the wrong reasons such as data leaks, hacking incidents, systems crashes, and other preventable issues highlight the consequences of inadequate testing.

The best way to work is to keep things short and simple. Look ahead, plan, protect, prepare, and test. Remain task focused. I experienced overly complex processes in many projects. Streamlined critical pathways using only value-added activities will see costs significantly reduced and timescales shortened, giving you the speed to market advantage needed to retain a competitive edge.

Key Considerations for Software Testing Costs

I have found that gaining a thorough understanding of software testing costs and ROI can greatly benefit many people. Here are some key considerations for the typical costs associated with testing delivery that can provide valuable insights:

• Initial Costs - Environment set up, licensing testing tools, personnel and technical support, training costs etc.
• Test Subject Data – Obtaining test sample data, adherence to GDPR, and generating suitable data safely. Re-testing needs clean duplicate data to restage the scenarios for retesting.
• Test Subject Data State - There are costs in getting the state of the test subject data ready to meet test prerequisites.
• Recruitment, Staff Retention & Flexible Resource Costs - Use of outsourced resources can help to mirror the peaks and troughs in delivery, without overheads of permanent staff and bolster the capabilities of your internal test resource.
• Ongoing Costs - Such as system maintenance, IT project testing support, reporting and storage.
• Test Script Writing and Peer Review - There are costs in writing the necessary steps to test the conditions which need checking, and making sure the test steps are accurate and cover the test conditions. These are vital as they feed reporting statistics on daily progress to stakeholders.
• Test execution, raising and retesting of defects - the actual testing itself needs experienced, qualified people with relevant skill sets to verify and validate changes. 
• Virtual Machine Logon
• Remote Login Access
• Activity Time Out for Logged In States

Testing can reduce costs, increase, or improve the overall ROI in so many ways. Consideration needs to be factored in for customers, too. Defects cost money to fix as they need analysis, application of fixes and thorough testing in the development regions before implementation by development team into live systems.

Key Benefits of Testing

• Improve User Experience: Less product system bugs help to retain customers, by personalising and enhancing their customer experience, preventing loss of business and complaints.
• Compliance and Security: Businesses can get fined, or hacked, which leads to loss of revenue, bad publicity, and poor reputation.

Maximising Testing ROI

• Effective Communication: This is fundamental in reducing time lags and inherent costs. Clear, ongoing communication channels can streamline processes, minimise delays, and ensure efficient project progression.
• Flexible Solutions: The level of support can be customised to your needs and budget.
• Health Checks: Collaborating on proof of concepts to establish strategic feasibility of plans.
• Ongoing Support: My colleagues and I strive to provide comprehensive support for your business.
• Ad Hoc Support: Engaging with nFocus at key stages, incrementally to adopt quality assurance in a staggered way.
• Organised Reviews: By reviewing your organisational landscape and future commercial direction, you can receive a gap analysis, a risk assessment, and a roadmap to help achieve your goals.
• ROI in Test Automation: Automating testing tasks can perform human-equivalent tasks in a fraction of the time, greatly enhancing efficiency and ROI.

The Fiscal Impact of Automated Testing

Investing in automated testing can significantly influence your project's financial health, addressing both short-term expenses and long-term gains. The initial cost of setting up automated testing includes expenses for tools, training, and integration into the software development lifecycle.

However, this initial investment can lead to substantial savings on maintenance costs and improve cash flow by reducing the time and resources required for regression tests and unit testing.

To understand the return on investment (ROI), it's crucial to calculate the ROI by comparing the total cost of the investment against the benefits accrued over time. This includes examining the annualised ROI, which offers a clear picture of the long-term financial benefits.

Improved testing efficiency can enhance net profit by accelerating release cycles and ensuring higher quality, reducing post-release defect costs. Engaging team members in creating and maintaining test cases helps in making informed decisions that align with project goals.

While there is a risk of negative ROI if not effectively managed, careful planning and execution can turn this into a positive outcome, ensuring the cost of the investment is justified by the value delivered. This approach not only supports better financial management but also enhances the overall effectiveness and reliability of your software products.

Conclusion

Transparency, and ethical business practices, coupled with a test strategy that aligned with the strategic goals of your project, can significantly mitigate the issues discussed. By identifying dependencies, constraints, risk appetite, budget considerations, and target areas, we can tailor our approach to suit your specific needs. 

As an experienced tester, I prioritise effectively within your test team, ensuring honest communication and relevant contingency planning to establish where spend is justified and where savings can be made.

Striking the right balance for business projects, helps you adopt robust quality assurance processes, embedding those into the deliverables across your delivery lifecycle. This not only improves current projects but also protects your commercial standing for future projects.

I provide an objective review of your commercial goals with full transparency on risk mitigation. This provides you with a clear tangible proposal to meet your needs in a cost-effective, secure manner, considering business continuity, future scalability, operational performance, and protection of systems & data.

Effective collaboration and proactive communication are key to achieving high-quality results in software development. By addressing fundamental issues and providing holistic support, I continue to enjoy helping my clients overcome testing challenges and deliver successful projects while maintaining cost control.

I hope you enjoyed reading about my experiences as a tester, from design to implementation, increasing customer satisfaction and preventing issues being released in the real-world.