Introduction to Black Box Testing
Today, we welcome back Jane Kelly with a new blog.
Welcome to the world of Black Box testing! It remains one of my firm favourites because it’s a method of software testing that looks at the functions of an application. It doesn’t delve into the murky and technical depths of the internal working and structure, nor does it dive into detailed workings within the design or code. So, let’s talk about black box testing techniques and how they can be applied to create test cases
Black box testing requires and looks at the SUT (the subject under test), without concern of how it’s implemented. It can be used early in the life cycle and it’s often the first time test teams get their hands on the released code.
Black box testing is a method of software testing that can be performed without prior knowledge of the application, detail and structure. It’s very popular and can be used to explore the system from an end-to-end perspective. Other less common names for this type of testing are ‘Behavioural Testing’, ‘Opaque Testing’, ‘Closed Box Testing’, ‘Functional’ or ‘Function Centric Testing’.
In simple terms it does just that, it looks at valid and invalid inputs as well as demonstrates how the system subsequently behaves or functions. Black box tests are run by checking the results against a given set of requirements which are set out and hopefully baselined before testing begins. It can be used to give early insights into how the system might behave and be interpreted from a user perspective.
Black box testing is typically applied to three main areas of testing; functional, non-functional and regression testing. It can be used across the various stages of the testing life cycle, for example, in regression, beta testing, user acceptance, unit integration system testing and even load testing to name a few.
Black box tests are often run manually - it’s often referred to as ‘User Interface’ testing and it can produce some interesting and useful results early in the process. For example, user error, individual interpretation, system response times, usability, performance, unexpected application failure and even identifies invalid design assumptions.
Black box testing can be run in an automated fashion, it can be time-consuming to set up but pays dividends later and can be reused with minimal rework.
It’s great that developers can continue their work on the next level of coding whilst testers can explore the already-released software, allowing both teams to progress with their workload at the same time. There are many different types of black box test techniques, here are a few of the common ones:
Boundary Value Analysis
Boundaries can often contain error code, so testing just before, just after and on the boundary are good black box tests. An example of this, if an insurance policy must refer to underwriters when there are more than 3 existing claims for the main driver, we can test 2, 3, 4 x claims. We might also question or test what should happen for a secondary or temporary driver.
Equivalence Partitioning
Equivalence partitioning (or equivalence class partition) is also a good technique, testing each class of data. For example, if values 1-10 are allowed in a data field, we would test -1, 1, 11 and maybe for good measure decimals. It’s faster and more realistic than trying to test every value which would not satisfy a cost benefit assessment.
State Transition Testing
State Transition testing is a type of black box testing that looks at the changes in a data state as it goes through its data journey. A good example is password login states, after too many attempts, a ‘forgot password’ link might be displayed, and a further state may be ‘locked out’ if the allowed given number of login attempts is exceeded. An extension of this might be temporary disablement where lock out state can be left for 10 minutes then normal log in attempt is allowed again.
Decision Tree Testing
This decision table is a common technique often displayed as a Boolean (True or False) / (Yes or No) type flow chart which specifies certain journey data flows depending upon if the criteria has or has not been met. For example, if the user has entered a valid email address on registering an account, there may be an option which is only displayed when a valid email is provided to receive confirmation documents as opposed to hard copy postal documents.
Graph-based Testing
Graph-based testing is a similar spin off technique that considers the relationship between links and input. It uses graphs not tables to visualise the inputs and outputs and the subsequent flow. Put simply, a visual graphical drawing depicting causes (inputs) and effects (outputs).
Error Guessing Technique
Error Guessing technique is a type of exploratory black box testing technique where the tester can create or not create the defect by altering the input values to get clues and evidence to support suspected cause. It can also be used to test around the issue to see what else might be affected. An example of this would be a tester found a defect on a car insurance policy where a drink driving record didn’t load the premium of the policy for the increased risk. It might also be a good idea to check the defect appears in mid-term policy amend stage or renewal stage for van and motorcycle insurance cases.
Sometimes with data version load, we need to test the before state version and after state version to compare results to make sure they reflect stated specifications.
We do black box testing and find that it’s fun, the approach is straightforward and users don’t need prior technical system knowledge. It’s a good early indicator of software quality but can be plagued by environmental stability issues if performed directly after coding. It’s fun because we can legitimately try to break the system by trying different techniques and exploring what might cause the software to behave abnormally. System users don’t always use the system exactly the way it’s designed. They might use shortcuts, tabbing, copy/paste and fill fields out in a different order to that which they are presented in a web page, or bypass certain values to speed up processing.
Black box testing gets its name because the inner workings are not seen or studied by the user. It can include clicks, scrolls, touch, voice, data feeds and enter/key depressions.
We love it because it’s effective and focused which can give quick results on initial code quality confidence and it enhances the user experience.
You may be surprised to learn there are other colour boxes of testing:
- Yellow box looks at warnings.
- Red box validates error messages.
- Green box determines in release testing if the system is environment friendly and will not have social implications.
- Grey box is a mix of white and black testing with maintenance testing in mind.
It might be amusing to some that when a tester ‘sees red’, it doesn’t automatically mean they are testing error messages necessarily.
.png)
