Black Box Testing
Black box testing, which is also called functional or system testing, ignores the internals of the program and instead focuses on the inputs and the expected results of the program as defined by the requirements. The tester treats the program as a “black box”; the program implementation that generates the program output is unknown. The tester identifies the expected program output from the requirements and can compare the actual output with the expected output. Figure 2 presents a pictorial representation of black box test cases.
Black box testing can be used to find the following types of failures1:
- Incorrect or missing functions,
- Interface errors,
- Errors in data structures or external data base access,
- Behavior or performance errors, and
- Initialization and termination errors.
Typically, black box tests are run by an unbiased third-party, not the programmer who developed the code. However, in the absence of the third-party tester, a developer can test their own code by ignoring the internals. One good strategy for effective black box testing is to write the test cases before starting to write the code. Black box tests test what the customer wants the program to do and are obtained from the program requirements.
Like all test cases, black box test cases have an identifier, a set of specific and repeatable inputs, a set of expected results, and the actual results from executing the test case. The test information can be organized into a Black Box Test Plan document using the table. The Black Box Test Plan should include an overview of the program, tests, and information about how to start testing the program. If the tests require any files, information about where those files are and the file contents should be included.
| Test ID | Description | Expected Results | Actual Results |
|---|---|---|---|
The test ID column of the test plan table contains a unique identifier for the test case. The unique identifier provides a common language for talking about test cases with developers and testers. The test ID could be a number, but a descriptive phrase about the test case is best. The test ID column can also contain additional information like the test author’s name, the requirement under test, the type of test, etc.
The description column contains the steps and inputs the tester enters into the program. The description may also include preconditions for running the test. For example, the program must be started. The information that is recorded in the description should be specific and repeatable, which means that actual values are given. Instead of saying, “enter a dollar amount between 10 and 20” the description should say “enter 15.” For black box testing, the description provides all inputs a user must enter into a program.
The expected results column reports what the program should do as defined by the requirements. This means that the expected results are determined prior to running the program. Like the description column, the expected results should be specific. For black box tests cases, the expected results report what the program presents to the user through the user interface.
The actual results column is filled in when the test is executed. Ideally, the actual results should match the expected results. When they do not match, the test fails, and work to debug the program commences. Failing and passing test cases should be rerun after debugging. Passing tests should be rerun to ensure that any changes to the program under test did not cause a new fault.
We are covering three black box testing strategies: testing requirements, testing representative values of equivalence classes, and testing boundary values. Some of the test cases generated by one strategy may also be generated by another strategy.
References
-
Pressman, R. S. (2005). Software Engineering: A Practitioner’s Approach (6th ed.). McGraw-Hill. ↩