QuestionQ58

VerifiedVerifying the Test Automation Solution

You executed an automated test suite for a product released into production. Even though every test passed, a major production failure occurred in an area that your automated tests covered well.

You run the automated tests again, and one test now fails; this failure is directly related to the reported production defect. You decide to run the automated test suite once more on the same version of the SUT, and the test now passes.

What SHOULD you do now to verify the validity of the automated tests?

Explanation

A test that alternates between passing and failing against the same SUT version is intermittent and produces an untrustworthy result. Isolating it from the suite prevents it from undermining reliable test execution while its cause—such as test-data dependence, timing, environment instability, or nondeterministic SUT behavior—is investigated.

Community Discussion

No comments yet. Be the first to start the discussion!