QuestionQ59

VerifiedContinuous Improvement

The functional-regression test automation suite ran successfully during the first two sprints, with no failures encountered in its runs. The automation suite records every test case as either ‘pass’ or ‘fail’ and includes excellent recovery capability.

In the third sprint, the TAS log reported several test cases as ‘fail’. You investigated every test case and determined that most failures resulted from a defect in a keyword script rather than in the SUT. For cases in which the failure was in the SUT, defect reports were raised, but developers returned several of them requesting more information to reproduce the problem.

Which additional TAS log items SHOULD be added to BEST improve failure analysis and defect reporting in future sprints?

  • a. Dynamic measurement information about the SUT.
  • b. A ‘TAS error’ status, in addition to ‘pass’ and ‘fail’, for each test case.
  • c. A colour-coding scheme in which ‘pass’ is red and ‘fail’ is green.
  • d. A counter that determines how many times each test case has run.
  • e. System-configuration information, including software/firmware and operating-system versions.
  • f. A copy of the source code for every executed Keyword script.
Explanation

A TAS-error status classifies an execution problem originating in the test automation solution rather than the SUT, preventing automation-script defects from being reported as product failures. Recording software, firmware, SUT, and operating-system configuration provides the execution environment needed to analyze and reproduce genuine SUT defects. ISTQB’s Test Automation Engineering syllabus specifically identifies passed, failed, and TAS-failure execution statuses, and calls for configuration logging that includes software/firmware and operating-system versions.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!