CT-AI: Certified Tester AI Testing Version 2.0

By ISTQB · Question Mode

QuestionQ47

Testing AI-Specific Quality Characteristics

“BioSearch” is developing an AI model to predict cancer occurrence by examining X-ray images. The model’s accuracy in isolation has been found to be good. However, when used in the diagnosis lab, its users began complaining about poor-quality results—particularly its inability to detect actual cancer cases—leading to the model’s use being stopped.

A testing expert was asked to identify the deficiencies in test planning that led to this scenario.

Which ONE of the following options is MOST likely to be the reason identified by the test expert?

Explanation

A model can show good accuracy on evaluation data yet fail in practice when the training and test datasets are not sufficiently representative of the operational data. This distribution mismatch produces an overly optimistic assessment and poor generalization to real diagnostic X-rays, including missed cancer cases. The ISTQB AI Testing syllabus notes the risk that datasets may not be representative of expected operational data and requires system testing in an environment closely representative of operation.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!