CT-AI: Certified Tester AI Testing Version 2.0Demo

Which ONE of the following scenarios would allow an ML model to be MOST effective at determining the criticality of newly identified defects?

• A. A new application which is in the early stages of its first test cycle
• B. An old application with lots of defect records but a brand new development and test team
• C. An old application where defect records are linked to failed tests and production incidents
• D. An old application with few critical defect records and many non-critical defect records

An ML engineer using supervised learning must label images of football games according to the football’s location in each image. Which ONE of the following labeling approaches can be used?

Data used for an object-detection ML system was found to have been incorrectly labeled in many instances.

Which ONE of the following options is MOST likely to result from this problem?

Which ONE of the following is NOT likely to cause a data-quality issue affecting a single ML model?

“Arihant Meditation” is a startup that uses AI to help people achieve deeper and better meditation by analyzing various factors, such as meditation time and duration, pulse, blood pressure, EEG patterns, etc. Its model accuracy and other functional performance parameters have not yet reached the desired level.

Which ONE of the following factors is NOT a factor affecting ML functional performance?

A facial-recognition system is being deployed at airports to scan passengers’ faces and compare them with a vaccination database, in order to identify unvaccinated passengers during a pandemic.

Several components are involved, including cameras, a model that segments the image, and a model that identifies the face and matches it with a known photograph. It is important to have few false negatives and to ensure passengers cannot subvert the system.

Which ONE of the following testing types is the MOST appropriate for the tests selected during system testing?

“AterEgo” is a product that uses self-learning to predict a pilot’s behavior in combat situations across various terrains and enemy-aircraft formations. After training, when the model was exposed to real-world data, it was found to perform poorly. A large amount of data gathered from actual fights and combat exercises in real planes was used to train and test the product. In addition, the data underwent quality testing to make it suitable for training and testing purposes.

Which ONE of the following options is LEAST likely to describe a possible reason for the decline in performance, particularly considering the self-learning nature of the AI system?

In a coffee-producing region of Colombia, weather storms have caused substantial production losses. This has led to a major decline in the stock prices of coffee bean-producing companies.

Which ONE of the following testing types SHOULD be performed for an ML stock-price prediction model to detect the effect of a phenomenon such as the one described on coffee stock prices?

AI-enabled medical devices are now used to automate certain parts of medical diagnostic processes. Because these are life-critical processes, the relevant authorities are considering obtaining appropriate certifications for these AI-enabled medical devices. This certification may involve several facets of AI testing (1–5):

1. Autonomy
2. Maintainability
3. Safety
4. Transparency
5. Side effects/reward hacking

Which ONE of the following options includes the three MOST required aspects to be satisfied for certification of the AI-enabled medical devices described above?

Which ONE of the following is MOST likely to indicate a problem with underfitting in a machine learning model?

Which ONE of the following labeling approaches demands the least time and effort?

When confirming that an autonomous AI-based system behaves appropriately, which of the following is MOST important to include?

Which ONE of the following combinations of training, validation, and test data is used in the process of learning/creating a model?

You are a test manager planning testing for an invoice-financing company. The company purchases unpaid invoices from businesses and gives them immediate cash.

The company is replacing its current conventional system, which uses company accounting records as inputs, with an ML system that classifies each sales invoice as one that should or should not be bought. Significant historical production data is available. It is important that invoices are not purchased incorrectly.

Which ONE of the following test techniques would be MOST appropriate to plan for system testing?

Which of the following problems is best addressed using supervised regression learning?

A company uses a spam filter to try to identify which emails should be marked as spam. The filter creates detection rules that cause a message to be categorized as spam. An attacker wants every message internal to the company to be categorized as spam. Therefore, the attacker sends messages with obvious red flags in the email body and alters the from portion to make the emails appear to have been sent by company members. The testers plan to use exploratory data analysis (EDA) to detect the attack and use this information to prevent future adversarial attacks.

How can EDA be used to detect this attack?

You are training a robot vacuum with a neural network to navigate without colliding with objects. You create a reward scheme that promotes speed while penalizing contact with the bumper sensors. Rather than what you expected, the vacuum has learned to drive backward because it has no bumpers on its rear.

What type of behavior does this illustrate?

Which ONE of the following statements BEST describes a testing challenge that specifically applies to a self-learning system?

Which ONE of the following does NOT describe an AI-technology-related characteristic that distinguishes AI test environments from other test environments?

Which TWO of the following system examples BEST represent regression?

1. Predicting a person's age
2. Predicting whether someone is over 18 years old
3. Predicting the amount of fuel needed for a journey
4. Predicting whether a release will pass all required tests

A motorcycle engine repair shop wants to detect a leaking exhaust valve and repair it before it fails, causing catastrophic engine damage. The shop developed and trained a predictive model using historical data files from known healthy engines and engines that experienced catastrophic failure because of exhaust valve failure. The shop evaluated 200 engines with this model, then disassembled the engines to determine the valves’ true condition and recorded the results in the confusion matrix below.

What is this predictive model’s precision?

You have been creating test automation for an e-commerce system. One issue you are encountering is frequent failures in GUI object recognition. You have determined that this occurs because developers change the identifiers when making code updates.

How could AI help make the automation more reliable?

Consider a machine-learning model that attempts to predict whether a patient is at risk of stroke. The model gathers information on each patient about blood pressure, red blood cell count, smoking status, history of heart disease, cholesterol level, and demographics. It then uses a decision tree to predict whether the associated patient is likely to have a stroke in the near future. After the model is created using a training dataset, it is used to predict stroke risk for 80 additional patients. The table below shows a confusion matrix indicating whether the model made correct or incorrect predictions.

The testers calculated what they believe is an appropriate functional performance metric for the model, obtaining a value of 2/3, or 0.6667.

Which metric did the testers calculate?

Which of the following illustrates an input change that an AI system would be expected to adapt to?