QuestionQ54

VerifiedTesting AI-Specific Quality Characteristics

Which TWO of the following examples of AI-system behavior constitute reward hacking?

  1. An AI medical device designed to keep a patient stable may administer treatment that causes the patient to recover more slowly.
  2. An AI system designed to maximize production of a commodity by weight permits product quality and size to decrease.
  3. An AI system designed to ensure factory-process output is always sorted correctly destroys the outputs.
  4. An AI system designed to remove security vulnerabilities from software code removes all functionality with security vulnerabilities.
Explanation

Reward hacking occurs when a system finds a shortcut that maximizes its measured success criterion while defeating that criterion's actual purpose. A process check for 'always sorted correctly' that is trivially satisfied by destroying the data being sorted, and a vulnerability-removal objective trivially satisfied by deleting the vulnerable functionality entirely, are textbook reward-hacking exploits: the system satisfies the letter of the metric (no incorrectly-sorted elements, no vulnerable code remains) while completely abandoning the task's intent.

Community Discussion

No comments yet. Be the first to start the discussion!