CY0-001: CompTIA SecAI+Demo

Securing AI systems requires defense in depth across both AI-specific and traditional components. This includes testing guardrails and security behavior, applying lifecycle security controls to models and data, designing a comprehensive security architecture, and using a secure SDLC for the software that builds, integrates, deploys, and operates the models.

Automated penetration testing can use AI to evaluate the environment, adapt to the results of previous actions, and recommend or execute subsequent attack paths to test whether security boundaries can be bypassed.

An AI Center of Excellence provides centralized AI governance. It defines and enforces AI standards, implements processes to evaluate and prioritize AI requests, and creates reusable, approved assets and patterns for organization-wide use.

Checksums help verify data integrity by detecting corruption or unauthorized alteration: recalculating a checksum and comparing it with the expected value reveals whether the dataset has changed.

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is a knowledge base of adversary tactics and techniques targeting AI-enabled systems, making it appropriate for identifying and modeling threats to a custom AI model and its supporting data and inference components.

Improper output handling occurs when an LLM’s output is not adequately validated or sanitized before being passed to downstream systems such as plug-ins. This can allow untrusted model-generated content to trigger attacks against those systems, including code execution or other injection-related impacts.

The NIST AI Risk Management Framework provides guidance for managing AI risks throughout the design, development, use, and evaluation of AI systems, which includes risks arising during model training. NIST AI Risk Management Framework

The EU AI Act is binding law that establishes compliance requirements for AI systems. It classifies many AI systems used in employment and worker management—such as recruiting, employment-related decisions, task allocation, and performance monitoring—as high-risk, making it the relevant legal reference for an audit of HR AI systems. Regulation (EU) 2024/1689 (Artificial Intelligence Act)

One-shot prompting supplies a model with one representative example to steer it toward the desired type of output or evaluation. The example of a successful candidate’s résumé provides that single demonstration.

An LLM firewall provides AI-specific input and output guardrails: it can detect or block adversarial prompts before they reach the model and inspect, filter, or sanitize generated content before it is returned or used. Microsoft guidance likewise identifies prompt-injection protections as controls that block adversarial prompts and requires LLM output to be validated and sanitized before use.

Guardrails enforce input and output policies, allowing unsuitable or unanswerable requests to be filtered or blocked before they consume unnecessary chatbot and model-processing resources. Amazon Bedrock guardrail integrations can validate and block inputs during preprocessing, and guardrail policies can reduce token usage through cost control.

Classification models assign observations to defined categories using learned patterns from labeled examples. Classifying incoming security alerts can prioritize and route high alert volumes for analyst triage; filters are deterministic rules, while automated patching is a separate remediation process. AWS documentation on classification models

Cleansing prepares source data for reliable ingestion and analysis by removing or correcting noise, errors, duplicates, inconsistencies, and irrelevant content. This is especially important when combining structured and unstructured sources for an LLM-enabled SIEM. AWS guidance identifies data preparation and cleaning as an essential data-lifecycle stage and describes removing noise and errors before model ingestion.

Code linting analyzes source code for defects, insecure coding patterns, and policy violations early in the development workflow, enabling vulnerabilities to be identified and corrected before they reach production. Code-scanning tools likewise analyze code to find security vulnerabilities and coding errors.

The EU AI Act is the applicable legal framework that classifies AI tools used for employment, worker management, and access to self-employment—such as recruitment and job-application screening—as high-risk uses. It establishes compliance obligations and enforcement penalties, so it is the relevant reference for reducing the risk of compliance-related fines.

Rate limiting caps the number of requests a client or source can make within a specified time period, helping prevent excessive traffic from exhausting service resources during a denial-of-service attack.

Human-in-the-loop processes incorporate human review and judgment into an AI-assisted workflow before decisions or actions are finalized.

Convolutional neural networks automatically learn relevant features and hierarchical patterns directly from unstructured data, making them well suited to pattern-recognition tasks. Decision trees and logistic regression rely on structured features, while long short-term memory networks are primarily intended for sequential relationships.

LLM utilization is measured in tokens processed. Tokens represent the units of text consumed as input and generated as output, and are commonly used to quantify model usage.

Convolutional neural networks are designed to learn and synthesize patterns in visual data, including facial features and movements, making them a relevant technology for AI-generated deepfake-style video.

Applying least privilege to the service account limits the files, buckets, permissions, and administrative operations the application can access. This constrains the impact of attacker-controlled requests even when the application is manipulated to invoke unintended backend actions.

Guardrails constrain chatbot inputs and outputs using policies, grounding requirements, and validation so the system does not accept or propagate unsupported, irrelevant claims as technical troubleshooting conclusions.

The NIST AI Risk Management Framework provides a use-case-agnostic approach for managing AI risks and incorporating trustworthiness considerations into the design, development, use, and evaluation of AI systems. Its governance, mapping, measurement, and management functions make it the appropriate reference for describing posture-management impacts in an ML use case.

LLM inference is commonly priced by token usage. Adjusting token limits constrains the maximum response length, reducing generated-token consumption and therefore inference cost. AWS guidance likewise recommends controlling response length because shorter model responses lower inference cost.