CCA-F: Claude Certified Architect — Foundations

By Anthropic · Question Mode

QuestionQ95

Prompt Engineering & Structured Output

You are developing a structured data-extraction system with Claude. The system pulls information from unstructured documents, validates output with JavaScript Object Notation (JSON) schemas, and preserves high accuracy. It must gracefully handle edge cases and integrate with downstream systems.

Your system has operated with 100% human review for 3 months. Analysis indicates that extractions with model confidence ≥90% achieve 97% accuracy overall. To lower reviewer workload, you intend to automate high-confidence extractions.

Before deployment, which validation step is most critical?

Explanation

Aggregate accuracy does not establish that a confidence threshold is reliable for every document type, extracted field, and edge-case segment. A high overall rate can mask concentrated errors in a segment that downstream systems may depend on, so performance must be evaluated by those segments before high-confidence outputs are automated. Anthropic’s evaluation guidance emphasizes testing across scenarios and reviewing results across test cases to find performance patterns.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!