NCA-GENL: Generative AI and LLMsDemo

Positional encoding adds information about each token’s position in the input sequence, allowing a transformer to account for token order.

Attention calculates normalized relevance weights over input tokens and uses them to form a weighted combination of their representations. This lets the model focus more on the input words that are most relevant to the current prediction. Positional information is supplied separately, rather than being captured by attention alone. Neural machine translation with attention | TensorFlow

BLEU score is a standard automatic machine-translation evaluation metric. It measures n-gram precision of generated translations against reference translations, with a brevity penalty to discourage overly short output.

Confidential computing secures sensitive data during processing by isolating computation in hardware-based trusted execution environments, helping protect applications and data from unauthorized access or modification. This is a hardware-and-software security technique, making choice A the closest available definition.

Dynamic batching is a Triton Inference Server feature that combines inference requests into dynamically created batches, typically improving throughput.

Certification verifies that an AI model is suitable for its intended purpose and conforms to applicable standards, which may vary by region or industry. Transparency, ethical consideration, and legal compliance are distinct governance and trustworthy-AI concerns.

An autoregressive language model estimates the conditional probability of the next token given the tokens that precede it, using causal context only.

The sigmoid function is a neural-network activation function that maps an input to a value between 0 and 1, providing a nonlinear transformation.

A/B testing provides a controlled comparison of two model versions with comparable user populations, enabling performance differences to be measured and the better-performing version to be identified.

Perplexity measures how well a language model predicts the tokens in text. Lower perplexity indicates that the model assigns higher probability to the observed text, making it a standard intrinsic metric for language-model quality.

RAG grounds generated responses in retrieved material from an external knowledge base and can return citations to that source material. Those citations make the response transparent and allow its factual basis to be verified.

GLUE (General Language Understanding Evaluation) is a collection of tasks used to train, evaluate, and analyze natural language understanding systems across a diverse range of NLU capabilities.

RAPIDS provides GPU-accelerated data-science libraries that use optimized CUDA primitives and GPU memory for end-to-end processing. Performing expensive work on the GPU and avoiding unnecessary CPU–GPU data movement reduces transfer overhead and improves performance.

Larger context windows let an LLM use more of the relevant input when generating a response, which can improve results on long-context tasks. However, handling additional tokens generally requires more computation during inference, so response time is likely to increase.

Machine-translation performance is commonly assessed on a standard benchmark dataset by comparing model outputs with human-produced reference translations, typically using established automatic evaluation metrics.

NVIDIA NeMo is an LLM development framework that supports prompt-based customization, prompt learning/P-Tuning, and parameter-efficient fine-tuning. Its supported PEFT methods include LoRA, P-Tuning, adapters, and IA3.

The NVIDIA NGC Catalog provides a curated collection of GPU-optimized software and resources for AI and data science, such as containers, pretrained models, and SDKs.

Quantization lowers the numerical precision used to represent model values, reducing model and memory requirements. The smaller memory footprint can improve cache utilization, while reduced memory movement and computation can lower power consumption and heat generation.

Response relevancy measures how pertinent the generated answer is to the input query. Context precision measures how effectively the retrieval stage ranks query-relevant context, which is essential for producing accurate, relevant RAG responses. Latency and token throughput are operational performance metrics, not response-accuracy metrics.