NCA-GENL: Generative AI and LLMsDemo

By NVIDIA · Question Mode

QuestionQ16

Core Machine Learning and AI Knowledge

Your company has moved from a legacy LLM model to a newer model that supports larger sequences and higher token limits. What is the most likely outcome of upgrading to the new model?

  • A The number of tokens is fixed for all existing language models, so there is no benefit to upgrading to higher token limits.
  • B The newer model allows for larger context, so the outputs will improve without increasing inference time overhead.
  • C The newer model allows the same context lengths, but the larger token limit will result in more comprehensive and longer outputs with more detail.
  • D The newer model allows larger context, so outputs will improve, but you will likely incur longer inference times.
Explanation

Larger context windows let an LLM use more of the relevant input when generating a response, which can improve results on long-context tasks. However, handling additional tokens generally requires more computation during inference, so response time is likely to increase.

Community Discussion

No comments yet. Be the first to start the discussion!