QuestionQ3
VerifiedIntroduction to Generative AI for Software TestingIn the context of Large Language Models (LLMs), which of the following statements (i–v) about tokenization and the context window are correct?
- Tokenization is the process of splitting textual input into smaller units called tokens.
- The context window determines the maximum number of tokens an LLM can consider at one time to maintain coherence.
- Increasing an LLM’s context-window size generally reduces computational complexity and processing time.
- Tokenization primarily consists of converting tokens into high-dimensional vectors to capture their semantic relationships.
- A larger context window enables an LLM to maintain coherence across longer passages, such as when analyzing large test logs.
Community Discussion