QuestionQ98

VerifiedUnderstand fundamental AI concepts

What challenge is addressed by distributed training that uses model parallelization?

Explanation

Model parallelization splits one model across multiple GPUs, allowing training to proceed when the complete model cannot fit in the memory of a single GPU. PyTorch documentation states that model parallelism is required when a model does not fit on a GPU.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!