QuestionQ98Verified/Understand fundamental AI conceptsCopy linkReport a problemAsk AstroTutorSave questionWhat challenge is addressed by distributed training that uses model parallelization?AFine-tuning models when data scientists are unsure which pretrained model will work best for their use caseBAccelerating experimentation on servers with multi-core processorsCTraining very large models that cannot fit on a single GPUDAvoiding drift by training multiple different models that check each other's resultsShow AnswerAnswerExplanationModel parallelization splits one model across multiple GPUs, allowing training to proceed when the complete model cannot fit in the memory of a single GPU. PyTorch documentation states that model parallelism is required when a model does not fit on a GPU.Learn morePyTorch Distributed OverviewShow comments0Show commentsCommunity DiscussionNo comments yet. Be the first to start the discussion!
Community Discussion