Skip to content

Commit

Permalink
Update correct DMP link on Tutorial Notebook (#2722)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #2722

Noticed this link was not updated from recent update to docs when i went through the tutorial again

Reviewed By: iamzainhuda

Differential Revision: D68924131

fbshipit-source-id: c6434b08cd2a5c953e342341037c3d08132683f0
  • Loading branch information
aporialiao authored and facebook-github-bot committed Feb 4, 2025
1 parent fa52570 commit 9269e73
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion TorchRec_Interactive_Tutorial_Notebook_OSS_version.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1818,7 +1818,7 @@
"source": [
"### DistributedModelParallel\n",
"\n",
"We have now explored sharding a single EmbeddingBagCollection! We were able to take the `EmbeddingBagCollectionSharder` and use the unsharded `EmbeddingBagCollection` to generate a `ShardedEmbeddingBagCollection` module. This workflow is fine, but typically when doing model parallel, [`DistributedModelParallel`](https://pytorch.org/torchrec/torchrec.distributed.html#torchrec.distributed.model_parallel.DistributedModelParallel) (DMP) is used as the standard interface. When wrapping your model (in our case `ebc`), with DMP, the following will occur:\n",
"We have now explored sharding a single EmbeddingBagCollection! We were able to take the `EmbeddingBagCollectionSharder` and use the unsharded `EmbeddingBagCollection` to generate a `ShardedEmbeddingBagCollection` module. This workflow is fine, but typically when doing model parallel, [`DistributedModelParallel`](https://pytorch.org/torchrec/model-parallel-api-reference.html#model-parallel) (DMP) is used as the standard interface. When wrapping your model (in our case `ebc`), with DMP, the following will occur:\n",
"\n",
"1. Decide how to shard the model. DMP will collect the available ‘sharders’ and come up with a ‘plan’ of the optimal way to shard the embedding table(s) (i.e, the EmbeddingBagCollection)\n",
"2. Actually shard the model. This includes allocating memory for each embedding table on the appropriate device(s).\n",
Expand Down

0 comments on commit 9269e73

Please sign in to comment.