Skip to content

Commit 5f34f2b

Browse files
edenlightningBorda
andauthored
Update installation instructions for FairScale (Lightning-AI#5099)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
1 parent 63fb7f9 commit 5f34f2b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/source/multi_gpu.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -663,7 +663,7 @@ It is highly recommended to use Sharded Training in multi-GPU environments where
663663
A technical note: as batch size scales, storing activations for the backwards pass becomes the bottleneck in training. As a result, sharding optimizer state and gradients becomes less impactful.
664664
Work within the future will bring optional sharding to activations and model parameters to reduce memory further, but come with a speed cost.
665665

666-
To use Sharded Training, you need to first install FairScale using the command below or install all extras using ``pip install pytorch-lightning["extra"]``.
666+
To use Sharded Training, you need to first install FairScale using the command below.
667667

668668
.. code-block:: bash
669669

0 commit comments

Comments
 (0)