Search results

Tutorial for using DeepSpeed's activation checkpointing instead of PyTorch's

DeepSpeed

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#32409 In huggingface/transformers;

· huyiwen opened on Aug 4, 2024

DeepSpeed sequence parallelism (aka Ulysses) integration with HF transformer

DeepSpeed

Status: Open (in progress).

huggingface/transformersnumber 32305

#32305 In huggingface/transformers;

· samadejacobs opened on Jul 29, 2024

Support saving models trained with DeepSpeed in Trainer callbacks

DeepSpeed

Feature requestRequest for a new featureRequest for a new feature

trainer

Status: Open.

#31338 In huggingface/transformers;

· dwyatte opened on Jun 9, 2024

modeling_t5 incompatible with multiprocessing

DeepSpeed

Good Second IssueIssues that are more difficult to do than "Good First" issues - give it a try if you want!Issues that are more difficult to do than "Good First" issues - give it a try if you want!

Status: Open.

#30280 In huggingface/transformers;

· rangehow opened on Apr 17, 2024

add deepspeed grad ckpt

DeepSpeed

Status: Open (in progress).

huggingface/transformersnumber 30233

#30233 In huggingface/transformers;

· SeunghyunSEO opened on Apr 13, 2024

Support H100 training with FP8 in Trainer and Deepspeed

DeepSpeed

Feature requestRequest for a new featureRequest for a new feature

trainer

Status: Open.

#25333 In huggingface/transformers;

· michaelroyzen opened on Aug 5, 2023

[Deepspeed] [performance] inefficient load with <code>from_pretrained</code> w/ zero3

DeepSpeed

WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progressLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Status: Open.

#12273 In huggingface/transformers;

· stas00 opened on Jun 20, 2021

[Deepspeed zero3] lazy weights init

DeepSpeed

WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progressLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Status: Open.

#12272 In huggingface/transformers;

· stas00 opened on Jun 20, 2021

[2D Parallelism] Tracking feasibility

DeepSpeed

Model ParallelModel Parallelilsm ImplementationsModel Parallelilsm Implementations

Pipeline Parallel

WIPLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progressLabel your PR/Issue with WIP for some long outstanding Issues/PRs that are work in progress

Status: Open.

#9931 In huggingface/transformers;

· stas00 opened on Feb 1, 2021

[DeepSpeed] Features to integrate / Optimizations to add / Experiments to do

DeepSpeed

Feature requestRequest for a new featureRequest for a new feature

Status: Open.

#9606 In huggingface/transformers;

· stas00 opened on Jan 14, 2021