Skip to content

Conversation

@DrownFish19
Copy link
Collaborator

PR types

Bug fixes

PR changes

Others

Description

  1. PipelineLayer will create parameters for each layer and call _synchronize_shared_weights() to synchronize the shared parameters.
  2. When setting the model state_dict, _synchronize_shared_weights will be called to synchronize the shared parameters. However, self._init_weights will re-initialize the parameters without synchronizing the shared parameters. If the following step does not load a checkpoint, the shared parameters will be different.
@paddle-bot
Copy link

paddle-bot bot commented Jan 4, 2024

Thanks for your contribution!

@codecov
Copy link

codecov bot commented Jan 4, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (adf9e6f) 57.30% compared to head (2abb57b) 57.30%.
Report is 8 commits behind head on develop.

Files Patch % Lines
paddlenlp/transformers/llama/modeling_auto.py 0.00% 2 Missing ⚠️
paddlenlp/trainer/training_args.py 0.00% 1 Missing ⚠️
paddlenlp/transformers/model_utils.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@ Coverage Diff @@ ## develop #7772 +/- ## =========================================== - Coverage 57.30% 57.30% -0.01%  =========================================== Files 584 584 Lines 87688 87690 +2 =========================================== + Hits 50252 50253 +1  - Misses 37436 37437 +1 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@wawltor wawltor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wawltor wawltor merged commit 487428b into PaddlePaddle:develop Jan 8, 2024
@DrownFish19 DrownFish19 deleted the fix-gpt3-pp branch January 17, 2024 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants