Skip to content

Fix GLM-4 PP Missing Layer When using with PP. #21531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 25, 2025
Merged

Fix GLM-4 PP Missing Layer When using with PP. #21531

merged 1 commit into from
Jul 25, 2025

Conversation

@zRzRzRzRzRzRzR
Copy link
Contributor

@zRzRzRzRzRzRzR zRzRzRzRzRzRzR commented Jul 24, 2025

No description provided.

@zRzRzRzRzRzRzR
fix pp missing layer problem
425ba32 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
gemini-code-assist[bot]
gemini-code-assist bot reviewed Jul 24, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request addresses an issue with pipeline parallelism by handling PPMissingLayer when iterating through model layers. However, the changes introduce a potential RuntimeError on pipeline parallel ranks that do not contain any MoE layers. The review includes suggestions to improve the logic and prevent the error from being raised unnecessarily.
vllm/model_executor/models/glm4_moe.py
Comment on lines +626 to +627
if example_moe is None:
raise RuntimeError("No Glm4MoE layer found in model.layers.")
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This RuntimeError will be raised on pipeline parallel ranks that do not have any MoE layers. This will cause the model initialization to fail. Ensure that this error is only raised when there are supposed to be MoE layers, but none are found.
vllm/model_executor/models/glm4_moe.py
Comment on lines +615 to +618
example_moe = None
for layer in self.model.layers:
if isinstance(layer, PPMissingLayer):
continue
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Consider skipping the loop entirely if self.model.layers is empty to avoid unnecessary iterations and potential errors. This can happen in pipeline parallel settings where some ranks might not have any layers.

Suggested change
example_moe = None
for layer in self.model.layers:
if isinstance(layer, PPMissingLayer):
continue
if self.model.layers:
for layer in self.model.layers:
if isinstance(layer, PPMissingLayer):
continue
@github-actions
Copy link

github-actions bot commented Jul 24, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀
jeejeelee
jeejeelee approved these changes Jul 24, 2025
View reviewed changes
Copy link
Collaborator

@jeejeelee jeejeelee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you for this fix
@jeejeelee jeejeelee added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 24, 2025
@jeejeelee jeejeelee added this to the v0.10.0 milestone Jul 24, 2025
@jeejeelee jeejeelee enabled auto-merge (squash) July 24, 2025 16:07
@vllm-bot vllm-bot merged commit 2ce90e5 into vllm-project:main Jul 25, 2025
80 of 82 checks passed
liuyumoye pushed a commit to liuyumoye/vllm that referenced this pull request Jul 31, 2025
@zRzRzRzRzRzRzR
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
9f84dce 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
x22x22 pushed a commit to x22x22/vllm that referenced this pull request Aug 5, 2025
@zRzRzRzRzRzRzR @x22x22
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
82d366e 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com> Signed-off-by: x22x22 <wadeking@qq.com>
Pradyun92 pushed a commit to Pradyun92/vllm that referenced this pull request Aug 6, 2025
@zRzRzRzRzRzRzR
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
a045b3f 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025
@zRzRzRzRzRzRzR @npanpaliya
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
6d1cbb8 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
wenbinc-Bin pushed a commit to wenbinc-Bin/vllm-fork that referenced this pull request Aug 7, 2025
@zRzRzRzRzRzRzR @wenbinc-Bin
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
fcd88db 
Cherry-pick: vllm-project@2ce90e5 Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025
@zRzRzRzRzRzRzR @jinzhen-lin
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
9f9b433 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com> Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
@zRzRzRzRzRzRzR @paulpak58
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
f7bfaa9 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com> Signed-off-by: Paul Pak <paulpak58@gmail.com>
wenbinc-Bin pushed a commit to wenbinc-Bin/vllm-fork that referenced this pull request Aug 14, 2025
@zRzRzRzRzRzRzR @wenbinc-Bin
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
abe4ade 
Cherry-pick: vllm-project@2ce90e5 Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
@zRzRzRzRzRzRzR @diegocastanibm
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
ae34ee6 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com> Signed-off-by: Diego-Castan <diego.castan@ibm.com>
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
@zRzRzRzRzRzRzR @epwalsh
Fix GLM-4 PP Missing Layer When using with PP. (vllm-project#21531)
5129407 
Signed-off-by: zRzRzRzRzRzRzR <2448370773@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

force-merge ready ONLY add when PR is ready to merge/full CI is needed

3 participants

@zRzRzRzRzRzRzR @jeejeelee @vllm-bot