Skip to content

Conversation

phantomlei3
Copy link
Contributor

@phantomlei3 phantomlei3 commented Aug 4, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Fix a random bug from tests/v1/sample/test_sampler.py::test_sampler_bad_words when batch size is 32 and bad words lengths is (2, 2).
There are two corner cases that would trigger this unit test to fail but the input is random generated so this error is diffcult to replicate during CI test in vLLM.

Corner Case 1:

When two randomly generated token ID sequences share the same first token ID — for example, [[522, 2], [522, 3]] — only [522, 2] will be considered in the bad words filtering process. [522, 3] will be ignored, leading to incomplete coverage.

Corner Case 2:

When the first token IDs of two randomly generated token ID sequences differ by only 1 — for example, [[524, 2], [523, 3]] — and the has_bad_words check happens to return False for both, the last token in the output may incorrectly be replaced with 524 (a bad word), causing an error.

Test Plan

100 runs of the following command and make sure all passes

python3 -m pytest -s -v -p no:faulthandler /vllm/tests/v1/sample/test_sampler.py -k test_sampler_bad_words 

Test Result

all passed

================================================================ 18 passed, 54 deselected, 2 warnings in 6.60s ================================================================ 

(Optional) Documentation Update

@mergify mergify bot added the v1 label Aug 4, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to fix two corner cases in the bad word testing logic by introducing new helper functions. While the approach is sound, the implementation contains critical bugs due to variable shadowing. The new helper functions are called with arguments of the wrong type, which will lead to runtime errors. I have provided comments with suggestions to fix these issues. Additionally, I've pointed out a maintainability issue in one of the new helper functions, also related to variable shadowing, and suggested an improvement.

Copy link

github-actions bot commented Aug 4, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@phantomlei3 phantomlei3 force-pushed the fix-random-bad-words branch 5 times, most recently from 4dbea9e to 4d051be Compare August 5, 2025 02:33
Signed-off-by: phantomlei <phantomlei3@gmail.com>
@phantomlei3 phantomlei3 force-pushed the fix-random-bad-words branch from 4d051be to eb0208b Compare August 5, 2025 06:42
@phantomlei3
Copy link
Contributor Author

cc @22quinn request for review based on your implementation on #13376

Copy link
Collaborator

@22quinn 22quinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching these subtle edge cases! LGTM, requested committers for stamping/review.

@22quinn 22quinn added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 8, 2025
@simon-mo simon-mo enabled auto-merge (squash) August 12, 2025 05:30
@vllm-bot vllm-bot merged commit bc8372e into vllm-project:main Aug 12, 2025
26 of 28 checks passed
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
…llm-project#22170) Signed-off-by: phantomlei <phantomlei3@gmail.com> Signed-off-by: Paul Pak <paulpak58@gmail.com>
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
…llm-project#22170) Signed-off-by: phantomlei <phantomlei3@gmail.com> Signed-off-by: Diego-Castan <diego.castan@ibm.com>
yiliu30 pushed a commit to yiliu30/vllm-fork that referenced this pull request Aug 19, 2025
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025
…llm-project#22170) Signed-off-by: phantomlei <phantomlei3@gmail.com> Signed-off-by: Xiao Yu <xiao.yu@amd.com>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed v1

4 participants