Fix changes the behavior from overriding kv_connector_extra_config to updating

[akakakakakaa]

When using pooling models together with LMCache, the system crashes due to a missing guard in the LMCache implementation: it does not check whether sampling_params is None. Pooling models do not use sampling parameters, so this leads to a runtime error. (LMCache implementation, vLLM native implementation (maybe works correctly))

To work around this, I attempted to use vLLM’s native LMCache implementation, which already handles this case correctly. However, there is a configuration issue that prevents enabling the native implementation because of overriding.

So, the fix changes the behavior from overriding kv_connector_extra_config to updating it, ensuring that user-provided options such as use_native are preserved.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[gemini-code-assist]

Code Review

This pull request correctly addresses a configuration override issue for the LMCache connector. The change modifies the population of kv_connector_extra_config from a direct assignment to an update, which prevents user-provided settings from being discarded. This is a necessary bug fix that enables users to customize the LMCache connector's behavior, such as enabling the native implementation. The implementation is clean, correct, and consistent with how other backends are configured in the same file. I approve this change.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[mergify]

Hi @akakakakakaa, the pre-commit checks have failed. Please run:

uv pip install pre-commit pre-commit install pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed) pre-commit run --hook-stage manual mypy-3.10 # For markdownlint pre-commit run --hook-stage manual markdownlint

[yewentao256]

LGTM, thanks for the work!

[yewentao256]

Please merge from main and fix the DCO issue

[mergify]

Documentation preview: https://vllm--30856.org.readthedocs.build/en/30856/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @akakakakakaa.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[akakakakakaa]

Sorry for my mistake. I'll fix it