[Serve][LLM] Add Pause/Resume Control Plane API for Ray Serve LLM

[kouroshHakha]

Summary

This PR adds pause/resume control plane endpoints to DevIngress, enabling RL training workflows to temporarily halt generation without offloading model weights from GPU memory.

Must merge before this PR:

• [Serve][LLM] Refactor Control Plane Endpoints into Mixins #59502

Motivation

During RL training, there's often a need to pause inference for weight updates. While sleep/wakeup exists for memory management (offloads weights to CPU), pause/resume provides a lightweight alternative that:

• Halts generation immediately
• Keeps model weights in GPU memory
• Enables faster resume since no weight reloading is needed

Changes

1. New Control Plane Endpoints

Added three new HTTP endpoints via PausableIngressMixin:

• POST /pause - Pause generation on all replicas
• POST /resume - Resume generation after pause
• GET /is_paused - Check pause status

2. vLLM Integration

Integrated with vLLM's pause_generation() / resume_generation() / is_paused() API with configurable options:

• wait_for_inflight_requests: Wait for in-flight requests to complete vs abort immediately
• clear_cache: Whether to clear KV cache on pause

Testing

• Unit tests for pause/resume cycle and model isolation
• End-to-end release test (test_llm_serve_pause_resume.py)

[gemini-code-assist]

Code Review

This pull request introduces a pause/resume control plane API, which is a valuable addition for RL training workflows. The refactoring of existing control plane endpoints into modular mixins (SleepableIngressMixin, PausableIngressMixin, CacheManagerIngressMixin) is a significant improvement, making the code cleaner and more extensible. The test coverage, including unit, isolation, and a new end-to-end release test, is thorough and well-executed. I've found one high-severity issue related to a blocking call in an async context that could impact server performance, which I've detailed in a specific comment. Overall, this is a high-quality contribution.

[kouroshHakha]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a pause/resume control plane API for Ray Serve LLM, which is a valuable addition for workflows like reinforcement learning training. The implementation is well-structured, following existing patterns for control plane endpoints. The changes are spread across the ingress, server, and engine layers, and are accompanied by both unit tests and a new end-to-end release test.

My review focuses on improving the robustness of the new release test by replacing fixed-time sleeps with polling, which will help prevent flakiness and unnecessary delays. The core logic of the feature itself appears solid.

[eicherseiji]

LGTM

[eicherseiji]

Some debug logs about the results could be helpful?

[kouroshHakha]

discussed offline. I think we can add it if we need to debug. I am not sure how to tune the verbosity in a nice way.