- Notifications
You must be signed in to change notification settings - Fork 418
Description
Hi,
I was struggling to find a good title, so let me explain with a minimal working example:
from fastapi import FastAPI from fastapi.responses import RedirectResponse from fastrtc import ReplyOnPause, Stream import gradio as gr import numpy as np def echo(audio: tuple[int, np.ndarray]): yield audio stream = Stream( handler=ReplyOnPause(echo), modality="audio", mode="send-receive", ) app = FastAPI() stream.mount(app) @app.get("/") async def _(): return RedirectResponse(url="/ui") @app.get("/connections") async def _(): return list(stream.connections.keys()) app = gr.mount_gradio_app(app, stream.ui, path="/ui") import uvicorn uvicorn.run(app, host="0.0.0.0", port=7860) Here we set up a dummy stream and add basic API endpoints, in particular /connections which should return all WebRTC connections. The Streams UI components are mounted to /ui and going from root redirects you here.
Now, the problem is that if I connect through the gradio interface, and then call my /connections endpoint, it will return empty. This design isn't great.
Why is this happening?
I had to dig a little and realized it is because the WebRTC component is itself a Stream object and accessible through stream.webrtc_component. This means that I can modify the code above to
@app.get("/connections") async def _(): all_connections = list(stream.connections.keys()) if stream.webrtc_component: all_connections.extend(list(stream.webrtc_component.connections.keys())) return all_connections which does indeed return all webrtc_ids regardless of how they were created (directly from an API call or through the Gradio UI).
Why this is a problem?
This is of course a simple issue with a simple fix, but it is hard to understand why it happens. Furthermore, since most API calls rely on webrtc_ids, e.g., stream.stream_outputs(webrtc_id) that is called in many demo applications with additional_outputs configured, this type of approach would have to be implemented for all endpoints. It will just result in boilerplate code that is bloating your code.
Possible solutions
Some sort of way to handle this internally in the stream class and expose the WebRTC streams of the stream.webrtc_component directly.
I could prepare an MR for this, should you agree with the above. Or perhaps this is not the intended way of usage? I do like it myself...