Remove Gunicorn and use Uvicorn only for gateway #11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Summary
This PR removes Gunicorn and replaces it with Uvicorn for the FastAPI server in a Kubernetes environment, aiming to simplify deployment and potentially improve performance.
- Updated
start_fastapi_server.pyto use Uvicorn directly with a high concurrency limit (10000) - Removed
worker.py, eliminating the custom LaunchWorker class for Gunicorn workers - Significantly updated Uvicorn from version 0.17.6 to 0.30.0 in
requirements.inandrequirements.txt - Adjusted server configuration parameters in
start_fastapi_server.pyfor Uvicorn-only setup - Removed Gunicorn-related dependencies from
requirements.txt
5 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings
| def entrypoint(): | ||
| """Entrypoint for starting a local server.""" | ||
| | ||
| # We can probably use asyncio since this service is going to be more I/O bound. |
There was a problem hiding this comment.
style: This comment about asyncio seems outdated now that we're using Uvicorn. Consider removing or updating it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Pull Request Summary
in kubernetes environment we don't really need multiple workers in the same pod, rather it's simpler to just have kubernetes autoscale the number of pods. based on some internal benchmarks gunicorn has some known load balancing issues, also removing this layer results in less error and better latency
Test Plan and Usage Guide
will run simple load testing for get requests with and without gunicorn