bentoml / BentoML Star 8.3k Code Issues Pull requests Discussions The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more! python machine-learning deep-learning model-serving multimodal mlops ml-engineering ai-inference llm generative-ai llmops llm-serving model-inference-service llm-inference inference-platform Updated Dec 18, 2025 Python
InftyAI / llmaz Star 278 Code Issues Pull requests Discussions ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work! kubernetes inference huggingface llm modelscope llamacpp vllm text-generation-inference ollama sglang inference-platform Updated Dec 15, 2025 Go
TheStableFoundation / stbl-protocol Star 0 Code Issues Pull requests apa hayo... artificial-intelligence inference-platform web333 Updated Nov 14, 2025 Rust