Discussions

• 1 You must be logged in to vote

  💬

  Routing request: H100 “OOM wall” at N=500k + indexed O(N) retrieval (seeking owner for NDA evaluation)

  StanByriukov02 started Dec 18, 2025 in General

  

  

  0
• 1 You must be logged in to vote

  💡

  Support for Jetson Orin (NX or Nano)

  WilliamTambellini started Dec 16, 2025 in Ideas

  

  

  

  

  3
• 0 You must be logged in to vote

  🙏

  When trying to use json_schema in response_format, wrapper field(schema) cannot be used.

  khe0613 asked Dec 15, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Layerwise benchmarks

  youki-sada asked Dec 11, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  💬

  Support for Qwen3-VL 2B/4B?

  JimikoSK started Nov 3, 2025 in General

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  What each type of buffer is doing?

  alxmamaev asked Sep 14, 2025 in Q&A · Answered

  

  

  7
• 1 You must be logged in to vote

  🙏

  Does the TensorRT-LLM Triton backend support BERT model inference in Triton Inference Server?

  unix1986 asked Dec 1, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💡

  Support for ASR Omnilingual model from Meta

  protonicage started Nov 25, 2025 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  Is there any way to capture the executorWorker's errors?

  matiaslin asked Nov 19, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Do we really need tensorrt dependency?

  RobertLou asked Oct 29, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  💬

  Offering Refactoring Expertise for Multimodal Bugs (#8652) – 20+ Yrs Exp in AI/ML

  pythonjavaerlang started Oct 24, 2025 in General

  

  

  0
• 8 You must be logged in to vote

  📣

  📢 Announcing the TensorRT LLM Software Architecture Roadmap (August 2025 Update)

  QiJune announced Sep 18, 2025 in Announcements

  

  

  1
• 1 You must be logged in to vote

  💬

  reduce docker image size

  geraldstanje1 started Oct 15, 2025 in General

  

  

  0
• 2 You must be logged in to vote

  🙌

  RTX 5090 + FP4 + Open WebUI via TensorRT-LLM (because VLLM made me cry at 2am)

  rdumasia303 started Oct 13, 2025 in Show and tell

  

  

  0
• 2 You must be logged in to vote

  🙏

  qwen omni

  piotr-frankowski-tcl asked May 28, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  RTX 6000 PRO sm120 and gpt-oss

  voipmonitor asked Sep 24, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Does TensorrtLLM support v1/response api?

  sowmen asked Aug 28, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  About diverse beam search implementation

  shenlanyilang asked Aug 22, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Where can I download DeepSeek-V3-Lite?

  wangdongxuking61 asked Aug 20, 2025 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  failed gemma3 convert checkpoint

  BayRanger asked Aug 14, 2025 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  About UB strategy

  Neveryoun asked Aug 18, 2025 in Q&A · Unanswered

  

  

  0
• 2 You must be logged in to vote

  📣

  v0.21.0

  QiJune announced Aug 4, 2025 in Announcements

  

  

  0
• 1 You must be logged in to vote

  📣

  v1.0.0rc5

  QiJune announced Aug 4, 2025 in Announcements

  

  

  0
• 1 You must be logged in to vote

  📣

  v1.0.0rc4

  QiJune announced Jul 22, 2025 in Announcements

  

  

  0
• 1 You must be logged in to vote

  📣

  v1.0.0rc3

  QiJune announced Jul 16, 2025 in Announcements

  

  

  0