Hi everyone,
I ran into an issue with the nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:latest container where the startup process would get stuck.
The Problem
The inference.py script loops indefinitely, showing the message: INFO:inference:Waiting for Riva gRPC Server to be READY…
Based on the logs, the Riva gRPC server itself actually finishes starting (it logs Riva Conversational AI Server listening on 0.0.0.0:50051), but the grpc_health_probe check inside the while True: loop in main() never seems to succeed (it doesn’t return 0).
The Solution (Workaround)
To fix this, I modified the inference.py script. I removed the entire while True: health check loop and replaced it with a simple time.sleep() to wait for the server to initialize.
This workaround skips the failing grpc_health_probe and just waits for a fixed amount of time (I used 60 seconds to be safe) before proceeding to start the HTTP interface.
Original Code Snippet (in main()):
while True: logger.info("Waiting for Riva gRPC Server to be READY...") time.sleep(1) result = subprocess.run( ["grpc_health_probe", "-addr", f"0.0.0.0:{nim_grpc_api_port}"] + grpc_probe_tls_args.split(), capture_output=True, ) if result.returncode == 0: logger.info("Riva gRPC Server is READY") break if riva_process.poll() is not None: logger.error("Riva gRPC Server failed to start") return Modified Code (Replaced the loop above with this):
# Wait 120 seconds for Riva gRPC Server to fully initialize logger.info("Waiting 120 seconds for Riva gRPC Server to fully initialize...") time.sleep(120) # Check if the server process failed during the wait if riva_process.poll() is not None: logger.error("Riva gRPC Server failed to start during wait time.") return else: logger.info("Riva gRPC Server is presumed READY. Proceeding to start HTTP server.") After mounting this modified my_inference.py file into the container (using -v ./my_inference.py:/opt/nim/inference.py), the server started up correctly.
Hope this helps anyone else facing the same issue!