Async inference in Ray Serve runs long-running model calls off the request path, backed by a message queue, with automatic retries and queue-depth autoscaling. The post is a practical follow-up that builds a real service on top of that feature: a video-indexing pipeline, run under heavy load and compared against a common managed alternative.
