Serving & Orchestration
Ray Serve
anyscaleScalable model serving built on Ray with request routing and autoscaling
Editorial compatibility: these components can be used together in representative deployments. That's not a requirement or a validated implementation. For real deployments, see the sourced reference architectures.