Amazon Web Services · Cross-Region Inference Routing

Cross-Region inference routing in Amazon SageMaker AI

Amazon SageMaker AI now offers cross-Region inference routing through a new inference router resource, giving applications one invocation path across real-time endpoints, inference components and asynchronous endpoints in one or more Regions and accounts. The router keeps requests in its local Region while that destination is healthy, shifts new requests to other registered destinations based on a health score, and retries a real-time request once on another destination only when the model did not run. The router does not create capacity, scale endpoints or copy model artefacts, so each destination must already be running in its Region.

Open the source ↗

Similar Developer Tools releases

More from Amazon Web Services

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.