A service by AWS that routes traffic based on simple logic Background
we currently use google distance matrix (GDM) apis a lot to help in core business functions. As the cost of GDM went up, we worked on switching on cheaper alternatives. We decided to use OpenStreetMap (OSM) as an alternative. Now we have OSM hosted on aws Elastic Service Container (ECS).
Problem
Sometimes (for reasons beyond the scope of this question) the OSM service fails or times out. We put logic in the application code that if it fails (or times out, with a timeout being 1s), then switch back to GDM as a backup. The problem is that if there is a sudden burst of requests, using the ECS architecture these requests will stack up, and so if we have 20,000 requests that's a total potential lag of 20,000 seconds, and the the system crashes.
One way of solving this problem is putting in the application code logic that keeps track of the number of OSM timeouts TOs per time T(persisting that number into redis or something) and once it reaches past a certain number, we will switch back to GDM, and after Z time elapsing, we want to try using OSM again (the idea is that the server would be operational again).
Question
Is it possible to do the above in an AWS Service that is specialized for such routing problems? I was thinking something like AWS Gateway but I couldn't find a place where routing logic can be put there.
You can use AWS APP MESH to route traffic between OSM and GDM. First, all traffic will be sent to OSM using App Mesh's VirtualRouter.
VirtualRouter is an object to route traffic between K8S services. One service will be pointed to the ECS hosted OSM. Another service will be pointed to GDM using External Service
You can use a Cronjob which will run in a pod and sense timeout-related metrics from ElasticCache. If the condition matched - the code host in the pod and running as a CronJob will modify VirtualRouter.