When I ran the test suite for an AWS-based application I noticed that my home internet went down and the tests couldn't finish. I could reproduce the problem every time, so there was a clear causation link between the two.
But how could running tests bring down the internet?
There was a function that roughly looked like this:
const fn = async () => {
await new LambdaClient().send(/*...*/);
}
And this was called many times during the test runs, so the LambdaClient was instantiated many times. And every time the SDK sends a command to the Lambda service it needs to connect to an endpoint.
When I ran the tests, I noticed a lot of outgoing connections. Then I checked the DNS records for one of the Lambda endpoint (https://dnschecker.org/#A/lambda.us-west-1.amazonaws.com) and it returns a lot of different IP addresses.
My hypothesis is that creating (and maintaining) too many connections overloads the router and it restarts. Maybe it's related to NAT port mapping.
The solution was simple: instantiate the client once so that it connects to only one endpoint:
const lambdaClient = new LambdaClient();
const fn = async () => {
await lambdaClient.send(/*...*/);
}
This is not the first time I got bitten by using multiple instances of an AWS SDK client. When using SSO, the SDK calls an AWS API that returns the credentials that will be used for signing requests. Instantiating the service multiple times will fetch credentials multiple times and that hit rate limits.
So a best practice is to save the SDK clients and reuse them outside of any function call.
Originally published at advancedweb.hu
