API Response Error 429: Too Many Requests
How to manage the API rate limit.
Like most APIs, the Foundry API has a rate limit to prevent an excessive number of API requests from impacting performance and reliability, and to guard against cyber attacks. This page outlines some strategies and recommendations for how to handle the rate limit and make the most of your API Integration.
What is the rate limit?
Foundry has a rate limit of 200 requests per rolling 60 seconds. Once the limit is exceeded, Foundry responds to a request with a 429 http status code in the response. The request count is summed up across all the API accounts within an organization. So if you have three different API accounts for your organization, all three of them contribute to the same request count.
When you exceed the rate Limit
First, be aware that hitting a rate limit is not a failure in your integration. When you do hit the rate limit, your integration needs to respond appropriately with an attempt to get under the rate limit and to retry the request that was rejected. If your integration produces too many 429 requests, then Foundry may be forced to disable the API credentials as a defense against a potential denial of service attack or an API integration that has fallen into an unexpected endless loop.
If you find that you chronically exceed the rate limit, then consider implementing a throttling mechanism in your code. Find a way to keep track of your requests in the past 60 seconds, and as you get close to the limit, sleep threads to slow down your code.
Avoid unnecessary requests
Avoid making the same request over and over, for example to get the list of information that rarely changes such as custom categories or locations. Try to use one or more GET requests to retrieve all the information you need in a smaller number of requests, instead of getting your data one request at a time.
Use bulk operations
If you’re setting a learner’s categories one by one, consider using the bulk label operation at admin/bulk_actions/category instead to vastly decrease the number of requests you need to send.
Implement retries with exponential backoff
Even if you optimize all your requests, you can still hit the rate limit, especially if you’re adding a large number of new learners or mass updating existing learners, which requires one request per user.
When an API request receives a 429 response, this request is not processed. Any requests with 429 responses are always safe to retry. Here’s a selection of documentation from major cloud providers discussing their best practices for retries with exponential backoff.
Backoff is where you pause the thread for a short time (like 1 second) when you get a 429, then try again. If you still get a 429 then pause 2 seconds and retry. If you still get a 429, then double the pause time to 4 seconds, and so on. The links below go into more specific details:
- Google – Implementing exponential backoff, explained in a succinct and easy-to-implement way: https://cloud.google.com/iot/docs/how-tos/exponential-backoff
- Amazon – Error Retries and Exponential Backoff in AWS: https://docs.aws.amazon.com/general/latest/gr/api-retries.html
- Microsoft – Implement retries with exponential backoff: https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/implement-retries-exponential-backoff
Contact us for help
If you have implemented these best practices, but continue to receive 429 responses, please contact our Support team with your relevant use case information.