Retry and reliability practices
Networks fail and services get busy. This guide shows how to read the retry hint in errors, how to back off, and how to retry a write without doing it twice: by checking state before you resend, and optionally by sending an idempotency key.
The goal is simple to state: every call you intend to place is placed at most once, and you always find out whether it was. Placing calls and starting campaigns cost money in live mode and reach real people, so "probably did not happen" is never good enough.
Before you begin
Read Error handling. Idempotency explains the optional Idempotency-Key header used in one section below.
Use the retry hint
Every error carries error.retry, which tells you what to do without a table of codes:
retry | Meaning | What to do |
|---|---|---|
no | The request cannot succeed as sent | Do not retry. Fix the request, the configuration, or accept the refusal (for example dnc_listed). |
yes | A transient failure | Retry with exponential backoff and jitter, after checking state for a write that may have happened |
after | Wait first | Retry after the number of seconds in the Retry-After header |
Each code's page in the error-code reference also says whether work may have started before the error (its "execution" fact). For example dnc_listed is "not started": nothing happened, and sending the request again after fixing the cause is safe. internal_error (500) is "may have started": for a write, the API does not know whether the change was applied. Never assume a failed request had no side effects unless its code says so.
Every request runs
The API does not require an idempotency key. Without one, every request you send is executed. If a POST /calls times out after the API accepted it and you simply send it again, you place a second call. So the rule for any write whose outcome you did not see:
- Put your own id in the request where you can find it again:
metadataon a call (for example your record id and the attempt), the phone number on a contact, the campaignname. - After a timeout, a dropped connection or a
5xx, check the state first: read or list the resource and look for your id. - Send the request again only if it did not happen.
Errors that say nothing happened (retry: "no" refusals before any work, 429 rate_limited, 429 calling_capacity_reached, 503 service_unavailable) can be resent without a check.
A call that can be retried safely
The whole rule for POST/calls, as a sequence of requests. The complete runnable example below implements it with backoff, jitter and the retry hint.
1. Place the call with your own id in metadata:
2. If it times out, fails to connect or answers 5xx, look for the call before sending again: list calls to that number since just before your first attempt, and match your metadata:
If a call with your logical_id is in data, it was placed: store its id and stop. Calls appear in the list as soon as they are accepted, but allow a few seconds after a timeout before you conclude it is not there. Only if it is not, send the request from step 1 again.
3. If it answers 429 or 503, nothing was done: wait Retry-After and send it again.
Complete runnable example
Place a call so that retries can never place it twice.
The same steps as a complete script, with error handling, tested against test mode on every docs build. Each script reads your test key from an environment variable named in its first lines.
- cURL script (bash)
create-call-reliably.sh - JavaScript (Node.js 20+)
create-call-reliably.mjs - Python 3.9+ (requests)
create-call-reliably.py
The same pattern applies to campaign starts and resumes (read the campaign: running means it started), contact upserts (repeating one is harmless) and every other write.
Optional: an idempotency key
For extra safety you can send an Idempotency-Key header (any unique value, a UUID is ideal) on a POST or DELETE, and the same key on every retry of that request. Then a retry of a request that already succeeded returns the original answer with Idempotent-Replayed: true instead of running again, for 24 hours. Create the key once per action and save it before the first attempt, so a retry after a crash reuses it. The rules, including the 409 answers for a key reused with a different request, a first attempt still running, or an interrupted one, are in Idempotency.
A key does not replace the state check after a 500: a 500 is not stored under the key, so a retry with the same key runs the request again.
PATCH requests are protected by If-Match instead (see Concurrency): repeating the same PATCH with a fresh ETag is harmless.
Backoff
- Exponential with jitter. Wait a random time between 0 and
min(cap, base × 2^attempt). Jitter spreads retries from many workers so they do not hit the API in synchronised waves. The complete example uses a base of 1 second and a cap of 30. - Honour
Retry-Afteron429and503(and on any error withretry: "after"); it is the earliest time the request can succeed. - Cap the attempts (for example 5 or 6) and the total time. Then park the action, check its state later, or alert a person.
- Retry only what can succeed. Never retry
retry: "no". Do not retry400,401,403,404or422in a loop.
Rate limits and the daily call cap
Limits are per key and per workspace, separately in test and live mode, in three classes. Each operation's class is in the API reference (x-rate-limit-class):
| Class | Operations | Per key per minute | Per workspace per minute |
|---|---|---|---|
read | Every GET | 600 | 1200 |
write | Other writes (contacts, DNC, campaign create, pause, stop, edits, webhooks) | 120 | 240 |
dial | POST /calls, campaign start and resume | (workspace limit) | 30 |
Every response carries RateLimit-Limit, RateLimit-Remaining and RateLimit-Reset (seconds until the window resets). Over the limit you get rate_limited (429) with Retry-After. Read the headers and slow down before you hit the limit: a batch job should pace itself (see Common integration patterns).
Two more limits concern calls, and neither is a rate limit:
calling_capacity_reached(429): call starts are paced across the telephony network. The request was valid; nothing was dialled. WaitRetry-Afterand send it again, or use a campaign for volume.daily_call_limit_reached(429): live mode caps calls placed through the API per workspace per UTC day, as protection against runaway scripts and retry storms (limits.daily_call_limiton GET/workspace). It resets at 00:00 UTC;Retry-Afteranddetails.resets_atsay when. A campaign start counts every recipient and is refused whole when they do not all fit (details.requested,details.remaining); a resume never counts. If your expected volume is higher, write to support@relationsai.com. Do not "retry until it works".
When the API is unavailable it fails closed with service_unavailable (503): nothing was done; send the request again after Retry-After.
When the outcome is unknown
Some failures leave you not knowing whether the action happened. Treat them as "may have happened" until you have checked.
- Timeout or connection error on a POST: the request may have been processed. Check the state (step 2 above) before sending it again. If you sent an idempotency key, sending the same request with the same key is also safe: a replay tells you it happened.
internal_error(500): the write may have been applied. Read the resource first; send it again only if it did not happen.202with the callqueued: accepted; the start is still being confirmed. Do nothing new. Follow the call: it proceeds, or ends withend_reason: not_confirmed(the start was never confirmed; not charged).202from a campaign start: started, confirmation pending. Read the campaign in a minute; do not start it again.campaign_outcome_unknown(409): a start or resume was sent and not confirmed; dialling may have begun. WaitRetry-After, then read the campaign:runningmeans it started. Only if it did not, send the action again (the campaign refuses to dial the same run twice). If you sent an idempotency key, the same key returns this answer again. Ifdetails.needs_supportis true, contact support first.campaign_pause_not_confirmed(409): the pause may not have taken effect; the campaign may still be dialling. Read the campaign; if it is still running, pause again or stop it.idempotency_outcome_unknown(409): only with an idempotency key: the first attempt with that key was interrupted mid-way. Check whether it happened before sending anything; only if it did not, send it again with a new key or none.
If the state is still ambiguous after a few minutes, write to support@relationsai.com with the request_id.
What can go wrong
- Resending a call after a timeout without checking: a second call. Instead: list calls by
toandcreated_after, match yourmetadata, then decide. - No id of your own in the request: you cannot tell your call from others after a timeout. Instead: put your record id in
metadata. - A new Idempotency-Key per retry (if you use keys): no protection at all. Instead: one key per action, saved before the first attempt.
- The same key for a different request:
idempotency_key_reused. Instead: a new key for each new action. - Retrying
retry: "no"errors: wasted requests, possible rate limiting. Instead: fix the cause or accept the refusal. - Retrying in a tight loop:
rate_limited, thendaily_call_limit_reached. Instead: backoff with jitter and a cap.