Running Promises in Series with a delay between each to bypass API rate limit

I wanted to call an endpoint updating an entity and I wanted to do it for 500 entities.
The API server had “rate limit” and I couldn’t just use Promise.all since it will call the endpoint in parallel and the rate-limit would block my calls.

So I decid…


This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Farmin Farzin

I wanted to call an endpoint updating an entity and I wanted to do it for 500 entities.
The API server had "rate limit" and I couldn't just use Promise.all since it will call the endpoint in parallel and the rate-limit would block my calls.

So I decided to write a script to bulk call the endpoint in series and have a 2 seconds wait before each call to make sure API server would not block them.

As I did that and it worked great, I thought it'd be advantageous to share!
Is there a better way for doing this?

How I'm doing this is starting from 0 and then adding up to that based on the array of data that I want to update. In the chain of promises that I have, I just added a delay before the API call, every time that I add 1 to the value that I'm passing to update the data till that number would be equal to my array length.

import fetch from "node-fetch";
import { data } from "./data.js";

const bearerToken = ""

const updateData = (id) => {
  return fetch(
    `https://www.url/${id}`,
    {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${bearerToken}`,
      },
    }
  );
};

const delay = () => {
  console.log(`Waiting: 2 seconds.`);
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve("2");
    }, 2000);
  });
};

const startTime = Date.now();

const doNextPromise = async (id) => {
  delay()
    .then((x) => {
      console.log(
        `Waited: ${x} seconds now calling the endpoint for updating data ${data[id]}`
      );
      return updateData(data[id])
        .then((res) => {
          if (res.status !== 200) {
            throw `Error updating data ${data[id]}: ${res.status}`;
          }
          return res.json();
        })
        .then((res) =>
          console.log(`Response: ${JSON.stringify(res)} for data ${data[id]}`)
        )
        .catch((e) => console.log(`Error: ${e}`));
    })
    .then((res) => {
      id++;
      if (id < data.length) doNextPromise(id);
      else console.log(`Total: ${(Date.now() - startTime) / 1000} seconds.`);
    });
};
doNextPromise(0);


This content originally appeared on DEV Community 👩‍💻👨‍💻 and was authored by Farmin Farzin


Print Share Comment Cite Upload Translate Updates
APA

Farmin Farzin | Sciencx (2022-09-08T02:37:42+00:00) Running Promises in Series with a delay between each to bypass API rate limit. Retrieved from https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/

MLA
" » Running Promises in Series with a delay between each to bypass API rate limit." Farmin Farzin | Sciencx - Thursday September 8, 2022, https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/
HARVARD
Farmin Farzin | Sciencx Thursday September 8, 2022 » Running Promises in Series with a delay between each to bypass API rate limit., viewed ,<https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/>
VANCOUVER
Farmin Farzin | Sciencx - » Running Promises in Series with a delay between each to bypass API rate limit. [Internet]. [Accessed ]. Available from: https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/
CHICAGO
" » Running Promises in Series with a delay between each to bypass API rate limit." Farmin Farzin | Sciencx - Accessed . https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/
IEEE
" » Running Promises in Series with a delay between each to bypass API rate limit." Farmin Farzin | Sciencx [Online]. Available: https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/. [Accessed: ]
rf:citation
» Running Promises in Series with a delay between each to bypass API rate limit | Farmin Farzin | Sciencx | https://www.scien.cx/2022/09/08/running-promises-in-series-with-a-delay-between-each-to-bypass-api-rate-limit/ |

Please log in to upload a file.




There are no updates yet.
Click the Upload button above to add an update.

You must be logged in to translate posts. Please log in or register.