Koa service current limiting method example

The requirement of current limiting is to limit the number of simultaneous executions. After this number is exceeded, it must be cached in a queue. This article mainly introduces the practice of Koa service current limiting method. The editor thinks it is quite good. Now I will share it with you and give you a reference. Let’s follow the editor to take a look, I hope it can help everyone.

I recently received a request. It is very simple, just set up a server, call a provided interface when receiving the request, and then return the result. Because of the performance issues of this interface, the number of requests at the same time cannot exceed a certain number, and the current must be limited in the service.

koa middleware does not call next

The original idea is to count in koa middleware, and cache the next function when there are more than 6. When the ongoing task ends, call next to continue other requests.

Later I found that in koa middleware, if the next function request is not executed, it will not stop. Instead, the subsequent middleware will no longer be called and the content will be returned directly.

const Koa = require('koa');
const app = new Koa();
app.use((ctx, next) => {
 console.log('middleware 1');
 setTimeout(() => {
  next();
 }, 3000);
 ctx.body = 'hello';
});
app.use((ctx, next) => {
 console.log('middleware 2');
});
app.listen(8989);

The above code first prints ‘middleware 1’ on the console => The browser receives ‘hello’ => The console prints ‘middleware 2’.

Another thing to note here is that after a request has ended (finish), its next method can still be called, and the subsequent middleware will continue to run (but the modification to ctx will not take effect , because the request has been returned). Similarly, the close request (close) behaves in the same way.

Use await to make the request wait

Delaying the execution of the next function cannot achieve the purpose. The next natural thing to think of is to use await to wait for the current request. The await function returns a Promise. We store the resolve function in this Promise in the queue and delay the call.

const Koa = require('koa');
const app = new Koa();
const queue = [];
app.use(async (ctx, next) => {
 setTimeout(() => {
  queue.shift()();
 }, 3000);
 await delay();
 ctx.body = 'hello';
});
function delay() {
 return new Promise((resolve, reject) => {
  queue.push(resolve);
 });
}
app.listen(8989);

The above code returns a Promise in the delay function, and the resolve function of the Promise is stored in the queue. Set the timer to execute the resolve function in the queue after 3 seconds, so that the request can continue to be executed.

Is the current flow limited for routes or for requests?

After the basic principle of current limiting is implemented, the next question is where should the current limiting code be written? Basically, there are two locations:

Current limiting for interfaces

Due to our needs, current limiting is due to the limited performance of the requested interface. So we can limit the flow of this request separately:

async function requestSomeApi() {
 // 如果已经超过了最大并发
 if (counter > maxAllowedRequest) {
  await delay();
 }
 counter++;
 const result = await request('http://some.api');
 counter--;
 queue.shift()();
 return result;
}

There is also a convenient reusable version below.

async function limitWrapper(func, maxAllowedRequest) {
 const queue = [];
 const counter = 0;
 return async function () {
  if (counter > maxAllowedRequest) {
   await new Promise((resolve, reject) => {
    queue.push(resolve);
   });
  }
  counter++;
  const result = await func();
  counter--;
  queue.shift()();
  return result;
 }
}

Current limiting for routing

This method is to write a koa middleware and limit the current in the middleware:

async function limiter(ctx, next) => {
 // 如果超过了最大并发数目
 if (counter >= maxAllowedRequest) {
  // 如果当前队列中已经过长
  await new Promise((resolve, reject) => {
   queue.push(resolve);
  });
 }
 store.counter++;
 await next();
 store.counter--;
 queue.shift()();
};

After Just use this middleware in the router for different routes:

router.use('/api', rateLimiter);

Comparison

implemented current limiting for the interface. I felt that the logic was a bit confusing, so I used In order to limit the flow of routing, everything works perfectly.

Until I received another request, I had to request this interface three times to return the result array of the three requests. Now the problem comes, we can't call the interface directly because the current is limited. You cannot directly call the function of the request interface because our current limit is based on routing. then what should we do? We can only request this route and request it ourselves. . .

Things to note

Listen to the close event and remove the request from the queue
Requests that have been stored in the queue may be canceled by the user Condition. As mentioned before, even if the request is canceled in koa, subsequent middleware will still run, which means that interfaces that require current limiting will also be executed, causing waste.

You can listen to the close event to achieve this purpose. We need to mark each request with a hash value:

ctx.res.on('close', () => {
 const index = queue.findIndex(item => item.hash === hash);
 if (index > -1) {
  queue.splice(index, 1);
 }
});

Set the timeout

In order to prevent the user from waiting too long For a long time, you need to set a timeout, which is easy to implement in koa:

const server = app.listen(config.port);
server.timeout = DEFAULT_TIMEOUT;

The current queue is too long

If the current queue is too long, even if you join the queue will time out. Therefore, we also need to deal with the situation where the queue is too long:

if (queue.length > maxAllowedRequest) {
 ctx.body = 'error message';
 return;
}

Related recommendations:

The Road of Java Architect--Current Limiting Technology and Various Programming Languages

nginx current limiting algorithm

Use redis for current limiting

The above is the detailed content of Koa service current limiting method example. For more information, please follow other related articles on the PHP Chinese website!