javascript - Running promises in small concurrent batches (no more than X at a time) - Stack Overflow

admin管理员组

文章数量:1277377

The Async library has functions like eachLimit which can be used to efficiently spread a big batch of jobs over multiple CPU cores, like this:

var numCPUs = require('os').cpus().length;
var exec = require('child_process').exec;

async.eachLimit(someArray, numCPUs, function (value, done) {
  exec('something --input' + value, done);
}, finalCallback);

This avoids overloading the system with too many mands at once, but still exploits multiple CPUs.

I want to do the same thing but with Promises.

In the Bluebird API, I can't see any obvious way to do this kind of batching in such a concise, expressive way as with Async.

Is there a good pattern for doing this with Bluebird (or with Promises generally)? Or any other utility library I could use for this?

The Async library has functions like eachLimit which can be used to efficiently spread a big batch of jobs over multiple CPU cores, like this:

var numCPUs = require('os').cpus().length;
var exec = require('child_process').exec;

async.eachLimit(someArray, numCPUs, function (value, done) {
  exec('something --input' + value, done);
}, finalCallback);

This avoids overloading the system with too many mands at once, but still exploits multiple CPUs.

I want to do the same thing but with Promises.

In the Bluebird API, I can't see any obvious way to do this kind of batching in such a concise, expressive way as with Async.

Is there a good pattern for doing this with Bluebird (or with Promises generally)? Or any other utility library I could use for this?

• javascript
• node.js
• asynchronous
• promise
• bluebird

Share Improve this question Follow edited Jun 8, 2015 at 15:13 thefourtheye 240k5353 gold badges465465 silver badges500500 bronze badges asked Jun 8, 2015 at 11:48 callumcallum 37.8k3939 gold badges113113 silver badges175175 bronze badges 1

• It can be done without dependencies too: stackoverflow./questions/37213316/… – Annarfych Commented Nov 28, 2016 at 18:18

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 11

In Bluebird, you can use Promise.map function with the concurrency option, like this

require('bluebird').map([1, 2, 3, 4, 5, 6], function (currentNumber) {
    console.log(currentNumber);
    return Promise.delay(currentNumber * 2, 1000);
}, {concurrency: 2}).then(console.error.bind(console));

Now, you can see that it is processing two values at a time. The value being processed may be in a different order, but the items in the result array will be in the same order corresponding to the original array.

PS: I introduced a delay of 1 second with Promise.delay, so that we can observe what is being processed currently.

---

Demo

function start() {
  document.getElementById("result").innerHTML = "";

  Promise.map([1, 2, 3, 4, 5, 6], function(currentNumber) {
    document.getElementById("result").innerHTML += currentNumber + "<br />";
    return Promise.delay(currentNumber * 2, 1000);
  }, {
    concurrency: parseInt(document.getElementById("concurrency").value, 10)
  }).then(function(result) {
    document.getElementById("result").innerHTML += result + "<br />";
  });
}

<script src="https://cdnjs.cloudflare./ajax/libs/bluebird/2.9.27/bluebird.min.js"></script>
Limit: <input id="concurrency" value="2" />
<input type="button" onclick="start()" value="Start" />
<pre id="result" />

本文标签： javascriptRunning promises in small concurrent batches (no more than X at a time)Stack Overflow