javascript - Solving async producer-consumer problem in JS - Stack Overflow

admin管理员组

文章数量:1344954

I have a producer of data, and a consumer of data. The producer produces asynchronously, and in turn I would like the consumer to consume asynchronously when there is data to consume.

My immediate thought to solve this problem is to use some queue object that has an awaitable shift/get, much like this async queue in the python standard

However, I searched and I couldn't find any JS libraries that have this type of data structure for me to use. I would have thought this would be a mon pattern.

What is the mon pattern for solving this problem in JS, and are there any libraries to help?

I have a producer of data, and a consumer of data. The producer produces asynchronously, and in turn I would like the consumer to consume asynchronously when there is data to consume.

My immediate thought to solve this problem is to use some queue object that has an awaitable shift/get, much like this async queue in the python standard

However, I searched and I couldn't find any JS libraries that have this type of data structure for me to use. I would have thought this would be a mon pattern.

What is the mon pattern for solving this problem in JS, and are there any libraries to help?

• javascript
• node.js
• ecmascript-6

Share Follow asked Jun 27, 2019 at 4:59 ThomasThomas 6,19688 gold badges4747 silver badges8282 bronze badges 1

• 1 jfriend's EventEmitter suggestion seems like it's probably the way to go in your case, but you may also want to read up on Observables. – JLRishe Commented Jun 27, 2019 at 5:17

Add a ment  | 

3 Answers 3

Sorted by: Reset to default 5

If the producer of the data is just spontaneously producing data and the consumer just wants to know when there's some new data, then this sounds like the consumer should just subscribe to an event that will be triggered any time there is new data. You can just use the EventEmitter object in node.js to create an emitter that the consumer can listen to and the producer will trigger and event whenever there's new data. No external library is needed to implement this as the built-in EventEmitter object has all the tools you need to register for notifications and trigger notifications.

If the consumer of the data requests data and the producer then goes and gets it asynchronously, then this is just a typical asynchronous API. The API should probably return a promise and the producer will resolve the promise with the new data when it's ready or reject it if there was an error retrieving the data.

With the little bit of description you've provided, I don't see any particular need for an elaborate queuing system. It just sounds like publish/subscribe or a simple event notification system. If the problem is more plicated, then please give us more details on the producer of the data so we can better match the tools available in node.js to the needs of your particular problem.

In case of small simple program, I would just simply write something like this.

var data = [];

function Consumer()
{
    this.isConsuming = false;

    this.notify = function(){
        if(this.isConsuming)
        {
            return;
        }
        this.consumeNext();
    }

    this.consumeNext = async function(){
        this.isConsuming = true;
        if(data.length > 0)
        {
            //consume one datum
            console.log(await this.consume(data.shift()));

            //consume next datum
            this.consumeNext();
        }
        else
        {
            this.isConsuming = false;
        }
    }

    this.consume = async function(datum){
        return datum * datum;
    }
}



var consumer = new Consumer();
//call consumer.notify() when your producer produces
data.push(1,2,3,4,5);
consumer.notify();

This will give you another idea. In my scenario the producer creates data every 1000 milliseconds and consumer waits until producer created new data and resolved its promise.

let dataArray = []
let consumerResolver = null

function producer() {
    setInterval(() => {
        const newData = "my new Data"
        dataArray.push(newData)

        if (consumerResolver) {
            consumerResolver()
        }
    }, 1000);
}

async function consumer() {
    while (true) {
        if (dataArray.length === 0) {
            const producerPromise = new Promise((resolve) => {
                consumerResolver = resolve
            })
            await producerPromise
        }

        consumerResolver = null
        const data = dataArray.shift()
        console.log(data)
    }
}

本文标签： javascriptSolving async producerconsumer problem in JSStack Overflow