javascript - How to map over async generators? - Stack Overflow

admin管理员组

文章数量:1340292

Let's say we have an async generator:

exports.asyncGen = async function* (items) {
  for (const item of items) {
    const result = await someAsyncFunc(item)
    yield result;
  }
}

is it possible to map over this generator? Essentially I want to do this:

const { asyncGen } = require('./asyncGen.js')

exports.process = async function (items) {
  return asyncGen(items).map(item => {
    //... do something
  })
}

As of now .map fails to recognize async iterator.

The alternative is to use for await ... of but that's nowhere near elegant as with .map

Let's say we have an async generator:

exports.asyncGen = async function* (items) {
  for (const item of items) {
    const result = await someAsyncFunc(item)
    yield result;
  }
}

is it possible to map over this generator? Essentially I want to do this:

const { asyncGen } = require('./asyncGen.js')

exports.process = async function (items) {
  return asyncGen(items).map(item => {
    //... do something
  })
}

As of now .map fails to recognize async iterator.

The alternative is to use for await ... of but that's nowhere near elegant as with .map

• javascript
• node.js
• iterator
• map-function
• async-iterator

Share Improve this question Follow edited Jun 1, 2021 at 19:33 Bergi 666k161161 gold badges1k1k silver badges1.5k1.5k bronze badges asked Jun 1, 2021 at 19:15 Dmitry KankalovichDmitry Kankalovich 62322 gold badges99 silver badges2222 bronze badges 1

• 1 .map() exists only on arrays, not on generators - async or not. – VLAZ Commented Jun 1, 2021 at 19:17

Add a ment  | 

3 Answers 3

Sorted by: Reset to default 8

The iterator methods proposal that would provide this method is still at stage 2 only. You can use some polyfill, or write your own map helper function though:

async function* map(asyncIterable, callback) {
    let i = 0;
    for await (const val of asyncIterable)
        yield callback(val, i++);
}

exports.process = function(items) {
    return map(asyncGen(items), item => {
       //... do something
    });
};

TL;DR - If the mapping function is async:

To make asyncIter not wait for each mapping before producing the next value, do

async function asyncIterMap(asyncIter, asyncFunc) {
    const promises = [];
    for await (const value of asyncIter) {
        promises.push(asyncFunc(value))
    }
    return await Promise.all(promises)
}

// example - how to use:
const results = await asyncIterMap(myAsyncIter(), async (str) => {
    await sleep(3000)
    return str.toUpperCase()
});

More Demoing:

// dummy asyncIter for demonstration

const sleep = (ms) => new Promise(res => setTimeout(res, ms))

async function* myAsyncIter() {
    await sleep(1000)
    yield 'first thing'
    await sleep(1000)
    yield 'second thing'
    await sleep(1000)
    yield 'third thing'
}

Then

// THIS IS BAD! our asyncIter waits for each mapping.

for await (const thing of myAsyncIter()) {
    console.log('starting with', thing)
    await sleep(3000)
    console.log('finished with', thing)
}

// total run time: ~12 seconds

Better version:

// this is better.

const promises = [];

for await (const thing of myAsyncIter()) {
    const task = async () => {
        console.log('starting with', thing)
        await sleep(3000)
        console.log('finished with', thing)
    };
    promises.push(task())
}

await Promise.all(promises)

// total run time: ~6 seconds

The alternative is to use for await ... of, but that's nowhere near elegant as with .map

For an elegant and efficient solution, here's one using iter-ops library:

import {pipe, map} from 'iter-ops';

const i = pipe(
    asyncGen(), // your async generator result
    map(value => /*map logic*/)
); //=> AsyncIterable

• It is elegant, because the syntax is clean, simple, and applicable to any iterable or iterators, not just asynchronous generators.
• It is more flexible and reusable, as you can add lots of other operators to the same pipeline.

Since it produces a standard JavaScript AsyncIterable, you can do:

for await(const a of i) {
    console.log(a); //=> print values
}

---

P.S. I'm the author of iter-ops.

本文标签： javascriptHow to map over async generatorsStack Overflow