javascript - Queueing Promises with P-Queue node module - Stack Overflow

admin管理员组

文章数量:1401176

I'm using node-fetch to make a HEAD request to an Amazon bucket to check the status of a file, but I am trying to set up the module P-Queue to give me control over concurrency.

I have an array, resources, that contain all of my data including file paths to check within that bucket. Then map the array like so:

resources.map((resource) => {
    queue
        .add(validateImage(resource))
        .catch((err) => console.error(`Failed to add resource to queue: ${err}`));
});

And this is what my validateImage function looks like:

const validateImage = (resource) => {
    const path = `https://${bucketUrl}/${env}/${resource.path}`;
    verbose && console.log(`Fetching resource: ${path}`);
    return fetch(
        path,
        {
            method: 'HEAD',
            timeout
        }
    )
    .then(res => {
        validatedResources.push({
            ...resource,
            statusCode: res.status
        });
    })
    .catch(err => {
        validatedResources.push({
            ...resource,
            error: err.type
        });
    });
}

But whenever I run this script I get the error Failed to add resource to queue: TypeError: fn is not a function as many times as I have resources.

I've checked that what is being returned from validateImage is a Promise and I also tried rewriting the example from the p-queue github to use fetch (suspecting that being the reason):

const PQueue = require('p-queue');
const fetch = require('node-fetch');

const queue = new PQueue({concurrency: 1});

queue.add(() => fetch(
    ''),
    { method: 'HEAD'}
).then(() => {
    console.log('Done: sindresorhus');
});

queue.add(() => fetch('')).then(() => {
    console.log('Done: ava.li');
});

This does work, so I'm not really sure where to look anymore?

I'm using node-fetch to make a HEAD request to an Amazon bucket to check the status of a file, but I am trying to set up the module P-Queue to give me control over concurrency.

I have an array, resources, that contain all of my data including file paths to check within that bucket. Then map the array like so:

resources.map((resource) => {
    queue
        .add(validateImage(resource))
        .catch((err) => console.error(`Failed to add resource to queue: ${err}`));
});

And this is what my validateImage function looks like:

const validateImage = (resource) => {
    const path = `https://${bucketUrl}/${env}/${resource.path}`;
    verbose && console.log(`Fetching resource: ${path}`);
    return fetch(
        path,
        {
            method: 'HEAD',
            timeout
        }
    )
    .then(res => {
        validatedResources.push({
            ...resource,
            statusCode: res.status
        });
    })
    .catch(err => {
        validatedResources.push({
            ...resource,
            error: err.type
        });
    });
}

But whenever I run this script I get the error Failed to add resource to queue: TypeError: fn is not a function as many times as I have resources.

I've checked that what is being returned from validateImage is a Promise and I also tried rewriting the example from the p-queue github to use fetch (suspecting that being the reason):

const PQueue = require('p-queue');
const fetch = require('node-fetch');

const queue = new PQueue({concurrency: 1});

queue.add(() => fetch(
    'http://sindresorhus.'),
    { method: 'HEAD'}
).then(() => {
    console.log('Done: sindresorhus.');
});

queue.add(() => fetch('http://ava.li')).then(() => {
    console.log('Done: ava.li');
});

This does work, so I'm not really sure where to look anymore?

• javascript
• node.js

Share Improve this question Follow asked Jul 21, 2018 at 14:59 zkwskzkwsk 2,13666 gold badges3131 silver badges3535 bronze badges 0

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 7

queue.add expects a function, however you are passing a Promise.

When you call queue.add(validateImage(resource)), the engine first evaluates validateImage(resource), which returns a Promise. Then it evaluates queue.add(<returned promise>).

Simply change the call to an arrow function: queue.add(() => validateImage(resource))

本文标签： javascriptQueueing Promises with PQueue node moduleStack Overflow