javascript - How to resolve "Target closed" error using nodeJS and Puppeteer? - Stack Overflow

admin管理员组

文章数量:1349704

I'm trying to crawl and scrape some sites to extract some links. I expect to see all the hrefs printed to my console. But, instead, I get the following error.

out [ Promise { } ] (node:15908) UnhandledPromiseRejectionWarning: Error: Protocol error (Page.navigate): Target closed.

What am I doing wrong?

This answer says the error message means browser.close() has already executed by the time I call my pageFunction.

But I'm using async await and apparently the browser is still closing on me.

How can I fix this error?

const domains = [...]

const pageFunction = async $posts => {
  const data = [];
  await $posts.forEach( $post => {
    data.push( $post.href );
  });
  return data;
}

(async () => {
  // start browser
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // loop over domains
  const out = await domains.slice(-1).map( async domain => {
    const url = [ 'http', domain, ].join(joiner2);
    await page.goto( url, waitUntilLoad, );
    const hrefs = await page.$$eval( 'a', pageFunction, );
    return hrefs;
  });

  // log hrefs
  console.log( 'out', out, );
  await browser.close();
})();

Also, note: When I use:

  const pageFunction = async $posts =>
    await $posts.map( $post => $post.href )

The error goes away. But I also get no output either. I would expect the console to log the hrefs but instead it logs nothing.

FWIW: Here is the question I wrote yesterday on the same piece of code. I modified the code by only doing one URL for now slice(-1) instead of the whole list. And now I get the above error instead of the one I describe in yesterday's question.

I'm trying to crawl and scrape some sites to extract some links. I expect to see all the hrefs printed to my console. But, instead, I get the following error.

out [ Promise { } ] (node:15908) UnhandledPromiseRejectionWarning: Error: Protocol error (Page.navigate): Target closed.

What am I doing wrong?

This answer says the error message means browser.close() has already executed by the time I call my pageFunction.

But I'm using async await and apparently the browser is still closing on me.

How can I fix this error?

const domains = [...]

const pageFunction = async $posts => {
  const data = [];
  await $posts.forEach( $post => {
    data.push( $post.href );
  });
  return data;
}

(async () => {
  // start browser
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  // loop over domains
  const out = await domains.slice(-1).map( async domain => {
    const url = [ 'http', domain, ].join(joiner2);
    await page.goto( url, waitUntilLoad, );
    const hrefs = await page.$$eval( 'a', pageFunction, );
    return hrefs;
  });

  // log hrefs
  console.log( 'out', out, );
  await browser.close();
})();

Also, note: When I use:

  const pageFunction = async $posts =>
    await $posts.map( $post => $post.href )

The error goes away. But I also get no output either. I would expect the console to log the hrefs but instead it logs nothing.

FWIW: Here is the question I wrote yesterday on the same piece of code. I modified the code by only doing one URL for now slice(-1) instead of the whole list. And now I get the above error instead of the one I describe in yesterday's question.

• javascript
• node.js
• asynchronous
• promise
• puppeteer

Share Improve this question Follow asked Dec 6, 2019 at 13:20 Let Me Tink About ItLet Me Tink About It 16.2k2121 gold badges108108 silver badges217217 bronze badges 2

• My two cents here. That error could also mean that the page (or chromium) crashed. – hardkoded Commented Dec 6, 2019 at 13:25
• I'm wondering whether await domains.slice(-1).map( async domain => { is doing what you expect. Are you sure you can make a mapping async? – hardkoded Commented Dec 6, 2019 at 13:30

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 7

I'm quite sure it is because inside the anonymous function you are awaiting for the result of map, which instantly returns an array of Promises. This means your code execution proceeds to closing the browser.

You should try the following:

const promises = domains.slice(-1).map( async domain => {...});
const out = await Promise.all(promises);

本文标签： javascriptHow to resolve quotTarget closedquot error using nodeJS and PuppeteerStack Overflow