ProtocolError: Protocol error (Runtime.callFunctionOn): Target closed. – express.js and puppeteer-cluster

Issue

I want to iterate over key-pairs of data.extractRules and get elements data from the page.

This snippet inside forEach loop is causing app crashes. I tried hardcoding key and cssSelector and tried this outside of forEach loop and it worked.

    const extractContent = {};
    if (data.extractRules !== null) {
      Object.entries(data.extractRules).forEach(async ([key, cssSelector]) => {
        extractContent[key] = await page.$$eval(cssSelector, (elements) =>
          elements.map((element) => element.outerHTML)
        );
      });
    }

Solution

I figured out solution 😁


async function getSelectorContent(page, cssSelector) {
  return page.$$eval(cssSelector, (elements) =>
    elements.map((element) => element.outerHTML)
  );
}

const extractContent = {};
if (data.extractRules !== null) {
    await Object.entries(data.extractRules).reduce(
        async (item, [key, cssSelector]) => {
            await item;
            extractContent[key] = await getSelectorContent(page, cssSelector);
        },
        Promise.resolve()
    );
}

Answered By – Kosaaaaa

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published