From single argument function application to a mapping over an array using async/await in node

Question

I'm new to await/async in modern javascript. I understand there are a lot of questions related to this in general, but in this question, I'm looking specifically for a template for thinking through how to take a single function and rewrite it as a map over an array.

I'm starting with a very simple screenshot function from here:

const puppeteer = require('puppeteer');

puppeteer.launch().then(async browser => {
  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({path: 'screenshot.png'});
  await browser.close();
});

This just does one screenshot. I'd like to give it an array, and have it take multiple screenshots.

For my application, I tweak this slightly with the mypage variable:

const puppeteer = require('puppeteer');
puppeteer.launch().then(async browser => {
  const mypage = 1
  const page = await browser.newPage();
  await page.goto(`https://localhost:9001/${mypage}`);
  await page.screenshot({path: `${mypage}.png`});
  await browser.close();
});

Now, I'd like to parameterize this list over mypages = [1 ... 1000]. In older javascript, using underscore.js, I'd try something like:

// define the function
function ss(mypage) {
  puppeteer.launch().then(async browser => {
    const page = await browser.newPage();
    await page.goto(`https://localhost:9001/${mypage}`);
    await page.screenshot({path: `${mypage}.png`});
    await browser.close();
});  
}

// populate the array of pages, eg make var mypages = [1,2,3...1000]
var mypages = [];
for (var i = 1; i <= 1000; i++) {
    mypages.push(i);
}

// call the screenshot function on each page in my pages.
_.each(mypages, ss)

It is not clear to me how I could use async/await to work over each page in mypages (and potentially to generate this array more idiomatically, too).

How can I rewrite my puppeteer function which works for mypage = 1, to map over the whole array of mypages?

Note for future readers, the code I have above will likely crash your machine if you try to run it. — Mittenchops
– Mittenchops, Commented Nov 1, 2018 at 21:17

Jonas Wilms · Accepted Answer · 2018-11-01 21:21:58Z

2

The screenshot function should be async:

 async function screenshot(mypage) {
  const browser = await  puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(`https://localhost:9001/${mypage}`);
  await page.screenshot({path: `${mypage}.png`});
  await browser.close();
}

Then do one screenshot after another:

 (async function() {
    for(let i = 0; i < 1000; i++)
      await screenshot(i);
 })();

Or all at the same time (not sure if that works):

 Promise.all(Array.from({ length: 1000 }, (_, i) => screenshot(i)))
  .then(/*...*/);

edited Nov 1, 2018 at 21:21

answered Nov 1, 2018 at 21:05

Jonas Wilms

139k20 gold badges164 silver badges164 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Bergi Over a year ago

This opens 1000 browsers at the same time, though. I would recommend to do the screenshots sequentially, preferably within the same puppeteer instance.

Mittenchops Over a year ago

Bergi, so something like having the screenshot function take the browser as an argument and initializing it in the outer async function?

Collectives™ on Stack Overflow

From single argument function application to a mapping over an array using async/await in node

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest