I'm trying to screenshot a website using puppeteer, and on slow sites I receive a TimeoutError.
In this case, I'd like to get the screenshot of the current page state - is this possible? if so, how?
Code sample:
const puppeteer = require('puppeteer');
let url = "http://...";
let timeout = 30000;
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page
.goto(url, {waitUntil: 'load', timeout: timeout}).then(async () => {
await page
.screenshot({path: 'example.png'})
.catch(error => console.error(error));
})
.catch(error => {
if (error.name === "TimeoutError") {
// -----> calling await page.screenshot({path: 'example.png'}) gets stuck
} else {
console.error(error);
}
});
await browser.close();
})();
Don't use browser.close when using puppeteer in development, as this may cause the browser closed and puppeteer crashed.
const puppeteer = require('puppeteer')
let url = "https://www.tokopedia.com"
let filename = 'timeout.png'
let timeoutNum = 30000
;(async () => {
const browser = await puppeteer.launch({
headless: false
});
const [page] = await browser.pages ()
page.setViewport ({ width: 1366, height: 768 })
try {
await page.goto(url, {waitUntil: 'networkidle0', timeout: timeoutNum}).then(async () => {
await page.screenshot({ path: 'example.png', fullPage: true })
})
} catch (error) {
if (error.name === "TimeoutError") {
console.log (error.name)
console.log (`Screenshot saved as ${filename}`)
await page.screenshot({ path: filename, fullPage: true })
} else {
console.log (error)
}
}
})()
Related
I am using foreach to run the test for multiple users.
I can access the variable from foreach inside of beforeAll, but does not work inside of test
the following line does not get the value as expected, i verified i have value for that column in the .csv
await page.locator('#searchByCode').fill(${user.Code}); // Does not Work , get undefined
import fs from 'fs';
import path from 'path';
import { test, expect, Page } from '#playwright/test';
import { parse } from 'csv-parse/sync'; //requires https://www.npmjs.com/package/csv-parse
const users = parse(fs.readFileSync(path.join(__dirname, '../users.csv')), {
columns: true,
skip_empty_lines: true
});
//Want test inside of describe to run in serial mode
test.describe.configure({ mode: 'serial' });
test.use({ viewport: { width: 600, height: 900 } }); //Affects all Tests
let page: Page;
users.forEach(user => {
console.log(user.Email);
test.describe(`Login for user ${user.Email}`, () => {
test.beforeAll(async ({ browser }) => {
// Create page once and sign in.
page = await browser.newPage();
await page.goto('https://XXXXXXX.com');
await page.fill("#logInEmail",`${user.Email}`); **//This one works**
//find password text(#loginInPassword) and enter password
await page.fill('#loginInPassword', `${user.Password}`); **//This one works**
await Promise.all([
page.waitForNavigation(/*{ url: 'https://getinline.net/' }*/),
page.locator('#loginSignIn').click()
]);
});
test.afterAll(async () => {
await page.close();
});
test('SearchBusinessByCode', async ({ browser }, testInfo) => {
await page.locator('#searchByCode').fill(`${user.Password}`); **// Does not Work , get undefined**
await page.locator('button', { hasText: 'Search' }).click();
});
});
}); //forEach
I want to send parallel POST requests in puppeteer. I have to change the payload with every request (URL remains the same).
I tried using puppeteer cluster, but how do I change payload with every request when I queue the same request?
Using normal puppeteer
(async() => {
const browser = await puppeteer.launch({
args: [
"--no-sandbox",
"--disable-setuid-sandbox",
"--disable-web-security",
],
executablePath: 'C:/Program Files/..',
headless: false,
});
for(const id of Ids) {
const page = await browser.newPage();
await page.setDefaultNavigationTimeout(60000);
await page.evaluateOnNewDocument(() => {
// Some code
})
await page.setRequestInterception(true);
// Request intercept handler... will be triggered with
// each page.goto() statement
page.on('request', interceptedRequest => {
// Here, is where you change the request method and
// add your post data
var data = {
'method': 'POST',
'postData': JSON.stringify({
....
"similarMaterialId": `${id}`,
}),
'headers': {
.....
},
};
// Request modified... finish sending!
interceptedRequest.continue(data);
});
const response = await page.goto('https://.../getProductInfo');
const responseBody = await response.json();
try {
let title = responseBody.description;
let price = responseBody.price;
fs.appendFile('temp.tsv', `${title}\t${price}\n`, function (err) {
if (err) throw err;
})
}
catch {
console.log(id)
}
await page.close();
}
console.log("Code ended!!")
await browser.close();
})();
I want to create many pages in parallel on a single browser.
Is it possible to define a browser with Javascript disabled to simulate how a crawler would view a page? There should be an option that can be passed.
You can pass javaScriptEnabled in the BrowserContext options:
const playwright = require("playwright");
(async () => {
const browser = await playwright.chromium.launch();
const context = await browser.newContext({
javaScriptEnabled: false
});
const page = await context.newPage();
// ...
await browser.close();
})();
In the case of #playwright/test (where you don't define browser.newContext yourself) you can set javaScriptEnabled in testOptions.
(1) In the playwright.config.js file:
const config = {
use: {
headless: false,
javaScriptEnabled: false
},
};
module.exports = config;
(2) or with test.use:
const { test, expect } = require('#playwright/test');
test.use({ javaScriptEnabled: false });
test('website without JavaScript', async ({ page }) => {
// ...
});
I have been stuck with this for a bit. I need to test a website and I need to post info in order to test if it appears on the page.
What I have so far is this
(async () => {
const browser = await webkit.launch();
const page = await browser.newPage();
await page.route('http://100.100.100.100/', route => route.fulfill({
status: 200,
body: body,
}));
await page.goto('https://theurlofmywebsite/');
await page.click('button')
await page.click('text=Login with LoadTest')
await page.fill('#Username','username')
await page.fill('#Password','password')
await page.click('#loginButton')
// await page.waitForSelector('text=Dropdown');
await page.click('css=span >> text=Test')
await page.click('#root > div > div > header > ul.nav.navbar-nav.area-tabs > li:nth-child(6) > a','Test')
await page.waitForSelector('text=Detail')
await page.screenshot({ path: `example3.png` })
await browser.close();
})();
const body = [ my json post request ]
jest.setTimeout(1000000);
let browser: any;
let page: any;
beforeAll(async () => {
browser = await chromium.launch();
});
afterAll(async () => {
await browser.close();
});
beforeEach(async () => {
page = await browser.newPage();
});
afterEach(async () => {
await page.close();
});
it("should work", async () => {
await fetch("http://YOUAWESOMEURL", {
method: "post",
body: JSON.stringify(body),
})
.then((response) => console.log(response))
.catch((error) => console.log(error));
await page.goto("https://YOUAWESOMEURL");
await page.click("button");
await page.click("text=Login");
await page.fill("#Username", "YOURUSERNAME");
await page.fill("#Password", "YOURPASSWORD");
await page.click("#loginButton");
// await page.click("css=span >> text=Load Test");
await page.click(
"#root > div > div > header > ul.nav.navbar-nav.area-tabs > li:nth-child(6) > a >> text=Test"
);
await page.waitForSelector("text=SOMETEXTYOUWANTTOCHECKIFTHERE");
// await page.waitForSelector(`text=SOMEOTHERTEXTYOUWANTTOCHECKIFTHERE`);
// Another way to check for success
// await expect(page).toHaveText(`SOMEOTHERTEXTYOUWANTTOCHECKIFTHERE`);
console.log("test was successful!");
});
With 1.19 version it looks easy.
test('get respons variable form post in Playwright', async ({ request }) => {
const responsMy= await request.post(`/repos/${USER}/${REPO}/issues`, {
data: {
title: '[Bug] report 1',
body: 'Bug description',
}
});
expect(responsMy.ok()).toBeTruthy();
}
See more on https://playwright.dev/docs/test-api-testing
import { expect, request } from '#playwright/test';
const baseApiUrl = "https://api.xxx.pro/node-api/graphql";
test('API Search', async ({ request }) => {
const search_query = `query {me { id username}} `;
const response = await request.post(baseApiUrl, {
data: {
query: search_query
},
headers: {
authorization: `Bearer eyJhbGciOiJIUzcCI6IkpXVCJ9.eyJzd`
}
});
const bodyResponse = (await response.body()).toString();
expect(response.ok(), `${JSON.stringify(bodyResponse)}`).toBeTruthy();
expect(response.status()).toBe(200);
const textResponse = JSON.stringify(bodyResponse);
expect(textResponse, textResponse).not.toContain('errors');
});
When using await page.goto('http://www.URL.edu') a PDF is generated, but when loading the same URL from a csv file, Puppeteer returns error.
Thought is might have been either a timing issue or a redirect from http to https, but the script proves both not to be the problem.
The commented out is where the url is loaded from a single CSV file with one row: "1,oldwestbury.edu,SUNY College at Old Westbury"
var dir1 = './screenshots';
var dir2 = './pdfs';
const fs = require('fs');
if (!fs.existsSync(dir1)) {
fs.mkdirSync(dir1);
}
if (!fs.existsSync(dir2)) {
fs.mkdirSync(dir2);
}
function readURLFile(path) {
return fs.readFileSync(path, 'utf-8')
.split('\n')
.map((elt) => {
const url = elt.split(',')[1].replace('\r', '');
return `http://${url.toLowerCase()}`;
});
}
const puppeteer = require('puppeteer');
(async () => {
const startDate = new Date().getTime();
const USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3239.108 Safari/537.36';
const urls = readURLFile('./UNVurls.csv');
const browser = await puppeteer.launch({
headless: true
});
for (let url of urls) {
console.log(`Visiting url: ${url}`);
let page = await browser.newPage();
try {
await page.setViewport({ width: 1440, height: 900, deviceScaleFactor: 2 });
await page.goto('http://www.oldwestbury.edu')
// await page.goto(url, {
// waitUntil: 'networkidle2',
// timeout: 0
// });
let fileName = url.replace(/(\.|\/|:|%|#)/g, "_");
if (fileName.length > 100) {
fileName = fileName.substring(0, 100);
}
await page.waitForSelector('title');
await page.screenshot({
path: `./screenshots/${fileName}.jpeg`,
omitBackground: true
});
await page.emulateMedia('screen');
await page.pdf({
path: `./pdfs/${fileName}.pdf`,
pageRanges: "1",
format: 'A4',
printBackground: true
});
} catch (err) {
console.log(`An error occured on url: ${url}`);
} finally {
await page.close();
}
}
await browser.close();
console.log(`Time elapsed ${Math.round((new Date().getTime() - startDate) / 1000)} s`);
})();
Hoping to determine WHY the PDF is created when using the url direct and why the page load fails when retrieving from CSV file.