For some of the below listed URLS, I am unable to fetch content of the page. I tried all options like load,dom,networkidle,commit and also tried using Chromium,WebKit and Firefox.
http://blog.naver.com/anupdown/221361918901
https://studio-sj.co/privacypolicy
https://www.leboncoin.fr/dc/cookies/
Code below
const { chromium } = require("playwright"); // Web scraper Library
let chromeBrowser = await chromium.launch({ headless: true }); // Chromium launch and options
let pageResponse = await page.goto("https://studio-sj.co/privacypolicy", {waitUntil: 'load', timeout: 30000 });
let content = await page.content();
Please note the content is inappropriate (i.e) HTML contains scripts.
Related
My JS work only if i reload page.
Description my problems
1.I have book
2.Book have description
3.In action show i have logic for present description
Logic:
1. If description.length > 250 i show 250 symbols and show view_more button
2. If user click on button then my js must work
4.When i on books#index => choose some book(#show)=> now i on book_page and i see view_button => when i click => nothing was happen
5. But if i reload page before click on button => my js work fine
My js
(function(window, document, undefined){
window.onload = init;
function init(){
let btn_view_more = document.getElementById('button_for_view_more')
let description_all = document.getElementById('book_description_all')
let description_short = document.getElementById('book_description_short')
btn_view_more.addEventListener('click', ()=>{
description_all.classList.remove('hide_description')
description_short.style.display = 'none'
});
};
})(window, document, undefined);
But i have new version of my js
document.addEventListener("turbolinks:load", function() {
myFunc();
})
function myFunc(){
let btn_view_more = document.getElementById('button_for_view_more')
let description_all = document.getElementById('book_description_all')
let description_short = document.getElementById('book_description_short')
btn_view_more.addEventListener('click', ()=>{
description_all.classList.remove('hide_description')
description_short.style.display = 'none'
});
}
New version work fine even i don't reload page, but i have error in my console
Cannot read properties of null (reading 'addEventListener') (can't find id button_for_view_more)
What i want:
1. Js work even i don't reload page
2. Zero error in console
My solution is connect js only to the page where it is used, delete from javascript/some.js and drop require from pack/application.js. You can also add a condition to your script that will check whether such an ID is currently on the page. And only after run the main block.
Following these instructions, I went to about:debugging -> This Firefox and clicked Inspect on my extension, which shows a console. But no logs show up there when I trigger my extension. Using the list-cookies example, I added the last two lines:
gettingAllCookies.then((cookies) => {
//set the header of the panel
var activeTabUrl = document.getElementById('header-title');
var text = document.createTextNode("Cookies at: "+tab.title);
var cookieList = document.getElementById('cookie-list');
console.log('I can't see this log!');
cookieList.parentNode.appendChild(document.createTextNode(Date()));
When I invoke the popup, I see the current date/time in the popup, but no log shows up in the console.
I tried setting extensions.sdk.console.logLevel and restarting as mentioned here (even though I think that's for older versions), but it didn't help.
I thought maybe there's a console permission or something I might need to add to the manifest, but didn't find any such thing.
Complete code for reference. I only changed the lines marked with +/-:
function showCookiesForTab(tabs) {
//get the first tab object in the array
let tab = tabs.pop();
//get all cookies in the domain
var gettingAllCookies = browser.cookies.getAll({url: tab.url});
gettingAllCookies.then((cookies) => {
//set the header of the panel
var activeTabUrl = document.getElementById('header-title');
var text = document.createTextNode("Cookies at: "+tab.title);
var cookieList = document.getElementById('cookie-list');
- activeTabUrl.appendChild(text);
+
+ console.log('I can't see this log!');
+ cookieList.parentNode.appendChild(document.createTextNode(Date())); // I see this updated even though I don't see the log
if (cookies.length > 0) {
//add an <li> item with the name and value of the cookie to the list
for (let cookie of cookies) {
let li = document.createElement("li");
let content = document.createTextNode(cookie.name + ": "+ cookie.value);
li.appendChild(content);
cookieList.appendChild(li);
}
} else {
let p = document.createElement("p");
let content = document.createTextNode("No cookies in this tab.");
let parent = cookieList.parentNode;
p.appendChild(content);
parent.appendChild(p);
}
});
}
//get active tab to run an callback function.
//it sends to our callback an array of tab objects
function getActiveTab() {
return browser.tabs.query({currentWindow: true, active: true});
}
getActiveTab().then(showCookiesForTab);
Firefox console has been divided into different areas. The result of console.log() can be viewed in the relative area.
Multiprocess Browser Console Ctrl+Shift+J
Mostly logs by Firefox itself
Web Developer Tools Ctrl+Shift+I or F12
Logs by Tab/Webpage and Content scripts of addons
Extension Toolbox about:debugging#/runtime/this-firefox ➜ XYZaddon ➜ Inspect
Logs by background scripts of XYZaddon
Update
Based on comments, here is a tested simplified code that you can work on. The log shows on Extension Toolbox.
async function showCookiesForTab() {
// get the first tab object in the array
const tabs = await browser.tabs.query({currentWindow: true, active: true});
// get all cookies in the domain
const cookies = await browser.cookies.getAll({url: tabs[0].url});
console.log(cookies);
// Array(6) [ {…}, {…}, {…}, {…}, {…}, {…} ]
}
showCookiesForTab();
I had a similar issue. I didn’t figure out the cause, but I find a way to see the console.log in the Extension Toolbox.
I added a background script to handle most of the popup.js logic.
And since there is a background script running, I can see the log.
Still don’t why I couldn’t see the log in the first place.
I want to record my electron app's content using Page.startScreencast command of Chrome DevTools Protocol?
Could not find out any exmaple use. How can achieve this?
Sorry my answer is based on Puppeteer, I have no experience in electron. But it seems it is possible to use them together. Not sure it will answer your use case though. There might be a better way to access Chrome Devtools Protocol directly with some Electron's APIs.
https://www.npmjs.com/package/puppeteer-in-electron
For a classical Puppeteer / Node solution
import { Browser, Page } from 'puppeteer'
const browser = await puppeteer.launch({
// your params
})
const page = await browser.newPage()
const yourWebSite = 'http://www.whatYouWantToCapture.com'
await page.goto(yourWebSite, {
waitUntil: 'networkidle0', // Ensure page is finished loading
})
const client = await page.target().createCDPSession()
//Register a callback on every frame rendered by the browser (framerate depends on a lot of factors)
client.on('Page.screencastFrame', async (frameObject) => {
// Do what you want with frame, ex write to file on disk
await fs.writeFile(Date.now()+'.jpeg', frameObject.data, 'base64')
await this.client.send('Page.screencastFrameAck', {
sessionId: frameObject.sessionId,
})
)
// When you want to start
client.send('Page.startScreencast',{
format: 'jpeg',
quality: 100,
maxWidth: 1920,
maxHeight: 1080,
everyNthFrame: 1,
})
// When you want to stop
client.send('Page.stopScreencast')
I'm using Electron I have a webview which display an external website but I can't succeed to show the additional window normally opened by links on this site and which have target = _blank.
Mentions légales
I tried with
webpreferences="nativeWindowOpen=yes" allowpopups
But it didn't change.
With a webview, you can actually handle these on the main process quite easily.
Which also allows you to disable nodeIntegration should that be a requirement.
// Listen for web contents being created
app.on('web-contents-created', (e, contents) => {
// Check for a webview
if (contents.getType() == 'webview') {
// Listen for any new window events
contents.on('new-window', (e, url) => {
e.preventDefault()
shell.openExternal(url)
})
}
})
After digging into the documentation I wrote this code (code located in the renderer):
const {BrowserWindow} = require('electron').remote
..........
webview1.addEventListener('new-window', (e) => {
const protocol = require('url').parse(e.url).protocol
if (protocol === 'http:' || protocol === 'https:') {
//shell.openExternal(e.url)
let win = new BrowserWindow({width: 800, height: 600})
win.loadURL(e.url);
}
})
The line shell.openExternal(e.url) open the url of the link in a tab of the default browser.
And by using a new BrowserWindow, the new windows are Electron Window.
I have a <webview> in my Electron app. I'd like to have safe "foreign" communication, the way I would with an iframe through postMessage. So for example:
webview.executeJavaScript("window.parent.postMessage('all done!')");
Is my only choice for communication with this subwebview to turn on nodeIntegration so that I can use sendToHost? Turning on all of nodeIntegration just for this one feature seems like overkill.
You can access Electron APIs in the webview preload script, including IPC, even when nodeIntegration is disabled. Your preload script can inject functions into the global namespace that will then be accessible within the page loaded in the webview. A simple example:
webview-preload.js:
const { ipcRenderer } = require('electron')
global.pingHost = () => {
ipcRenderer.sendToHost('ping')
}
webview-index.html:
<script>
pingHost()
</script>
window-index.html:
<script>
const webview = document.getElementById('mywebview')
webview.addEventListener('ipc-message', event => {
// prints "ping"
console.log(event.channel)
})
</script>
Easiest way
Communication is
Note:
(main.js or app.js or background.js or process.js ) no need to pass (directly pass component to component),i succesffully implemented in electron:3.1.10
for print html webview.
Window To Webview
example1.html
<webview id="paper" style="width:300px;height:800px" src="file:///static/mywebview.html" nodeintegration></webview>
example1.js
var webview = document.getElementById("paper");
webview.send("ping",data);
getting data from mycomponent or window(i send directly form component)
mywebview.html
<!---what data you want show----!>
mywebview.js
const {
ipcRenderer
} = require('electron')
//data from window
ipcRenderer.on('ping', (e, data) => { console.log(data) })
webview to window
Webview to window(direct pass to component)
mywebview.js
ipcRenderer.sendToHost("readyCompanyInfo",data)
in my window eg i use vue (mycomponent.vue or mypage)
example1.html
const ipcRenderer = require("electron").ipcRenderer;
webview.addEventListener("ipc-message",(event)=>{
const {args,channel}=event;
if(channel=="readyCompanyInfo")
{
console.log(channel,args)
//here you can see data what u passed from webview to window
console.log(args[0])
}
})