I have an electron application that loads a web page on the internet.
one of the sites main features is the ability to capture screen, it uses the
navigator.mediaDevices.getDisplayMedia({video: true});
but obviously, the electron will through the Permission denied because there will be no 'selecting window to capture' popped up to grant any permission to it.
I already check out some articles and saw desktopCapture
the problem is, this is happening and running through the web page javascript not my application's code so I don't know how to affect it.
so what should I do to make capturing the screen works in this situation?
You can override navigator.mediaDevices.getDisplayMedia to call Electron's desktopCapturer API like shown below. This implementation assumes you have contextIsolation enabled which is the default behaviour in Electron >= 12
// preload.js
const { desktopCapturer, contextBridge } = require("electron");
const { readFileSync } = require("fs");
const { join } = require("path");
// inject renderer.js into the web page
window.addEventListener("DOMContentLoaded", () => {
const rendererScript = document.createElement("script");
rendererScript.text = readFileSync(join(__dirname, "renderer.js"), "utf8");
document.body.appendChild(rendererScript);
});
contextBridge.exposeInMainWorld("myCustomGetDisplayMedia", async () => {
const sources = await desktopCapturer.getSources({
types: ["window", "screen"],
});
// you should create some kind of UI to prompt the user
// to select the correct source like Google Chrome does
const selectedSource = sources[0]; // this is just for testing purposes
return selectedSource;
});
// renderer.js
navigator.mediaDevices.getDisplayMedia = async () => {
const selectedSource = await globalThis.myCustomGetDisplayMedia();
// create MediaStream
const stream = await navigator.mediaDevices.getUserMedia({
audio: false,
video: {
mandatory: {
chromeMediaSource: "desktop",
chromeMediaSourceId: selectedSource.id,
minWidth: 1280,
maxWidth: 1280,
minHeight: 720,
maxHeight: 720,
},
},
});
return stream;
};
Now when this API is called, a stream will be returned to the caller as expected
navigator.mediaDevices.getDisplayMedia({video: true});
I have created a GitHub repo that has a working implementation of this solution
Related
I'm attempting to use playwright to automate an electron js application, but I can't seem to find any relevant information. To automate a simple programme, I used playwright:- https://playwright.dev/docs/api/class-electron and https://www.electronjs.org/docs/latest/tutorial/quick-start. However, I am unable to obtain the elements (buttons, dropdowns, and so on) in the electron application. Any reference or documentation that will deeply guide me to automate desktop application using playwright.
I got mine to work using their intro guide
for me since the installer installs additional components, i had to build and install, then supply the path to the exe
in my package.json i have.
"playwright": "^1.25.0",
"#playwright/test": "^1.25.0",
"eslint-plugin-playwright": "^0.10.0",
I created this class to help me have a cleaner code.
import { _electron as electron, ElectronApplication, Page } from 'playwright';
class ElectronAppController {
static electronApp: ElectronApplication;
static window1: Page;
static window2: Page;
static window3: Page;
static async launchApp() {
ElectronAppController.electronApp = await electron.launch({
executablePath: 'C:\\pathTo\\app.exe',
});
ElectronAppController.electronApp.on('window', async (page) => {
ElectronAppController.assignWindows(page);
});
const mywindows: Page[] =
await ElectronAppController.electronApp.windows();
for (
let index = 0, l = mywindows.length;
index < l;
index += 1
) {
ElectronAppController.assignWindows(
mywindows[index]
);
}
}
private static assignWindows(page: Page) {
const myurl = path.basename(page.url());
if (myurl === 'window1.html') {
ElectronAppController.window1= page;
}
if (myurl === 'window2.html') {
ElectronAppController.window2= page;
}
if (myurl === 'window3.html') {
ElectronAppController.window3= page;
}
return true;
}
}
the test file name should be [name].spec.ts, don't forget to import
test.describe('First Window Tests', async () => {
test.beforeAll(async () => {
await ElectronAppController.launchApp();
});
test('Check if first window opened', didLaunchApp);
test('name of the test', async () => {
// test body
// this will allow you to record a test very useful, but sometimes it has some problems check note bellow
await ElectronAppController.window1.pause;
});
test.afterAll(async () => {
await ElectronAppController.electronApp.close();
});
});
here is a didLaunchApp just as a simple test
const didLaunchApp = async () => {
const isVisible: boolean = await ElectronAppController.electronApp.evaluate(
async ({ BrowserWindow }) => {
const mainWindow = BrowserWindow.getAllWindows()[0];
const getState = () => mainWindow.isVisible();
return new Promise((resolve) => {
if (mainWindow.isVisible()) {
resolve(getState());
} else {
mainWindow.once('ready-to-show', () => {
setTimeout(() => resolve(getState()), 0);
});
}
});
}
);
await expect(isVisible).toBeTruthy();
};
you can record tests but sometimes that might make some problems if you have some popups or other effects on hovering over an element,
you can read more about selectors here
I'm just finishing a series of e2e tests using the same as you, Electron with React. What you don't see? Does it at least load the application?
Share the code of one test and how you launch using .launch method.
ok so I have been cracking my head over this problem for the last 2 days.
The default exit button of an electron.js app doesn't work.
Whenever I run my app and I click on the exit button, it doesn't show any error in the terminal and doesn't close the app either.
So far, these are the code snippets I have used:
// functions I used
app.on('window-all-closed') {
}
app.on('close') {
}
// the code inside the functions I used
_____________________________________
app.destroy()
_____________________________________
app.exit()
_____________________________________
app = null;
And I haven't used any HTML code since I am loading the URL of the page I want to make as an app like this:
function createWindow () {
const { BrowserWindow } = require('electron')
const win = new BrowserWindow({
width: 720,
height: 600,
icon:'assets/icons/win/icon.png',
})
win.loadURL('https://insert-link')
win.removeMenu()
win.on("closed", () => mainWindow.destroy());
}
const { app } = require('electron')
app.whenReady().then(createWindow)
app.once('window-all-closed', function() {
app.quit().then(app.destroy())
})
app.on('activate', () => {
if (BrowserWindow.getAllWindows().length === 0) {
createWindow()
}
})
Please help me on this.
I'm working on a JHipster application that I'm trying to get functioning in Electron. I have Golden Layout for window/pane management and cross-pane communication. I am having several problems with the combination of technologies, including:
I can't pop out more than one pane at the same time into their own Electron windows. I instead get an Uncaught Error: Can't create config, layout not yet initialised error in the console.
Two thirds of the panes don't display anything when popped out into Electron windows, and I'm not sure what the reason is. Any ideas or suggestions for this? One example of content is a leaflet map, another is a "PowerPoint preview" that is really just divs that mock the appearance of slides.
I haven't made it this far yet, but I assume that I will have trouble communicating between popped-out Electron windows when I get more than one open. Right now, the panes communicate between each other using Golden Layout's glEventHub emissions. I have an avenue to explore when I cross that bridge, namely Electron ipcRenderer.
Some borrowed code is here (most of it I can't share because it's company confidential):
electron.js:
const electron = require('electron');
const app = electron.app;
const BrowserWindow = electron.BrowserWindow;
const path = require('path');
const isDev = require('electron-is-dev');
let mainWindow;
function createWindow() {
mainWindow = new BrowserWindow({width: 900, height: 680});
mainWindow.loadURL(isDev ? 'http://localhost:9000' : `file://${path.join(__dirname, '../build/index.html')}`);
if (isDev) {
// Open the DevTools.
//BrowserWindow.addDevToolsExtension('<location to your react chrome extension>');
mainWindow.webContents.openDevTools();
}
mainWindow.on('closed', () => mainWindow = null);
}
app.on('ready', createWindow);
app.on('window-all-closed', () => {
if (process.platform !== 'darwin') {
app.quit();
}
});
app.on('activate', () => {
if (mainWindow === null) {
createWindow();
}
});
goldenLayoutComponent.tsx, a patch for Golden Layout:
import React from "react";
import ReactDOM from "react-dom";
// import "./goldenLayout-dependencies";
import GoldenLayout from "golden-layout";
import "golden-layout/src/css/goldenlayout-base.css";
import "golden-layout/src/css/goldenlayout-dark-theme.css";
import $ from "jquery";
interface IGoldenLayoutProps {
htmlAttrs: {},
config: any,
registerComponents: Function
}
interface IGoldenLayoutState {
renderPanels: Set<any>
}
interface IContainerRef {
current: any
}
export class GoldenLayoutComponent extends React.Component <IGoldenLayoutProps, IGoldenLayoutState> {
goldenLayoutInstance = undefined;
state = {
renderPanels: new Set<any>()
};
containerRef: IContainerRef = React.createRef();
render() {
const panels = Array.from(this.state.renderPanels || []);
return (
<div ref={this.containerRef as any} {...this.props.htmlAttrs}>
{panels.map((panel, index) => {
return ReactDOM.createPortal(
panel._getReactComponent(),
panel._container.getElement()[0]
);
})}
</div>
);
}
componentRender(reactComponentHandler) {
this.setState(state => {
const newRenderPanels = new Set(state.renderPanels);
newRenderPanels.add(reactComponentHandler);
return { renderPanels: newRenderPanels };
});
}
componentDestroy(reactComponentHandler) {
this.setState(state => {
const newRenderPanels = new Set(state.renderPanels);
newRenderPanels.delete(reactComponentHandler);
return { renderPanels: newRenderPanels };
});
}
componentDidMount() {
this.goldenLayoutInstance = new GoldenLayout(
this.props.config || {},
this.containerRef.current
);
if (this.props.registerComponents instanceof Function)
this.props.registerComponents(this.goldenLayoutInstance);
this.goldenLayoutInstance.reactContainer = this;
this.goldenLayoutInstance.init();
}
}
// Patching internal GoldenLayout.__lm.utils.ReactComponentHandler:
const ReactComponentHandler = GoldenLayout["__lm"].utils.ReactComponentHandler;
class ReactComponentHandlerPatched extends ReactComponentHandler {
_container: any;
_reactClass: any;
_render() {
const reactContainer = this._container.layoutManager.reactContainer; // Instance of GoldenLayoutComponent class
if (reactContainer && reactContainer.componentRender)
reactContainer.componentRender(this);
}
_destroy() {
// ReactDOM.unmountComponentAtNode( this._container.getElement()[ 0 ] );
this._container.off("open", this._render, this);
this._container.off("destroy", this._destroy, this);
}
_getReactComponent() {
// the following method is absolute copy of the original, provided to prevent depenency on window.React
const defaultProps = {
glEventHub: this._container.layoutManager.eventHub,
glContainer: this._container
};
const props = $.extend(defaultProps, this._container._config.props);
return React.createElement(this._reactClass, props);
}
}
GoldenLayout["__lm"].utils.ReactComponentHandler = ReactComponentHandlerPatched;
Any help or insight into these issues would be appreciated. Thanks in advance!
If you are still looking for a solutions, 1 and 2 I have solved, if you want to see my solution you could see in this repository.
But it was basically this:
1: The window that popups has a different path than the main window, so I just had to put a try catch in my requires, and you have to set
nativeWindowOpen = true
when creating the Browser window.
2: Solves it's self after 1 I think
I'm loading a window from data-uri:
pref.loadURL('data:text/html;charset=utf-8,' + encodeURI(str), { baseURLForDataURL: 'file://' + app.getAppPath() } );
The good news is that the dev tools console shows errors now for the css/js files that should be loading but aren't, but I can't make sense of what it expects. There are no examples anywhere, not even in the github issues that inspired this option.
Does it expect an absolute path (as in my example above)?
It would normally expect
'file://' + app.getAppPath().replace("\\", "/") + "/"
But at the moment, there seem to be an issue where at least when using the protocol 'file://' we get an error. (https://github.com/electron/electron/issues/20700)
One way of going around this issue is by generating a custom file protocol.
const { app, BrowserWindow, screen, protocol } = require('electron');
const path = require('path');
app.on('ready', () => {
const customProtocol = 'file2';
protocol.registerFileProtocol(customProtocol, (request, callback) => {
const url = request.url.substr(customProtocol.length + 2);
const file = { path: path.normalize(`${__dirname}/${url}`) }
callback(file)
});
let win = new BrowserWindow({
webPreferences: {
nodeIntegration: true,
contextIsolation: true
}
});
win.loadURL(`data:text/html;charset=UTF-8,${encodeURIComponent(indexHTML)}`, {
baseURLForDataURL: `${customProtocol}://${app.getAppPath().replace("\\", "/")}/`
});
});
I'm attempting to implement windows auto update functionality in an electron app (which may lead to my early death) and I'm getting this error.
This is the URL I'm passing for testing purposes
EDIT: my electron app is using the two package.json structure and this code is in my app>main.js file
const feedURL = 'C:\\Users\\p00009970\\Desktop\\update_test';
autoUpdater.setFeedURL(feedURL);
autoUpdater.checkForUpdates();
EDIT2: Thanks to #JuanMa, I was able to get it working. Here is the code.
// auto update functionality
const {autoUpdater} = require('electron')
// local file system example: const feedURL = 'C:\\Users\\john\\Desktop\\updates_folder';
// network file system example: const feedURL = '\\\\serverName\\updates_folder';
const feedURL = '\\\\serverName\\updates_folder';
app.on('ready', () => {
autoUpdater.setFeedURL(feedURL);
// auto update event listeners, these are fired as a result of autoUpdater.checkForUpdates();
autoUpdater.addListener("update-available", function(event) {
});
autoUpdater.addListener("update-downloaded", function(event, releaseNotes, releaseName, releaseDate, updateURL) {
//TODO: finess this a tad, as is after a few seconds of launching the app it will close without warning
// and reopen with the update which could confuse the user and possibly cause loss of work
autoUpdater.quitAndInstall();
});
autoUpdater.addListener("error", function(error) {
});
autoUpdater.addListener("checking-for-update", function(event) {
});
autoUpdater.addListener("update-not-available", function(event) {
});
// tell squirrel to check for updates
autoUpdater.checkForUpdates();
})
Are you including the autoUpdater module correctly?
const {autoUpdater} = require('electron')
If so try to execute the code after the app 'ready' event.
app.on('ready', () => {
const feedURL = 'C:\\Users\\p00009970\\Desktop\\update_test';
autoUpdater.setFeedURL(feedURL);
autoUpdater.checkForUpdates();
})