I'm researching reliable ways to programmatically control instances of Chrome/Chromium to leverage its capabilities of rendering web pages in Node.js/C#/Java application. In short, things I want to do are as follows:
Open/close a browser window.
Minimize, maximize browser window.
Navigate to a certain URL.
Set cookies.
To make it more clear: I need a headful browser to display web pages to the end users. It can be either embedded to my app or can be a standalone browser (separately shipped instance of Chromium for example).
I was not able to find information about any public APIs in Chrome/Chromium that I can use from the Node.js/C#/Java environment. The ones available for Chrome Extensions are not applicable to my project, as I want to control the browser from the outside, like Selenium WebDriver does for example. So far I found the following ways to control the browser the way I need:
To use Puppeteer/WebDriver APIs.
Use chrome-remote-interface NodeJS library.
Rely on Chrome Embedded Framework capabilities.
Rely on Electron.js capabilities.
Build my own library that somehow includes the Chromium modules as dependencies (similar to what Electron team implemented for example).
First two options are similar from the perspective of all the mentioned libraries eventually leveraging Chrome Devtools Protocol. The risk of CDP being retired/deprecated is quite substantial for our project. The other concern is that the intention of CDP is debugging and test automation and not application development. Moreover, having an open debugging port in Chrome open on user's machine seems vulnerable.
CEF and Electron paths concern me due to dependency on the embedded Chromium updates cadence. Although Electron team is targeting to update with every other release of Chromium it still can be a security concern due to inability to update the Chromium version right after a new version released with a security patch, for example. Moreover, in case when I need the real browser experience (and that's the case) I won't have it out of the box and I'll have to implement browser features like buttons, tabs address bar etc. myself.
Option #5 seems to be extremely complex in implementation as it seems to require team's competency on Chromium internals, C++ development and C++ build tooling.
Anything I missed in the options list? Anything I missed in my assumptions? Any tips, thoughts, suggestions will be greatly appreciated!
Some of your options are about controlling a browser (#1, #2) while others are about embedding a browser (#3, #4). These are two different use cases and what you need depends on what your goal is.
Controlling a browser
If you want to control a browser to execute tasks, maybe even in the background without the end user noticing, you should go for option 1 (puppeteer) or 2 (chrome-remote-interface).
I recommend to use puppeteer as this is the library developed by the Google Chrome developers and it comes with many functions for your use case (opening browser windows, navigating, setting cookies).
I do not see any reason to worry about the Chrome DevTools Protocol being abandoned anytime soon. The Chrome DevTools fully rely on this protocol. In addition, Firefox (Mozilla bug tracker: #1316741,#1523097) and Edge are already partly supporting the protocol making it even more unlikely to be abandoned in the future. (more information)
Embedding a browser
If you need to embed a browser, meaning you are trying to show a browser inside your application, you should focus on the options 3 (Chrome Embedded Framework) or 4 (Electron).
The Chrome Embedded Framework is a more low-level approach putting a separate browser into your application. But I cannot go into detail here, as I have never used this one myself.
Electron on the other hand is a browser, meaning the whole application is developed as web application. You can embed another browser window (webview) into your browser, which you can essentially control (similar to what puppeteer can do).
Directly using the Chromium code (option 5)
Although the Chromium project is split into multiple components, it sounds like you need a full browser. I once compiled the Chromium source code myself and it takes literally hours. Keep in mind, that he code consists of roughly 35 million lines of code (source). Even if you figure out what parts of the code to use, it is more realistic that some low-level parts of the code change and break your implementation than the DevTools Protocol being abandoned. So, I definitely recommend to not follow this idea.
Alternatives
Depending on your use case, you could also take a look at DOM simulation libraries like jsdom or cheerio. These libraries are very limited in terms of their functionality and you might have to implement parts of the browser yourself, e.g. downloading the document, reading and setting headers to deal with cookies, etc.
All in all, I recommend to go for puppeteer if you want to control a browser to execute tasks primarily in the background. If you need a browser window as part of your application go for Electron.
Related
I've been trying to work on a proof of concept (POC) where I can embed a UE4 project into an existing application (in my case NativeScript) but this could just as easily apply to Kotlin or ReactNative.
In the proof of concept I've been able to run the projects on my iPhone launching from UE4 pretty easily by following the Blueprint and C++ tutorials for the FPS. However the next stage of my POC requires that I embed the FPS into an existing NativeScript application, this application will manage the root menu, chat, and store aspects of the platform in the POC.
The struggle I'm running into is that I cannot find how to interact with the xcode project generated from the blueprint tutorial and the C++ tutorial generates a xcode project that i'm unsure where the actual root is that I need to wrap.
Has anyone seen a project doing this before and if so are there any blogs or guidance that you can point me to? I've been Googling and looking around for a couple weeks and have hit a dead end. I found a feedback post here from April of 2020, that was referring to a post in January 2020 that talked about how Unity has a way to embed into other applications additionally a question from 2014 here. But other than that it's a dead end.
A slightly different approach
Disclaimer: I'm not an UE4 developer. Guilty as charged for seeing an unanswered bounty too big to ignore. So I started thinking and looking - and I've found something that could be bent to your needs. Enters pixelstreaming.
Pixelstreaming is a beta feature that is primarily designed to allow for embedding the game into a browser. This opens a two way communication between a server where the GPU heavy computations happen and a browser where the player can interact with the content - the mouseclick & other events are sent back to the server. Apparently it allows some additional neat stuff, however that is not relevant for the question at hand.
Since you want to embedd the Unreal application into your NativeScript tool(menu of some kind if I understood correctly), you could make your application a from two separate parts:
One part would run the server.
The second part would handle the overlay via the pixelstreaming.
This reduces the issue of embedding the UE4 into an application to the(possibly easier) issue of embedding a browser into your application. (Or if your application is browser based - voila, problem solved.)
If you don't want to handle the remote communication, just have the server-side run on the localhost.(With the nice sideeffect of saving bandwidth.)
Alternatively, if you are feeling adventurous, you could go and write your own WebRTC support on the application side to bypass the need for the browser alltogether. It might not be worth the effort though.
Side note: The first of the links you provided is a feature request which hints at the unfortunate fact that UE4 doesn't support embedding. This is further enforced by the fact that one of the people there says somethig along the lines "Unity can to this, it would be nice if UE4 could as well."
Yet a different approach:
You could embedd and use a virtual display to insert the UE4 part into your controller - you would be basically tricking UE4 into thinking that the desired display device is a canvas inside your application.
This thread suggests a similar approach:
In general, the way to connect two libraries like this would be through a platform dependent window handle, e.g. a HWND under Windows. Check the UE api if you find any way to bind the render target to a HWND. Then you could create a wxWindow in wxWidgets and tell UE to render into that window. That would be a first step.
I'm not sure if anything I've listed will be of much help but hey, at least I tried :-). Good luck with your game.
At the same time, the author suggests to:
Reverse the problem:
Using the UE4 slate framework and online subsystem. You would use the former to create the menus you need directly in the UE4 and then use the latter to link to the logic you want to have outside of the UE4. However that is not what you asked for so I'm listing it only for the completeness sake.
I want to write a web app that can process a users presentation which when submitted, gets sent to their local system printers - one part of the order will go to B&W laser printer and another part goes to another printer for another process.
Ideally, this will be able to run on any of the major systems - Linux, OSX or Windows.
It could be a packaged Chrome or Firefox app or extension, but I can't tell if there is sufficient access to the system printers. (I can see all the printers available to my current chrome installation - but are they available to an extension?). I can't see the code in any demos or libraries.
I've seen reference to NPAPI, but I can't tell if that's what I need either - if so would I need to write an app for each of the major platforms or each kind of printer?
In Firefox it is possible to print from an extension. Using only javascript, that is (you're free to mess about with C++ if you like).
Extensions are essentially the browser: Everything the browser can do, an extension can, too.
There is however not much documentation about using the printer. To be precise, there is none that I'm aware of. However, the browser is open source, and the UI is mostly Javascript, so you can just read the code.
printUtils.js would be a good starting point to check out how this might work. Also there is a component implementing nsIPrinterEnumerator. See the firefox-addon info section for a collection of "Getting started" and documentation resources about add-on development in general.
I don't think the Chrome extension API provides ways to control printing the way you'd need.
The NPAPI plugin API does not provide enough control over printing. You just basically get a buffer to draw into, but no way to control printing setting or initiate printing yourself. You could cheat of course, and have your plugin directly print something via OS bypassing the browser, like e.g. those PDF readers do. However, it should be noted that NPAPI plugins are the past... Don't develop new ones, if possible.
I am working on a project I intended as an academic exercise for myself. What I am basically doing is trying to add a custom compression type between an IIS server and a web browser, but getting mired in the browser aspects of it. What is basically needed is to be able to alter the byte stream on the server before it is sent to the client, and on the client browser side, be able to alter received response byte stream before other normal DOM parsing and such take place. On the server side this turned out to be pretty trivial, but I am at a loss in how to do this in chrome or firefox. If anyone has insight for chrome, please share, but I'll focus on Firefox for now. My understanding is that previously Firefox had rather low level access available to extensions but that they are tightening the restrictions a bit going forward with their Firefox add-on sdk. I don't want to bother learning the old methods if they are slated to be removed so does anyone definitively know if the add-on sdk allows you to alter the byte stream directly in this manner? If so, do you have an example of such usage?
(I am aware that I need to have the compression negotiated with the Accept-Encoding header)
I don't want to bother learning the old methods if they are slated to be removed
There are no plans now, or in the future, to remove legacy add-on support. At least that's what add-on SDK folks and extension manager folks told me repeatedly.
Back to your question: The add-on sdk does not provide an explicit API for stuff like this, but you can always go even more low-level, even in the SDK (via the chrome module and/or window/utils).
To implement additional compression methods, you'd need to implement the nsIStreamConverter interface and properly register your component under the #mozilla.org/streamconv;1#?from=<yourcompression>&to=uncompressed contract. Then Firefox should be able to decode yourcompression.
See https://developer.mozilla.org/ for more information on how to implement and register XPCOM components in either javascript or C++.
Using the SDK or implementing your add-on restartless will require you register the component yourself instead of relying on chrome.manifest. There are a couple of add-on doing so already, e.g. Adblock Plus.
Binary (C++) components should be avoided because you'll obviously need to compile your component for each supported platform and you'll need to re-compile it for each Gecko version. If you have to go binary, a javascript component stub + js-ctypes might be better.
See the mozilla source code on how to implement nsIStreamConverter in the first place.
You may also need to modify the network.http.accept-encoding preference so that the compression may be actually negotiated with the server.
This is kind of a dumb question but I've aware of classic style JDE development for Blackberry but I've never tried using WebWorks. BB website says that it's possible to build applications for both smartphones (OS 6.0+) and tablets - sounds fantastic, but what's the price?
Is here anyone using WebWorks on a daily basis and capable of describing pros and cons?
Thanks in advance
I would suggest using it if you build webOS applications before hand. It make porting to the blackberry a breeze.
Use WebWorks if you know html5, Css3 and javascript over Java and C++.
I haven't ran into any issues with the webWorks, ported two applications without running into any issues. Its your standard html5, css3 and javascript you love with blackberry APIs
WebWorks is a good development choice, particularly as it allows easy migration from earlier BB OSes to BB10. It's mostly standard web technologies (HTML5, CSS3, etc.) and the team seems focused on making it perform well (e.g. hardware accelerated WebGL graphics) while at the same time providing BlackBerry-specific APIs to make WebWork apps capable and with good UX (e.g. you can make it look like a native app).
For native apps, you should look into Cascades. This is a modern development environment with good tooling, accelerated graphics, and APIs for building snazzy apps. It's the one that will most be a "BlackBerry app".
AIR remains an option, but I would recommend WebWorks over AIR, as even Adobe is migrating from Flash to web technologies. Likewise, you can develop Android apps on BB10, but unless you are keen on Java programming, you will get more cross-platform support from WebWorks (or even AIR) so there's no particular reason to go the Android route.
WebWorks API is limited, for example it does not have socket, so you cannot port a VNC (UltaVNC, tightVNC ..) to it but you can do it with JDE.
For UI, WebWorks allowed me to write UI of acceptable quality quickly and easily, a thing that I have never succeeded with JDE.
Still on the UI side, I can make use of multi-touch (PlayBook), I don't think this one is possible with JDE.
So depending on your needs you should go either WebWorks or Native, having heard that Java may not be supported in BB10, and Air may not be future proof (Adobe favors HTML5 instead of Flash). Android appli has some lag on start up when it is run on PlayBook, some customers are sensitive to the initial even just one time slow response time.
I'm a huge proponent of Webworks. Ever since I've started using it, it quickly became the default option for my apps going forward. Especially for someone like me who is just writing a few apps on the side, I don't have the time to do it in c++.
The apps I'm writing revolve around home automation. They are client/server based from the get go.
Here's why I like it:
First and foremost, native API support. I can very easily create my own active frames, import invocation from other apps (think camera, stuff like that). I can export portions of my webworks app as an invocation card! Which means I can write say 3 unique apps (in this case home automation, lights, thermostat, security cameras). And I can very easily pull features from each app into the other. Maybe I want to turn my lights on in the living room, I can also import the camera card from my IPcam app and view the results, without having to add that code into my lights app and maintain two separate code lines.
Rapid design. Since I've been dabbling in html since I was a kid, it's now very easy for me to whip up an appealing UI in little time. Because web engines these days offer good performance in terms of graphics capability, I also can make apps that behave very fluid.
Considering the time to make something beautiful, it's hard for me to leave webworks and go for something in c++. Also the big plus is often these apps I'm making are intended for multiple devices, namely an app on my phone and being hosted on my personal website. By maintaining two slightly different css files, most of the time I need no code changes, just load a different css depending on if it's a phone or a pc. (Exactly what you'd do if you were developing a regular old website).
For that matter, I actually don't put my code on the device, I host all of my html and javascript, images etc on my server. The webworks app is just the config.xml pointing it's source to my server, and an icon. A glorified website bookmark on the homescreen, only difference is I can use native API and there's no browser bar in the app.
Also, this way I can still continue to edit the same single codeline on my server, and instantly apply changes to the in-browser app and the on-device app.
This is especially cool if you're designing an app where all of it's data is out in the "cloud", say you work for a publication and you want to write a magazine app which pulls content from your servers on the net.
I'd like my Blackberry devices to get and possibly set data to the server, and am a little confused by all the options:
RIM-Push/Pap-Push. Use any library that is able to compose a HTTP GET request to the MDS-CS service (not MDS-IS). This is a one way operation from the server to the device.
RIM-Push (Push a URL and or shorcut to a device for possible offline viewing)
PAP-Push (Same as above, BUT allows for message confirmation of delivery)
WebServices: Use Visual Studio 2008 to create a Web Services only project. I'm unclear if this is supported anymore, or if MDS-IS is used. This is a 2 way operation.
MDS-IS: Use Eclipse to develop the applications and Java is required, I am unsure of what client libraries are able to do (2 way operation).
Let me know if missing any option, or if I need to revise my understanding of the basics
This is actually quite a broad question - there are a lot of ways for a BlackBerry device to interact with a server, and the ideal way depends heavily on your specific application.
Generally:
MDS Studio: the Visual Studio 2008 method is no longer actively supported. It was a mostly visual forms designer, but not a great solution for a lot of reasons. Basically, don't use it.
Mobile Web: Depending on what you need to do, this might work. Everything server side. The thing to be aware of (and why this doesn't work for a lot of problems) is that prior to BlackBerry 6 the browser wasn't very capable, either in terms of HTML support or JavaScript support. You can, however, create shortcuts to mobile web sites which appear as icons on the BlackBerry homescreen. And you can use BlackBerry Web Signals: http://na.blackberry.com/eng/developers/browserdev/websignals.jsp to push changes to mobile web pages to devices
Widgets: Supported in BlackBerry OS 5 and higher. http://na.blackberry.com/eng/developers/browserdev/widgetsdk.jsp These are packaged much like 'native' BlackBerry apps, but written using HTML and JavaScript. Can be a good option for rapid development if you're ok with just targeting OS 5, and with the limitations of the BlackBerry Browser. Lots of JavaScript hooks to system functions, so they give you more functionality than pure mobile web, and you can distribute them through App World and the like.
Java Apps: Most development work, but you get potentially the best user experience, and the most functionality. Too broad a topic to cover adequately here, but there are a lot of questions related to these on SO and elsewhere. A good starting point is the BlackBerry developer zone: http://www.blackberry.com/developers RIM Push and PAP Push both fall into this category, as they interact with Java apps, though there are other methods that you can use to achieve similar results (client polling, or I've even seen SMS or email used to push info or wake up a client, though this isn't recommended by RIM).
For a deeper overview of the options I'd recommend a book by my friend John Wargo, called BlackBerry Development Fundamentals. He covers the various options pretty well.