I have a simple electron app which loads a certain website.
This website is doing http get/post requests. I want to sniff the body of the server responses.
I already checked out the webRequest module which seemed pretty nice. Unfortunetely, it handles all data EXCEPT the body.
How can I sniff the body of the server responses?
Found out that this is not possible. Also its not a an Electron issue, rather a chromium one - Electron just wraps the chromium API.
Finally I managed to inject a javascript (inside the api service) which forwards me all data received from the server via ipc
https://electronjs.org/docs/api/ipc-main
Related
I'm looking to make a web extension for Firefox that stores HTML pages and other resources in local storage and serves them for offline viewing. To do that, I need to intercept requests that the browser makes for the pages and the content in them.
Problem is, I can't figure out how to do that. I've tried several approaches:
The webRequest API doesn't allow fulfilling a request entirely - it can only block or redirect a request, or edit the response after it's been done.
Service Workers can listen to the fetch event, which can do what I want, but calling navigator.serviceWorker.register in an addon page (the moz-extension://<id> domain) results in an error: DOMException: The operation is insecure. Relevant Firefox bug
I could possibly set up the service worker on a self hosted domain with a content script, but then it won't be completely offline.
Is there an API that I missed that can intercept requests from inside a web extension?
From the Electron renderer, I am accessing a local GraphQL endpoint served by a Django instance on my computer, which I'd like to do over HTTP, not HTTPS. But Electron's Chromium seems to intercept my fetch request and preemptively return a 307 redirect.
So if my fetch request is POST to http://local.myapp.com:3000/v1/graphql, then Chromium returns a 307 and forces a redirect to https://local.myapp.com:3000/v1/graphql, which fails because my server is listening on port 3000 and for my use case I can't do a local cert for local.myapp.com.
Theoretically the first insecure request should be hitting an nginx docker container listening on port 3000 without any SSL requirement. And nginx is proxying the request to a Hasura container. But I'm not even seeing the requests in the nginx access logs, so I'm pretty sure the request is being intercepted by Chromium.
I believe this StackOverflow comment summarizes well why this is happening: https://stackoverflow.com/a/34213531
Although I don't recall ever returning a Strict-Transport-Security header from my GraphQL endpoint or Django server.
I have tried the following code without success to turn off this Chromium behavior within my Electron app:
import { app, } from 'electron'
app.commandLine.appendSwitch('ignore-certificate-errors',)
app.commandLine.appendSwitch('allow-insecure-localhost', )
app.commandLine.appendSwitch('ignore-urlfetcher-cert-requests', )
app.commandLine.appendSwitch('allow-running-insecure-content', )
I have also tried setting the fetch options to include {redirect: 'manual'} and {redirect: 'error'}. I can prevent the redirect but that doesn't do me any good because I need to make a successful request to the endpoint to get my data.
I tried replacing the native fetch with electron-fetch (link) and cross-fetch (link) but there seems to be no change in behavior when I swap either of those out.
Edit: Also, making the request to my GraphQL outside of Electron with the exact same header and body info works fine (via Insomnia).
So I have a couple of questions:
Is there a way to programmatically view/clear the list of HSTS domains that is being used by Chromium within Electron?
Is there a better way to accomplish what I'm trying to do?
I think the issue might be from the server, most servers don't allow HTTP in any possible way, they'll drop the data transfer and redirect you to HTTPS and there's a clear reason why they would do that.
Imagine you have an app that connects through HTTPS to send your API in return for some data, if someone just changed the https:// to http:// that'd mean the data will be sent un-encrypted and no matter what you do with your API key, it'll be exposed, that's why the servers don't ever allow any HTTP request, they don't accept even a single bit of data.
I could think of two solutions.
Chromium is not the reason for the redirect, our Django instance might be configured as production or with HTTPS listeners.
Nginx might be the one who's doing the redirecting (having a little bit of SSL def on the configuration)
Last but not least, just generate a cert with OpenSSL (on host http://local.myapp.com:3000/) note: include the port and use that on your Django instance. You can trust the certificate so that it could work everywhere on your computer.
Is it possible to read the http request and response data from pages loading inside webview. What i want to do is get the binary data from a response after user clicks on a link inside the page in webview. Any help or clue would be greatly appreciated
Create your own URLStreamHandlerFactory initialized by URL.setURLStreamHandlerFactory which generates a URLStreamHandler that wraps the standard http and https URLStreamHandlers to intercept their traffic before forwarding.
Some of the concepts are explained in A New Era for Java Protocol Handlers whitepaper.
Another option is to listen to the WebEngine.location property and open a separate connection to a server to retrieve and process the binary data as needed. An example of this approach is the pdf handling code for the willow web browser.
Using fiddler I can intercept an ajax post request and with request composer I can resend the same request, causing the server to respond it normally. It doesnt matter if protocol is http or https (fiddler deciphers HTTPS traffic), with tools like Fiddler it is just possible.
On web applications side is it possible to understand or prevent such requests? How?
No. There's no way to prevent this. URLs are meant to be accessed. If it shouldn't be accessed, don't put it online or require some method of authentication. Some clients may pass an identifiable User-Agent header that can be restricted, and Fiddler probably does as well. However, the whole point of tools like Fiddler is to be able to make any type of custom request, which includes this User-Agent string. So, even if you block the default User-Agent, there's nothing stopping the user of Fiddler from changing the User-Agent to something that won't be blocked.
I am trying my hand in server applications using Indy Internet tools.
My client sends Post data (XML) in Unicode format.
Can I convey my preference to client (HTTP Client). I prefer Text. In general can a HTTP server send its preferences to its Clients?
Thanks for any hint or help.
The problem with this is the fact, that with only one POST the server has no way to respond, until the client has already sent the data.
The solution is to make two calls: One where the client asks for the server preferences and another to send the data. The OPTIONS HTTP method can be used for this scenario.
You can handle both requests on the same URL: If the clients makes an OPTIONS request the server responds with the configuration data. (via response headers) Then the client can make a POST request on the same URL and the server handles the data appropriately.
For further information see HTTP methods and HTTP headers, especially the Accept header.