I am trying to check the incomming request to my server. Another server which has hosted MVC application. An action method is sending some data to my server. I am using Fiddler. But somehow it is not showing the incoming request.
Below mentioned are my settings in Fiddler Custom Rules..
static function OnBeforeRequest(oSession: Session) {
if (oSession.host.toLowerCase() == "IP Address:8888")
oSession.host = "IP Address:82";
}
Below mentioned are my Fiddler Options.
Am I missing anything ?
It sounds like you're trying to use Fiddler as a reverse proxy. You should read the steps at http://www.fiddler2.com/r/?reverseproxy. The biggest thing to understand is that when running as a reverse proxy, you only see traffic in Fiddler if the client's URL is changed to point at Fiddler.
If it is ssl connection then you need to enable option 'capture https connection' from 'https' tab. Did you try to invoke request from other browser like chrome ? Does fiddler capture anything?
You don't need custom rule for this case. It should work if you enable these settings. I have faced only some problems in other browsers like FF.
I'm not sure I can answer your question fully without knowing a few additional pieces of information.
If the request being made is not a HTTP request, Fiddler will not be able to handle it.
Also, if you're using the loopback address localhost then Fiddler may not be able to find it.
Related
From the Electron renderer, I am accessing a local GraphQL endpoint served by a Django instance on my computer, which I'd like to do over HTTP, not HTTPS. But Electron's Chromium seems to intercept my fetch request and preemptively return a 307 redirect.
So if my fetch request is POST to http://local.myapp.com:3000/v1/graphql, then Chromium returns a 307 and forces a redirect to https://local.myapp.com:3000/v1/graphql, which fails because my server is listening on port 3000 and for my use case I can't do a local cert for local.myapp.com.
Theoretically the first insecure request should be hitting an nginx docker container listening on port 3000 without any SSL requirement. And nginx is proxying the request to a Hasura container. But I'm not even seeing the requests in the nginx access logs, so I'm pretty sure the request is being intercepted by Chromium.
I believe this StackOverflow comment summarizes well why this is happening: https://stackoverflow.com/a/34213531
Although I don't recall ever returning a Strict-Transport-Security header from my GraphQL endpoint or Django server.
I have tried the following code without success to turn off this Chromium behavior within my Electron app:
import { app, } from 'electron'
app.commandLine.appendSwitch('ignore-certificate-errors',)
app.commandLine.appendSwitch('allow-insecure-localhost', )
app.commandLine.appendSwitch('ignore-urlfetcher-cert-requests', )
app.commandLine.appendSwitch('allow-running-insecure-content', )
I have also tried setting the fetch options to include {redirect: 'manual'} and {redirect: 'error'}. I can prevent the redirect but that doesn't do me any good because I need to make a successful request to the endpoint to get my data.
I tried replacing the native fetch with electron-fetch (link) and cross-fetch (link) but there seems to be no change in behavior when I swap either of those out.
Edit: Also, making the request to my GraphQL outside of Electron with the exact same header and body info works fine (via Insomnia).
So I have a couple of questions:
Is there a way to programmatically view/clear the list of HSTS domains that is being used by Chromium within Electron?
Is there a better way to accomplish what I'm trying to do?
I think the issue might be from the server, most servers don't allow HTTP in any possible way, they'll drop the data transfer and redirect you to HTTPS and there's a clear reason why they would do that.
Imagine you have an app that connects through HTTPS to send your API in return for some data, if someone just changed the https:// to http:// that'd mean the data will be sent un-encrypted and no matter what you do with your API key, it'll be exposed, that's why the servers don't ever allow any HTTP request, they don't accept even a single bit of data.
I could think of two solutions.
Chromium is not the reason for the redirect, our Django instance might be configured as production or with HTTPS listeners.
Nginx might be the one who's doing the redirecting (having a little bit of SSL def on the configuration)
Last but not least, just generate a cert with OpenSSL (on host http://local.myapp.com:3000/) note: include the port and use that on your Django instance. You can trust the certificate so that it could work everywhere on your computer.
I am trying to make a request to a website with a proxy using httparty like so:
def self.fetch_page_with_html_response(url, proxy_id)
proxy = Proxy.find(proxy_id)
request_options = {
http_proxyaddr: proxy.url, http_proxyport: proxy.port, http_proxyuser: proxy.username, http_proxypass: proxy.password,
headers: {"User-Agent" => proxy.user_agent}
}
response = HTTParty.get(url, request_options)
response
end
On certain websites my requests either hangs or returns an error page where the website is blocking me from fetching the page.
When i use these same proxy settings in my Chrome browser using an extension like SwitchyOmega the requests goes through fine and the page loads.
Is there any reason why the request would be getting blocked from my web server but not through my browser?
I even tested using the same user agent and providing the same exact headers my browser is sending.
There could be some reasons in your case.
Can you check if proxy works correctly?
Please send get request to https://api.ipify.org/ using your proxy in code.
If it returns correct ip address, then the proxy works.
Please disable javascript run in chrome settings. Then browse the website via proxy.
And check if website load correctly.
Because some websites render html and css using javascript.
Please feel free to reply me if you still need help.
I would like to know some connections that depart from my application.
So I use a proxy (in particular I'm using Charles for OSX) This works fine.
I have noticed that some of these connections are HTTPS (TLS). This is ok, but for these connections, I can only see the base URL and not the complete URL that is invoked. For example I can read: https://www.thewesite.com:443
I would expect not to see the body of the request but at least the header, and then also the whole URL I would expect to see it.
is that correct? is there a way to display the complete URL?
Since URL and the rest of the HTTP header are inside the encrypted connection you would need to enable SSL Proxying (that is Man-In-The-Middle attack) for the specific hosts. See http://www.charlesproxy.com/documentation/proxying/ssl-proxying/
I am trying to create an OData model in SAP UI5 this way:
new sap.ui.model.odata.ODataModel("http://services.odata.org/Northwind/Northwind.svc/");
but I am getting a 501 not implemented error!
could you please check what's wrong?
Thanks
As far as I can see it, the service is not really CORS-enabled. I have the same problem with my own examples here, as soon as I am not using some kind of proxy, I get this error.
The reason behind it is that when you send a complex request to the service, you'll autmatically have a so-called preflight request sent by your browser (before the actual GET) which is not a GET-Request, but an HTTP OPTIONS request.
All the odata.org sample services return a 501 error at the moment for such requests.
You can e.g. use the simpleProxyServlet which is shipped with UI5, or of course any other proxy which would solve this.
You are getting this error as your browser will refuse this request due to same Origin Policy. Here is what you should do:
Deploy the app on the same server or domain as the service that you want to call, so that both resources are in the same origin (if possible)
Disable the same-origin policy in the browser for local testing. Run Chrome by running Chrome with the following command:
[your-path-to-chrome-installation-dir]\chrome.exe
--disable-web-security --user-data-dir. Make sure that all instances of Chrome are closed before you run the command. This allows all web
sites to break out of the same-origin policy and connect to the
remote service directly.
-> Don't do this in your productive app as it imposes a security risk.
Using a proxy
The following documentation should help you understand this more and implement:
Conncting with oData Service
Request failing due to Same-Origin Policy sharing(CORS)
Please use "proxy/http/services.odata.org/Northwind/Northwind.svc", I think it's solve your problem!
Using fiddler I can intercept an ajax post request and with request composer I can resend the same request, causing the server to respond it normally. It doesnt matter if protocol is http or https (fiddler deciphers HTTPS traffic), with tools like Fiddler it is just possible.
On web applications side is it possible to understand or prevent such requests? How?
No. There's no way to prevent this. URLs are meant to be accessed. If it shouldn't be accessed, don't put it online or require some method of authentication. Some clients may pass an identifiable User-Agent header that can be restricted, and Fiddler probably does as well. However, the whole point of tools like Fiddler is to be able to make any type of custom request, which includes this User-Agent string. So, even if you block the default User-Agent, there's nothing stopping the user of Fiddler from changing the User-Agent to something that won't be blocked.