I'm using Apache Wink to access a service, and trying to debug a problem where the server apparently does not recieve my request in the intended format (details below, but are probably immaterial). Is there a way I can make the Wink client to log the HTTP requests that it makes to the server, so that I can see what is being sent down the wire?
Details: I'm using Eclipse Lyo to create a ChangeRequest in RTC (rational team concert) using their OSLC v2 REST APIs.(Eclipse Lyo internally uses Apache Wink). In doing so, even though I've set a "Filed Against" property in the ChangeRequest being submitted, RTC does not recognize it and complains that it is missing.
I think it's better to use a proxy to monitor the traffic. If your client runs on Windows, Fiddler is a very nice tool.
Related
I am trying to create an OData model in SAP UI5 this way:
new sap.ui.model.odata.ODataModel("http://services.odata.org/Northwind/Northwind.svc/");
but I am getting a 501 not implemented error!
could you please check what's wrong?
Thanks
As far as I can see it, the service is not really CORS-enabled. I have the same problem with my own examples here, as soon as I am not using some kind of proxy, I get this error.
The reason behind it is that when you send a complex request to the service, you'll autmatically have a so-called preflight request sent by your browser (before the actual GET) which is not a GET-Request, but an HTTP OPTIONS request.
All the odata.org sample services return a 501 error at the moment for such requests.
You can e.g. use the simpleProxyServlet which is shipped with UI5, or of course any other proxy which would solve this.
You are getting this error as your browser will refuse this request due to same Origin Policy. Here is what you should do:
Deploy the app on the same server or domain as the service that you want to call, so that both resources are in the same origin (if possible)
Disable the same-origin policy in the browser for local testing. Run Chrome by running Chrome with the following command:
[your-path-to-chrome-installation-dir]\chrome.exe
--disable-web-security --user-data-dir. Make sure that all instances of Chrome are closed before you run the command. This allows all web
sites to break out of the same-origin policy and connect to the
remote service directly.
-> Don't do this in your productive app as it imposes a security risk.
Using a proxy
The following documentation should help you understand this more and implement:
Conncting with oData Service
Request failing due to Same-Origin Policy sharing(CORS)
Please use "proxy/http/services.odata.org/Northwind/Northwind.svc", I think it's solve your problem!
I am trying to build a java application to scrape a website using HTMLUnit. After extracting some data the application encounters following exception -
java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connection to siteURL refused.
If I run application again, it is able to extract some data again before failing with the same exception. Probably the server see lot of requests from same client IP and refuses connection for a request.
Also, when application encounters this problem, I am able to connect to the site using a browser.
How can I overcome this problem? In web scraping applications how are such problems approached and resolved?
This is how I debug such issues :
Download Fiddler
By default, fiddler listens on port 8888, all you have to do, is configure webClient to use fiddler as proxy & then all requests being sent can be seen (analyzed, modified & re-sent too) in fiddler.
client.getOptions().setProxyConfig(new ProxyConfig("127.0.0.1", 8888));
From what I can say from my previous experience is that the target site blocks after some-time, you can try adding a pause or rotating proxies & user-agents. You can also try clearing cookies.
I want to test the results from a WSDL service in a browser like IE9 or FireFox. I know that I can view the WSDL XML, but I want to test the return results of an endpoint called GetEmployeeById that accepts a parameter called Id and returns a class. I am assuming this is all serialized to XML, so a browser would be a good fit for testing this. Is this possible?
In case you are using Visual Studio for .net development, I think something much better than a browser probably would to use:
wcftestclient <url>
which can be called from the command prompt and is part of the tools from visual studio.
The tool will help you to build and receive complex objects and see the results already serialized.
internet explorer let's you make a request with simple string parameters if the WSDL provides enough information.
If you haven't seen it, then probably the WSDL is only for discoverability reasons, probably just points to another service in a different transport protocol, not port 80, if the service is not on port 80 you won't be able to use your browser.
You have another more complex tool called FIDDLER that you can format any kind of http request, as well as receiving any kind of requests, like json for example.
You can use this URL to test WSDL endpoints, send request and see response.
I am using Grails Ws-Client Plugin
but my application waits for the SOAP response back from the server from which i am consuming web service and my application waits from this code
def proxy = webService.getClient(wsdlUrl)
This mostly occours when the server is down or net connection is slow.
the wait also continues in case the webservice is temporarily removed from the server and the url containing the wsdl is redirecting to home page of website when try to access on web browser.
How can i detect that the wsdl is present or not and how can i set timeout like property so that the wait for response exist for 10 seconds and then it stops waiting for response and code start executing normally in case of stall .
I also don't get any exception or error as well.
Sounds like there's no read and/or connect timeouts set on the client by default. This should help if the web service is down: proxy.setConnectionTimeout(value_in_milliseconds)
I'm not sure about setting the read timeout though, which is what you'd see if the host was up and accepting connections but the web service wasn't available or not responding. The best solution we found for this was to use the Apache Commons HTTP client instead of the default client, which gave us much more granular configuration over the client's connection settings. It's possible they're in the WS-Client plugin also, but the relevant documentation (actually the GroovyWS documentation) doesn't appear to mention anything about read timeouts.
Using a Java based image uploader and having problems, seems my controller action is not being hit if I upload multiple files. I can't use fiddler or FireBug because they don't pickup the Java apps request to the server. I need to see what is being requested in debug mode so I can fix my Route - I am assuming this is my problem.
How can I hook into this and see what the request is?
You could still use a web debugging proxy (like Fiddler) if your Java based image uploader supports a proxy. Other than that you can use a packet sniffer like Wireshark or break your debug session at a point with a HttpContext, e.g. in Global.asax at BeginRequest.