I got an eccentric problem. I am trying to automate visiting a web site by using WebRequest and WebClient. I have observed all the post request header key-value pairs and posted data string in Firebug(request Header and Post tab). Then I simulated such request with WebRequest and put all the header parameter and posted data there. However when I do GetResponse() from this request instance, I always got an error page back that says some sessionID is short of.
Actually I have taken care to put previously(first step to open the Logon page) responded session cookie in the Header's cookie field for the request. And I can get the correct response back by simulating requesting the logon page(the 1st page), but cannot get through this authentication page. My post data is like userid=John&password=123456789&domain=highmark.And the authentication page request that carried out by browser succeeds every time.
Am I missing something in the request that may not be shown by firebug.If yes, can you give me some recommendation for the tools that may examine the entire request sent by browser.
I have solved this issue. The problem is I set the httpWebRequest instance's AllowAutoRedirect=true. Thus the effect is when I got the first response from the server, the httpWebRequest would continually to make another request asking for a different url that is replied in the response header's Location field.
The defect of HttpWebRequest class is when it is getting redirected, it does not include the Set-Cookies(Response's Header Field)'s cookies in the next request header, thus the server would deny such page request and may redirect again to another different page.
And the httpWebRequest.GetResponse() method only return the last responsed page back under the setting AllowAutoRedirect=true. And I got the totally different response than I expected.
Also in this solving process, I need to thank to a distinguish Http Traffic examining tool:IEInspector Http Analyzer(http://www.ieinspector.com/httpanalyzer/). The great feature of this tool is it can examine not only the http traffic from browser but also what your process's httpWebRequest made. And also it can display in text format the raw stream of those request and response. Although it is a commercial software, you can try it for 15 days. I am quite happy with what it tells me(in well-formed details) and I like to buy it as well.
Related
Auth 2.0.
"code" parameter is required to perform
POST /.../oauth2/v2.0/token
with code value.
In Fiddler code value could be found in Location header of response to /kmsi request:
However, here is no Location header in JMeter for the same request:
Why? Are there any tip to get Location header in JMeter too?
If you're seeing different response it means that
Either you're sending a different request. In this case inspect request details from JMeter and from the real browser using a 3rd-party sniffer tool like Fiddler or Burp, identify the inconsistencies and amend your JMeter configuration so it would send exactly the same request as the real browser does (apart from dynamic values which need to be correlated)
Or one of the previous requests fails somewhere somehow, JMeter automatically treats HTTP responses with status codes below 400 as successful, it might be the case they are not really successful, i.e. you're continuously hitting the login page (you can check it by inspecting response data tab of the View Results Tree listener). Try adding a Response Assertions to the HTTP Request samplers so there will be another layer of explicit checks of the response data, this way you will get confidence that JMeter is doing what it is supposed to be doing.
I have a Jmeter script where during replay, Post request is displaying as Get request and the parameters in the request are not sent to the server. Due to this, correlations are failing at this request.
One of the parameters in the request is ViewState with so many characters. Is this large parameter value causing the above issue? How to proceed now?
Most probably you're sending a malformed request therefore instead of properly responding to a POST request you're being redirected somewhere (most probably to Login page)
Use View Results Tree listener in HTML or Browser mode to see what page you're hitting in the reality
With regards to the ViewState, "so many characters" is not a problem, the problem is that these are not random characters. ViewState is being used for client-side state management and if you fail to provide the proper value you won't be able to move further so you need to design your test as follows:
Open first page
Extract ViewState using a suitable Post-Processor
Open second page
here you need to pass viewstate from the step 1 along with other parameters
More information: ASP.NET Login Testing with JMeter
Also don't forget to add HTTP Cookie Manager to your Test Plan
What I'm able to understand is the request may be getting redirected. This happens usually when the server expects a unique request. If you recorded the request, you may be possibly using older headers that carry old cookie information. Check your headers and then reconstruct the request.
Make sure you are not using old cookies anywhere. remove that cookie part from HTTP Header Manager everywhere.
I am attempting to use the REST api of Jenkins. Jenkins requires a POST request to a URL to delete a job. This results in the following:
I tell my chosen Client to send a POST to the appropriate URL.
The client sends a POST and authorizes itself with username and password.
Jenkins deletes the job.
Jenkins returns a "302 - Found" with the location of folder containing the deleted job.
Client automatically sends a POST to the location.
Jenkins answers with "200 - OK" and the full HTML of the folder page.
This works just fine with Postman (unless I disable "Automatically follow redirects" of course).
Jersey however keeps running into a "404" at step 5 because I blocked anonymous users from viewing the folder in question. (Or a "403" if I blocked anonymous users altogether.)
Note that the authentication works in step 1 because the job has been deleted successfully!
I was under the impression that Jersey should use the given authentication for all requests concerning the client.
Is there a way to actually make this true? I really don't want to forbid redirects just to do every single redirect myself.
To clarify: The problem is that while Jersey follows the redirect, but fails to authenticate itself again, leading to the server rejecting the second request.
Code in question:
HttpAuthenticationFeature auth = HttpAuthenticationFeature.basicBuilder()
.credentials(username, token)
.build();
Client client = ClientBuilder.newBuilder()
.register(auth)
.build();
WebTarget deleteTarget = client.target("http://[Jenkins-IP]/job/RestTestingArea/job/testJob/doDelete")
Response response = deleteTarget.request()
.post(null);
EDIT: The "302-Found" only has 5 headers according to Postman: Date, X-Content-Type-Options ("nosniff"), Location, Content-Length (0) and Server. So neither any cookies nor any tokens that Postman might use and Jersey disregard.
Question loosely related to this one - if I were able to log the second request I might be able to understand what's happening behind the scenes.
EDIT2: I have also determined that the problem is clearly with the authentication. If I allow anonymous users to view the folder in question, the error disappears and the server answers with a 200.
I found the answer with the help of Paul Samsotha and Gautham.
TL;DR: This is intended behavior and you have to set the System property http.strictPostRedirect=true to make it work or perform the second request yourself.
As also described here, HttpURLConnection decided to not implement a redirect as it is defined in the HTTP standard but instead as many browsers implemented it (so in laymans terms, "Do it like everyone else instead of how it is supposed to work"). This leads to the following behavior:
Send POST to URL_1.
Server answers with a "302 - Found" and includes URL_2.
Send GET to URL_2, dropping all the headers.
Server answers with a "404 - Not Found" as the second request does not included correct authentication headers.
The "404" response is the one received by the code, as steps 2 and 3 are "hidden" by the underlying code.
By dropping all headers, the authentication fails. As Jersey uses this class by default, this lead to the behavior I was experiencing.
I have a software that was developed in delphi and that makes a http get request to a specific url that I want to redirect to another url.
I have the source code but I can't recompile it because it wasn't developed by me, so it would take too much effort and time, and right now I can't afford it.
Anyway, back to my problem.
I tried using fiddler 2. It worked, but only for when I try accessing the url via browser. When my application sends the requests, it doesn't get redirected to the new url.
Does anyone have any other suggestion of what I can do?
I'm using Ruby on Rails. Here is the requirement: the client (a native mobile app developed by me) will send a http post request to my Ruby code, my code will add some extra http headers (based on some business logic), then I need to "forward" or "redirect" this post request to another backend server (which has a REST service) and return its response back to the client.
I have been able to write a rack middleware to intercept the post request and add the extra headers. Originally I thought I could just use http redirect (status code: 307 for post request). But the problem is that the extra headers could NOT be submitted, which is the whole point of my code. So this isn't http redirect or forwarding per se, it's more like transforming a request.
I'm able to make a separate post request from my code using net http. This works. But I have to COPY data from the incoming request to my outgoing request (eg form data, http headers). This copying seems a bit tedious.
I would prefer some kind of simple "repackaging" (which is akin to http redirect or forwarding), that is I copy the whole incoming request to the outgoing request, slap on the extra headers and send it to the destination URL and be done with. I am not sure how to do this, and if doing it this way is even a good idea. For example, HTTP_USER_AGENT shows the OS, browser type of the client, when I'm making a new request, I probably don't need to send this on.
Alternatively, I can copy only the application specific data, because they're all the backend server (the destination of this "redirect") cares about. But I'm averse to hardcoding attributes in my code, causing close-coupling with the client (our native mobile app). Ideally I only copy application-specific data without hardcoding their attribute names. Is this possible? If so, how?
Any advice would be appreciated.
Thank you.
HTTP does not allow redirects for anything other than GET request.
(This is not technically correct but using HTTP 307 is kind of sketchy - see https://softwareengineering.stackexchange.com/questions/99894/why-doesnt-http-have-post-redirect)
If you need too send a POST request to another server for processing then using a proxy as you already seem to be doing is the correct solution.
Recreating the request in the proxy may seem tedious but it actually serves as a guarantee that you are calling the other servers "API" correctly.
While you can simply loop through the request headers:
uri = URI('http://www.example.com/todo.cgi')
req = Net::HTTP::Post.new(uri)
request.headers.each do |key, value|
req[key] = value
end
And pass the request form data:
req.set_form_data = request.request_parameters
You should ask yourself if it really is prudent to proxy everything.
See http://api.rubyonrails.org/classes/ActionDispatch/Request.html