I am using this to post a form:
c := tidhttp.Create(self);
p := tstringlist.Create;
p.Add('field=value');
s := c.Post('http://www.test.com/action.php', p);
p.Free;
c.Free;
Tried hosting online and tried Xampp, I get the same results, no data received.
The form:
<form action="action.php" method="POST" name="test">
<input type="text" name="field"><br>
<input type="submit">
</form>
There are so many answers suggesting that I am doing it right but the data is not received
Tried TStringSTream.create(UTF8String('field=value')) but nothing
Your HTML webform is configured to submit its data to http://www.test.com/action.php (assuming the HTML page resides at http://www.test.com/).
Your Indy code is Post()ing its data to http://www.test.com instead. You need to use the correct URL when posting data in code:
s := c.Post('http://www.test.com/action.php', p);
If I had to guess (and you should never make people guess), when you are Post()'ing to http://www.test.com, you are likely getting redirected to http://www.test.com/ (notice the trailing slash). If the redirect uses reply code 303, or uses 302 and you have the hoTreat302Like303 option enabled in the TIdHTTP.HTTPOptions property, then TIdHTTP wwill send a GET instead of a POST to the new URL being redirected to.
Related
I'm calling an API URL with some parameters according to the documentation provided.
The way the API is set up, the response should be an auto-download of a .pdf file.
The parameters are:
number - which stands for a 13 digit number that represents the unique file indicator in their system (example: 5277110610029)
username and user_pass respectively - which stand for the login credentials to access the system
client_id - which stands for a unique client ID associated with my account in their system
language - where I indicate the language the file contents should be in (English or other of the available languages)
Documentation indicates it should be made as a Post request.
I have the following code:
var
FileName : string;
URL: String;
Params: TStringList;
memStream: TMemoryStream;
...
begin
FileName:= MainModule.PDFfileName + '.pdf';
begin
URL := 'https://someURL/view_integrated_pdf.php?';
Params := TStringList.Create;
memStream := TMemoryStream.Create;
try
Params.Add('number='+MainModule.number+'');
Params.Add('username='+MainModule.User+'');
Params.Add('client_id='+MainModule.clientID+'');
Params.Add('user_pass='+MainModule.Pass+'');
Params.Add('language=en');
MainModule.IdHTTP.Post(URL, Params, memStream);
finally
memStream.SaveToFile(ServerModule.FilesFolderPath+'\pdfs\'+FileName);
Params.Free;
memStream.Free;
end;
end;
pdfForm.ShowModal();
end;
If I try the resulting URL and parameters in the browser - it auto-downloads the pdf file the API gives me with the name numberParameter.pdf
If I do it in Delphi using the provided code, 8 out of 10 times it saves a pdf file with a 1 KB Size (normally it is between 32 and 100 for the successful file) and when I try to open it in the program using my pdfForm and subsequent viewer, the viewer throws an error "Invalid PDF structure"
What am I doing wrong? / How do you properly save a Post requests that returns a .pdf file to download from the API?
UPDATE
As per the comments, opening up the 1kb resulting PDF in Notepad++ displays the contents as simply Error username.
This is puzzling since I checked and the username being passed is accurate + if I paste the exact same URL with parameter values filed, in the browser, it works perfectly and gives me the correct PDF.
Is my code not the correct approach to Post and save the file being sent?
If I try the resulting URL and parameters in the browser - it auto-downloads the pdf file the API gives me with the name numberParameter.pdf
The only way to do that in a browser is to put the parameters in the URL's query component, eg:
https://someURL/view_integrated_pdf.php?number=...&username=...&client_id=...&user_pass=...&language=en
... and not in a POST body, as your code is doing. Also, a browser would send a GET request, not a POST request. Unless you are filling in an HTML webform (ie <form action="<URL>" method="POST" ...>) and submitting it. Is that what the API documentation says to do?
Since you did not provide any details from the documentation about what this server is actually expecting, we can't really tell you whether your code is correct or wrong. But it does seem that your manual tests are contradicting what you claim the documentation says the server is expecting.
If I do it in Delphi using the provided code, 8 out of 10 times it saves a pdf file with a 1 KB Size (normally it is between 32 and 100 for the successful file) and when I try to open it in the program using my pdfForm and subsequent viewer, the viewer throws an error "Invalid PDF structure"
From comments, you say your file is receiving a textual error message. TIdHTTP would save such text to your TMemoryStream ONLY IF either:
the HTTP server is NOT reporting an error at the HTTP level, ie it is sending the text message using an HTTP 2xx success response. I suspect this is what is happening in your case. By default, if the server uses a proper HTTP error code, TIdHTTP will raise an EIdHTTPProtocolException containing the text message and NOT save the text to your TMemoryStream at all.
the HTTP server IS reporting an error at the HTTP level, but you are using the hoNoProtocolErrorException and hoWantProtocolErrorContent flags together in the TIdHTTP.HTTPOptions property. In which case, TIdHTTP would not raise EIdHTTPProtocolException and would instead save whatever data the server sends as-is to your TMemoryStream.
Since there is clearly no HTTP exception being raised, you will have to validate the server's response before you can use the downloaded data in your pdfForm, ie by looking at the TIdHTTP.Response.ContentType and/or TIdHTTP.Response.ContentDisposition property to know whether the server actually sent a PDF file or not.
This is puzzling since I checked and the username being passed is accurate + if I paste the exact same URL with parameter values filed, in the browser, it works perfectly and gives me the correct PDF.
Well, for one thing, you have a typo in your code: the numberr field needs to be number instead.
Beyond that, putting the URL in a browser is NOT the same operation that your code is doing, so try changing your code to mimic what you are doing manually, eg:
uses
..., IdGlobalProtocols, IdHTTP, IdURI;
...
var
URL : string;
memStream: TMemoryStream;
begin
// All parameters into the URI for a HTTP GET request
URL := 'https://someURL/view_integrated_pdf.php'
+ '?number=' + TIdURI.ParamsEncode(MainModule.number)
+ '&username=' + TIdURI.ParamsEncode(MainModule.User)
+ '&client_id=' + TIdURI.ParamsEncode(MainModule.clientID)
+ '&user_pass=' + TIdURI.ParamsEncode(MainModule.Pass)
+ '&language=en';
memStream := TMemoryStream.Create;
try
MainModule.IdHTTP.Get(URL, memStream);
// Is it really PDF? Other formats such as plaintext is not wanted.
if not IsHeaderMediaType(MainModule.IdHTTP.ContentType, 'application/pdf') then Exit;
memStream.SaveToFile(ServerModule.FilesFolderPath + '\pdfs\' + MainModule.PDFfileName + '.pdf');
finally
memStream.Free;
end;
pdfForm.ShowModal;
end;
If that does not work, then please update your question to provide the actual documentation.
please note that i had asked this question a while back but i was unable to attend to it because of some personal issue, so it got deleted, meta user suggested i reask the question and so here it is
Recently i encountered a website which doesn't send any dynamic information when you try to login, not even Username and Password !, what i mean is that after i use Fiddler and Http Analyzer i can see that 4 event happens :
All of these happens in the web browser :
1- A simple Get for the login main page
2- A Post which sends some data (These data do not include my Username and Password and they seem static! and even though there is a __USERCONTROLPATH which can be extracted from the step 1 response, it doesn't change, meaning even if i try to login multiple times using different browser at different times everything including the __USERCONTROLPATH is the same)
3- A Post which is of JSON type and the request is empty and the response only contains a single line as you can see at the image below:
4- A Get happens which is the login main page but if you look at HTML you can see that the user is logged in!
I tried doing these steps one by one, but i am unable to successfully log in to the website and i am unable to understand how my Username and Password is sent!
Here you can see some of the code i have written:
//for the sake of cookies and ... i do a Get on the main page
idHttp.get('MainLogPage');
//i send the first post which contains some information which never changes, they seems static and do not include the username and password
idhttp.post('someURL', requestList);
jsonRawRequest := TJSONObject.Create;
jsonRequest := TStringStream.Create(jsonRawRequest.ToString, TEncoding.UTF8);
/here i try to do the post that you saw in the image, no value is sent so i send an empty json request.
idhttp.post('someOtherURL', jsonRequest)
In the end my question is how can i login to such a site ?, what am i missing ?, currently i get error on Step 3, when i post the JSON, here is a picture of the error:
Update 01 :
I meant to update the whole post and explain every step in more detail but as i was messing with fiddler and checking the posts again i noticed something, when the Browser sends the POST "https://www.somewebsite.com//Index.aspx/LoginUser" the type is application/json and in the "HTTP analyzer" i can see that it send and receive a JSON but in the Fiddler the Webforms is empty! so i check the TextView thats where i saw it !, i again went and check HTTP analyzer's Raw stream, and in there i also saw a user and password being sent, i don't know how the post contains this string that you can see below, but in fiddler you can see it in "TextView" and in http analyzer you can see it in raw stream, now that i know this, can you tell me how can i imitate such a post ?, by the way the user and password are encrypted which i presume is a matter of its own, and i probably have to contact the site's admin for the algorithm ?, anyway, first things first, how can i send such a POST ?
In Fiddler :
HTTP Analyzer :
Update 02:
I tried what Remy Lebeau said,
But the JSON in the site was not standard as you can see in:
The input is not a valid Base-64 string Error, My JSON uses double quotation marks instead of single quotations
After that i tried a hard coded solution and i read the JSON from a file as you can see below:
loader := TStringList.Create;
loader.LoadFromFile('jsonWithCustomFormat.txt');
jsonRequest := TStringStream.Create(loader[0], TEncoding.UTF8);
After that i set the refer to the login page:
idHttp.Request.Referer := 'myLoginPageURL';
idHttp.Request.ContentType := 'application/json'
And Finally i send the POST:
idhttp.Post(URL, jsonRequest, ms);
The result of what i did was an alert error as you can in the image below:
I am very confused as to what the problem is, i have to solve this so any hint or help is much appreciated.
i have already successfully logged in and posted data and downloaded data from 2 sites using idHttp.post(), but i am having trouble with the third one
in this new site, the login does work, but when i try to download a file (which uses __doPostBack for the download link) i get redirected to an error page
i have double and triple checked all post datas and they are exactly like what http analyzer showed me the only difference i noticed is that in my other attempts and sites that i accessed successfully,__viewstate is the same each time, and it never changes but in this third site it changes with each login (i mean when i go to the site manually and i check http analyzer results i can see that __viewstate parameter value differs each time)
what should i do ? is the problem with that changing __viewstate paramter ? if so how can i fix it ?
the code i use for posting :
try
Response := TMemoryStream.Create;
try
Request := TStringList.Create;
try
Request.Assign(TATDFileUtility.convertPairValueToRequestList(TATDFileUtility.extractPairValue('the site login parameters.txt', 3)));
IdHTTP := TIdHTTP.Create;
try
IdHTTP.AllowCookies := True;
IdHTTP.HandleRedirects := True;
IdHTTP.Request.ContentType := 'application/x-www-form-urlencoded';
IdHTTP.Post('thesite, the address for the login and main page and download page is the same', Request, Response);
Response.SaveToFile('responseCode0.txt');
Request.Clear;
Response.Clear;
Request.Assign(TATDFileUtility.convertPairValueToRequestList(TATDFileUtility.extractPairValue('httpDownloadParamters.txt', 3)));
IdHTTP.Post('thesite, the address for the login and main page and download page is the same', Request, Response);
Response.SaveToFile('responseCode1.txt');
as you can see after checking repsonsecode0, i can see that i am logged in, but the second response code shows me an error and tracking it shows that i am getting redirected to an error page.
ViewState is dynamic. You need to first GET the HTML page that defines the <form> element that normally submits the PostBack in a browser. This allows the webserver to generate the current ViewState. Then parse the HTML to extract the names and values of the <input> elements within the <form>, including the ViewState, and then you can POST those values to the URL specified in the <form>'s action attribute. This is what a web browser normally does, and what you need to simulate with TIdHTTP.
I'm developing a simple application to "talk" to the Amazon MWS API. Because a lot of existing code is at play here, I need to get this done in Delphi 2010 with Indy 10 (10.5.5) components, which I have used successfully to integrate with many other APIs in the past. However, the Amazon API seems to be incredibly sensitive to the smallest of details, to the point that all my calls are being denied with the already infamous "SignatureDoesNotMatch" error message.
Here's what I have accomplished so far:
1) My app will assemble a request, sign it with HMAC-SHA256 (using the OpenSSL libraries) and send it to the Amazon server endpoint.
2) The HMAC signature alone proved to be a challenge in itself, but it's now working correctly 100% of the time (as verified against requests generated by the Amazon Scrachpad).
However, as I pointed out earlier, my requests are always rejected by the MWS server with the SignatureDoesNotMatch error, even though they are verifiably correct. The only thing I can think of that could be causing problems is the way Indy may be handling the POST requests, specifically the text encoding process.
Has anyone been successful in connecting a Delphi/Indy client to MWS? If so, what kind of TIdHTTP settings were used? Here's what I have:
procedure TAmazon.TestGetOrder(OrderID:String);
const AwsAccessKey = 'MyAccessKey';
AwsSecretKey = 'MySecretKey';
MerchantID = 'MyMerchantID';
MarketplaceID = 'MyMarketplaceID';
ApiVersion = '2013-09-01';
CallUri = '/Orders/2013-09-01';
var HTTP:TIdHTTP;
SSL:TIdSSLIOHandlerSocketOpenSSL;
SS:TStringStream;
Params:TStringList;
S,Timestamp,QueryString,Key,Value:String;
i:Integer;
begin
HTTP:=TIdHTTP.Create(nil);
SSL:=TIdSSLIOHandlerSocketOpenSSL.Create(nil);
Params:=TStringList.Create;
try
Params.Delimiter:='&';
Params.StrictDelimiter:=True;
// HTTP Client Options
HTTP.HTTPOptions:=HTTP.HTTPOptions+[hoKeepOrigProtocol]-[hoForceEncodeParams];
HTTP.ConnectTimeout:=5000;
HTTP.ReadTimeout:=20000;
HTTP.ProtocolVersion:=pv1_1;
HTTP.IOHandler:=SSL;
HTTP.HandleRedirects:=True;
HTTP.Request.Accept:='text/plain, */*';
HTTP.Request.AcceptLanguage:='en-US';
HTTP.Request.ContentType:='application/x-www-form-urlencoded';
HTTP.Request.CharSet:='utf-8';
HTTP.Request.UserAgent:='MyApp/1.0 (Language=Delphi)';
HTTP.Request.CustomHeaders.AddValue('x-amazon-user-agent',HTTP.Request.UserAgent);
// generate the timestamp per Amazon specs
Timestamp:=TIso8601.UtcDateTimeToIso8601(TIso8601.ToUtc(Now));
// we can change the timestamp to match a value from the Scratchpad as a way to validate the signature:
//Timestamp:='2014-05-09T20:32:28Z';
// add required parameters from API function GetOrder
Params.Add('Action=GetOrder');
Params.Add('SellerId='+MerchantID);
Params.Add('AWSAccessKeyId='+AwsAccessKey);
Params.Add('Timestamp='+Timestamp);
Params.Add('Version='+ApiVersion);
Params.Add('SignatureVersion=2');
Params.Add('SignatureMethod=HmacSHA256');
Params.Add('AmazonOrderId.Id.1='+OrderID);
// generate the signature using the parameters above
Params.Add('Signature='+GetSignature(Params.Text,CallUri));
// after generating the signature, make sure all values are properly URL-Encoded
for i:=0 to Params.Count-1 do begin
Key:=Params.Names[i];
Value:=ParamEnc(Params.ValueFromIndex[i]);
QueryString:=QueryString+Key+'='+Value+'&';
end;
Delete(QueryString,Length(QueryString),1);
// there are two ways to make the call...
// #1: according to the documentation, all parameters are supposed to be in
// the URL, and the body stream is supposed to be empty
SS:=TStringStream.Create;
try
try
Log('POST '+CallUri+'?'+QueryString);
S:=HTTP.Post('https://mws.amazonservices.com'+CallUri+'?'+QueryString,SS);
except
on E1:EIdHTTPProtocolException do begin
Log('RawHeaders='+#$D#$A+HTTP.Request.RawHeaders.Text);
Log('Protocol Exception:'+#$D#$A+StringReplace(E1.ErrorMessage,#10,#$D#$A,[rfReplaceAll]));
end;
on E2:Exception do
Log('Unknown Exception: '+E2.Message);
end;
Log('ResponseText='+S);
finally
SS.Free;
end;
// #2: both the Scratchpad and the CSharp client sample provided by Amazon
// do things in a different way, though... they POST the parameters in the
// body of the call, not in the query string
SS:=TStringStream.Create(QueryString,TEncoding.UTF8);
try
try
SS.Seek(0,0);
Log('POST '+CallUri+' (parameters in body/stream)');
S:=HTTP.Post('https://mws.amazonservices.com'+CallUri,SS);
except
on E1:EIdHTTPProtocolException do begin
Log('RawHeaders='+#$D#$A+HTTP.Request.RawHeaders.Text);
Log('Protocol Exception:'+#$D#$A+StringReplace(E1.ErrorMessage,#10,#$D#$A,[rfReplaceAll]));
end;
on E2:Exception do
Log('Unknown Exception: '+E2.Message);
end;
Log('ResponseText='+S);
finally
SS.Free;
end;
finally
Params.Free;
SSL.Free;
HTTP.Free;
end;
end;
If I assemble a GetOrder call in Scratchpad, then paste the timestamp of that call into the code above, I get EXACTLY the same query string here, with the same signature and size, etc. But my Indy request must be encoding things differently, because the MWS server doesn't like the call.
I know MWS is at least "reading" the query string, because if I change the timestamp to an old date, it returns a "request expired" error instead.
Amazon's tech support is clueless, posting a message every day with basic stuff like "Make sure the secret key is correct" (as if getting a signature with HMAC-SHA256 and MD5 would work without a valid key!!!!).
One more thing: if I use Wireshark to "watch" the raw request from both the code above and the C-Sharp Amazon sample code, I can't tell a difference either. However, I'm not sure Wireshark makes a distinction between UTF-8 and ASCII or whatever encoding the text being shown has. I still think it has to do with bad UTC-8 encoding or something like that.
Ideas and suggestions on how to properly encode the API call to please the Amazon gods are welcome and appreciated.
Found the problem: Indy (and Synapse too) adds the port number to the "Host" header line, and I had not realized that extra bit until I watched the headers more closely with Fiddler (thanks, #Graymatter!!!!).
When I change the endpoint to be mws.amazonservices.com:443 (instead of just mws.amazonservices), then my signature is calculated the same way as the AWS server's, and everything works perfectly.
I have a issue when using TWebBrowser to Navigate and post data to a web server.
What happens is that when I initially setup the post data and navigate to the page all the information passes over fine. After filling out some information and clicking a button to process a request, the server attempts to process the request but everything just freezes..
At this screen.. The odd thing is that out of the thirty or so times I have tried this two times it has worked. ( gone through without freezing ) So i'm not sure what would be accounting for the inconsistencies.
Now If I generate this data and send it to the server via a get request just putting everything in the link it works every time.. On Chrome IE etc..
I thought that maybe the issue was with the post data so I tried to do a get request using the TWebBrowser but it still froze on me.
Browser.WebBrowser1.Navigate('https://test.com/ts/testAPI.asmx/Process?'+EncodedDataString);
Also while in the TWebBrowser - if I hit the start over button which clears most of the post/get data and then enter it all in myself and then process the information it works every time without freezing.
So the only thing I can come up with is that the freezing only seems to occur when sending post/get data via the TWebBrowser... But works on the TWebBrowser when no data is sent.
I'm assuming the server is trying to run some sort of jquery or javascript when it freezes but i'm not sure.
Below is my code for reference.
EncodedDataString := 'username='+FileForm.Firm.FieldByName('USER').AsString+
'&password='+FileForm.Firm.FieldByName('PASS').AsString+'&action='+action+
'&token='+eToken+'&retname='+HTTPEncode(MaskEditRetName.Text)+
'&retaddress1='+HTTPEncode(MaskEditRetAddr.Text)+
'&retcity='+HTTPEncode(MaskEditRetCity.Text)+
'&retstate='+HTTPEncode(MaskEditRetState.Text)+
'&retzip='+HTTPEncode(MaskEditRetZip.Text)+
'&accountno='+HTTPEncode(MaskEditAccount.Text)+
'&amount='+HTTPEncode(MaskEditAmount.Text)+
'&date='+HTTPEncode(MaskEditDate.Text)+
'&paymenttype='+HTTPEncode(paymentType);
PostData := VarArrayCreate([0, length(EncodedDataString)-1], varByte);
for i := 1 to length(EncodedDataString) do
PostData[i-1] := ord(EncodedDataString[i]);
Headers := 'Content-type: application/x-www-form-urlencoded'#10#13;
Application.CreateForm(TBrowser,Browser);
Browser.WebBrowser1.Silent := True;
Browser.WebBrowser1.Navigate('https://test.com/ts/testAPI.asmx/Process',
EmptyParam, EmptyParam, PostData, Headers);