I am using TIdHTTP from Indy. I am issuing 2 different requests, one header contains the 'Last-Modified' tag, the other one does not. The header with the tag, throws an exception:
'Invalid Argument to date encode'
I already came across this question where Remy Lebeau said, that TIdHttp is now able to parse ISO8601 dates, but it doesnt seem to work for me. As you can see below, i am not doing anything with the component other than changing the UserAgent. Am I missing something?
url := 'https://api.priceapi.com/v2/jobs/' + JobID+ '?token=' + Token;
http := TIdHTTP.Create(nil);
http.Request.UserAgent := 'XXXXX'; //Some UserAgent
try
ssl := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
try
http.IOHandler := ssl;
try
jo := TJsonObject.ParseJSONValue(http.get(url)) as TJSONObject;
result := jo.GetValue('status').Value;
finally
end;
finally
ssl.Free;
end;
finally
http.Free;
end;
Header with Last-Modified:
Cache-Control: no-cache
Content-Disposition: attachment;
filename="20181025145103_google_shopping_de_5bd1d857bbd7e520c12841d7.json"
Content-Transfer-Encoding: binary
Content-Type: application/json
Last-Modified: 2018-10-25 14:51:23 +0000
Vary: Origin
X-Accel-Buffering: no
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: b05aa8fe-7ea9-4152-8470-a75f9816549f
X-Runtime: 0.099212
X-XSS-Protection: 1; mode=block
transfer-encoding: chunked
Connection: keep-alive
Header without Last-Modified:
Cache-Control: max-age=0, private, must-revalidate', nil
Content-Type: application/json; charset=utf-8', nil
ETag: W/"43c4a8865a5ebe565f3920779a962e93"', nil
Vary: Origin', nil
X-Content-Type-Options: nosniff', nil
X-Frame-Options: SAMEORIGIN', nil
X-Request-Id: 344ac82e-0d14-4838-ae7e-627c79b78edc', nil
X-Runtime: 0.062357', nil
X-XSS-Protection: 1; mode=block', nil
Content-Length: 157', nil
Connection: Close', nil
StackTrace:
:744717d2 KERNELBASE.RaiseException + 0x62
HIWBase.System.SysUtils.ConvertError($3B68860)
HIWBase.System.SysUtils.EncodeDate(???,???,???)
HIWBase.IdGlobalProtocols.RawStrInternetToDateTime('07:53:37 +0000',0)
HIWBase.IdGlobalProtocols.GMTToLocalDateTime('07:53:37 +0000')
HIWBase.IdHTTPHeaderInfo.TIdEntityHeaderInfo.ProcessHeaders
HIWBase.IdHTTPHeaderInfo.TIdResponseHeaderInfo.ProcessHeaders
HIWBase.IdHTTP.TIdHTTPProtocol.RetrieveHeaders(255)
HIWBase.IdHTTP.TIdCustomHTTP.DoRequest('GET','My URL',nil,$ADF09E0,(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL',$ADF09E0,(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL',(...))
HIWBase.IdHTTP.TIdCustomHTTP.Get('My URL')
I am on Indy version 10.6.2.5311
The Last-Modified header is defined in RFC 2616 Section 14.29 1 as:
Last-Modified = "Last-Modified" ":" HTTP-date
1: An equivalent definition appears in RFC 7232 Section 2.2.
HTTP-date is defined in RFC 2616 Section 3.3 2 as:
HTTP-date = rfc1123-date | rfc850-date | asctime-date
rfc1123-date = wkday "," SP date1 SP time SP "GMT"
rfc850-date = weekday "," SP date2 SP time SP "GMT"
asctime-date = wkday SP date3 SP time SP 4DIGIT
date1 = 2DIGIT SP month SP 4DIGIT
; day month year (e.g., 02 Jun 1982)
date2 = 2DIGIT "-" month "-" 2DIGIT
; day-month-year (e.g., 02-Jun-82)
date3 = month SP ( 2DIGIT | ( SP 1DIGIT ))
; month day (e.g., Jun 2)
time = 2DIGIT ":" 2DIGIT ":" 2DIGIT
; 00:00:00 - 23:59:59
wkday = "Mon" | "Tue" | "Wed"
| "Thu" | "Fri" | "Sat" | "Sun"
weekday = "Monday" | "Tuesday" | "Wednesday"
| "Thursday" | "Friday" | "Saturday" | "Sunday"
month = "Jan" | "Feb" | "Mar" | "Apr"
| "May" | "Jun" | "Jul" | "Aug"
| "Sep" | "Oct" | "Nov" | "Dec"
2: Equivalent definitions appear in RFC 7231 Section 7.1.1.1.
The Last-Modified value you have shown does not match any of those formats allowed by HTTP.
TIdHTTP uses Indy's GMTToLocalDateTime() function to parse the Last-Modified (and Date and Expires) header. That function is shared by HTTP, IMAP, NNTP, and email components, so it is a little more flexible in the date/time formats that it supports. For instance, it does parse ISO 8601, which you claim the Last-Modified value is. However, the value you have shown does not actually conform to ISO 8601, either. If it had, it would have looked more like this instead:
Last-Modified: 2018-10-26T08:37:01+00:00
To make matters worse, according to the stack trace you have provided, GMTToLocalDateTime() is being called without any date portion at all:
HIWBase.IdGlobalProtocols.GMTToLocalDateTime('07:53:37 +0000')
The only way that can happen in TIdHTTP is if the HTTP server is sending a Last-Modified (or Date or Expires) header with that exact value, which is also not conformant to the HTTP or ISO 8601 standards, and is not handled as-is by GMTToLocalDateTime().
In short, the API you are querying is sending an illegal date/time format that TIdHTTP does not support parsing (which is ironic, because the main https://www.priceapi.com website does send properly formatted HTTP date/time strings). You should contact the website admin and report that their API server is violating HTTP protocol standards in this regard.
That being said, GMTToLocalDateTime() DOES NOT raise an 'Invalid Argument to date encode' exception when it encounters a malformed date/time string. It returns a TDateTime of 0.0 instead. The only way you could be seeing that exception is if you are running your code inside the debugger. When GMTToLocalDateTime() is given a malformed date/time string, it is possible that it may extract numeric components that it thinks are valid but then fails when it attempts to encode the final TDateTime with them. The exception you are seeing comes from the RTL's EncodeDate() function when it is given an invalid month/day/year as input. But GMTToLocalDateTime() catches that exception internally. Your code will never see it at runtime, only the debugger can see it.
Related
Im in east 8+ timezone, and this expression return nil on my device.
I know it returned a value counting my timezone infomation. But, Why? I don't get it. How this function implemented and what puporse of it?
Thanks.
Implemention can be found here: https://www.lua.org/source/5.1/loslib.c.html#os_time
os.time is equivalent to mktime in C, the result of the function is dependent on the OS lua is running on.
On Windows, it returns -1 if the local date is before 1970-1-1 UTC. That's why you got nil. (Since Lua 5.3, it throws an error instead). If you call this function on *inx system you can get a negative number.
I am using on Linux often Minus Epoch' for formating Dates/Timelines with os.date()
( My Birthday is before 1970 :-O )
Also os.setlocale()...
> os.setlocale('de_DE.UTF-8', 'time')
de_DE.UTF-8
> os.date('%c', os.time({year=1970, month=1, day=1, hour=0}))
Do 01 Jan 1970 00:00:00 CET
> os.date('%c', os.time({year=1945, month=8, day=6, hour=0}))
Mo 06 Aug 1945 00:00:00 CEMT
> os.date('!%c', os.time({year=1945, month=8, day=6, hour=0}))
So 05 Aug 1945 21:00:00 GMT
> os.setlocale('ja_JP.utf8', 'time')
ja_JP.utf8
> os.date('%c', os.time({year=1945, month=8, day=6, hour=0}))
1945年08月06日 00時00分00秒
When using the format-date function on a date formatted like "Fri, 10 May 2019 01:15:00 GMT", I get an error: Invalid date "Fri, 10 May 2019 01:15:00 GMT" (Non-numeric year component)
I would like to obtain a dd/MM/yyyy format .
If you use XSLT 2 chances are you use a processor like Saxon 9 or AltovaXML which have since 2017 (e.g. Saxon in release 9.8 or later or Altova in releas 2017 or later) updated to support XSLT 3 and XPath 3.1 where you could then make use of the parse-ietf-date function (https://www.w3.org/TR/xpath-functions/#func-parse-ietf-date) to parse your input string into a xs:dateTime on which you could then apply the format-dateTime function:
format-dateTime(parse-ietf-date('Fri, 10 May 2019 01:15:00 GMT'), '[D01]/[M01]/[Y0001]')
https://xqueryfiddle.liberty-development.net/6qM2e2m/1
Your current approach doesn't work as the format-date function works on xs:date and not on arbitrary formatted date strings. That particular format you have shown in your sample can be parsed by the parse-ietf-date function.
I have this datetime, or something that looks like it.
2014-11-17 23:02:03 +0000 UTC
I want to convert this to a time object and I've been unable to produce any output from time.Parse apart from:
0001-01-01 00:00:00 +0000 UTC
I've tried these layouts:
time.RFC3339
0001-01-01 00:00:00 0000 UTC
2016-10-10
time.UnixDate
And a few more - none have worked.
This is how I'm calling parse :
updatedAt, err := time.Parse(time.UnixDate, updatedAtVar)
How do I create a time object from a string?
Most likely you used a wrong layout, and you didn't check the returned error.
The layout must be this date/time, in the format your input time is:
Mon Jan 2 15:04:05 -0700 MST 2006
See this working code:
layout := "2006-01-02 15:04:05 -0700 MST"
t, err := time.Parse(layout, "2014-11-17 23:02:03 +0000 UTC")
fmt.Println(t, err)
Output (try it on the Go Playground):
2014-11-17 23:02:03 +0000 UTC <nil>
EDIT:
In your question you included a + sign in your input time (as part of the zone offset), but you have error with times of other formats.
Time.String() uses the following format string:
"2006-01-02 15:04:05.999999999 -0700 MST"
So either use this to parse the times, or use Time.Format() to produce your string representations where you can specify the layout, so you can use the same layout to parse the time strings.
2nd round:
You're including your time strings in URLs. The + sign is a special character in URL encoding: it denotes the space. So the + gets converted to space (and so it vanishes from your time string). Use proper URL encoding! Check out the net/url package, and this example.
Didn't see this yet but for those that don't know the formats, time has the formats builtin as constants. so you can reference them when parsing or formating.
time.Parse(time.RFC3339, <your time.Time object here>)
<time.Time object>.Format(time.RFC3339) //or other type of formats
Here they are for reference
ANSIC = "Mon Jan _2 15:04:05 2006"
UnixDate = "Mon Jan _2 15:04:05 MST 2006"
RubyDate = "Mon Jan 02 15:04:05 -0700 2006"
RFC822 = "02 Jan 06 15:04 MST"
RFC822Z = "02 Jan 06 15:04 -0700" // RFC822 with numeric zone
RFC850 = "Monday, 02-Jan-06 15:04:05 MST"
RFC1123 = "Mon, 02 Jan 2006 15:04:05 MST"
RFC1123Z = "Mon, 02 Jan 2006 15:04:05 -0700" // RFC1123 with numeric zone
RFC3339 = "2006-01-02T15:04:05Z07:00"
RFC3339Nano = "2006-01-02T15:04:05.999999999Z07:00"
Kitchen = "3:04PM"
You are likely using the wrong layout. As explained in time.Parse, you need to specify a layout that helps Go to understand how the date passed as input is formatted.
There are predefined layouts (like the ones you were using), but none matches your input. Hence you need to define a custom layout.
A layout uses the following date as reference:
Mon Jan 2 15:04:05 MST 2006
The layout is nothing else that a representation of that date, that matches the representation of your input:
t, err := time.Parse("2006-01-02 15:04:05 -0700 MST", "2014-11-17 23:02:03 +0000 UTC")
Also remember to check err for errors. It's likely your attempts returned an error, but you didn't check it.
package main
import (
"fmt"
"log"
"time"
)
func main() {
t, err := time.Parse("2006-01-02 15:04:05 -0700 UTC", "2014-11-17 23:02:03 +0000 UTC")
if err != nil {
log.Fatalln(err)
}
fmt.Println(t)
}
How do I pass date in RFC2822 format, to an URL in GET method:
e.g I want to pass 18th jan 2013 as min_date_created in the URL https://www.xyz.com/orders
18th jan 2013 in RFC format is ==> Fri, 18 Jan 2013 17:58:49 +0000
how can I pass it to URL?
TIA!
Instead of passing the date as a string, you could convert it to the number of milliseconds since midnight Jan 1, 1970:
var time = new Date('Fri, 18 Jan 2013 17:58:49 +0000').getTime(); // 1358531929000
widow.location.href = 'https://www.xyz.com/orders?min_date_created='+time;
This way you won't have encoding problems.
Or you could also use encodeURIComponent(), see URL encode sees “&” (ampersand) as “&” HTML entity
Is there any way (other than having to add custom code) to check if my application has exceeded the YQL limits that are allowed? (Not sure if per application limit or the per IP limit applies in my case) I assume that all queries would fail after the limit is reached, but I'd like to be proactive in adding measures to prevent this from happening.
I am using YQL in a critical piece in the application and since a query is issued every few seconds, I suspect I might be nearing the limit.
Unfortunately error code 999 is also returned in other situations, so it is not a reliable indicator of reaching the hourly 10k limit.
Example of response from Yahoo/YQL:
HTTP/1.1 999 Unable to process request at this time -- error 999
Date: Sun, 07 Nov 2010 13:46:10 GMT
Expires: Thu, 01 Jan 1970 22:00:00 GMT
Cache-Control: no-cache, private
Cache-Control: no-store
Pragma: no-cache
Vary: Accept-Encoding
Content-Type: text/html;charset=UTF-8
Age: 0
Transfer-Encoding: chunked
Connection: keep-alive
Server: YTS/1.17.21
I think it would be great if YQL could return a header field with the remaining YQL calls for per hour, like Twitter does.
Relevant parts of the HTTP Response Header of the Twitter API:
HTTP/1.1 200 OK
Date: Sun, 07 Nov 2010 13:52:18 GMT
X-RateLimit-Limit: 150
X-RateLimit-Remaining: 147
X-RateLimit-Class: api
X-Revision: DEV
X-RateLimit-Reset: 1289141505
If you exceed the hourly allowance of 10k hits you will get back an error with the code 999 - write your code accordingly to avoid being blocked.