Kibana - Display times series of tweets posted - Twitter - twitter

Question

I'm using the latest Kibana 4 / ES 1.4 version and I'm trying to display the number of tweets over time. My idea is to slice the 'created_at' field from the tweets documents.
The mapping defined for this field is the following
dynamic_templates": [
{
"created_at": {
"mapping": {
"locale": "US",
"format": "EEE MMM dd HH:mm:ss Z yyyy",
"type": "date"
},
"match": "created_at"
}
},
...
I can create basic charts in Kibana (with term aggregates field) and overall seems to be working but I cannot display any trends with line charts or date histogram .with the created_at field.
Below is the error
ElasticsearchParseException[failed to parse date field [2014-10-13T23:35:31.450Z],
tried both date format [EEE MMM dd HH:mm:ss Z yyyy], and timestamp number]; nested:
IllegalArgumentException[Invalid format: \"2014-10-13T23:35:31.450Z\"]; }
Thanks for your help,
Arnaud

Answer 1

I'm unsure if the format of your date is correct but if it is I had a similar problem in ElasticSearch 1.4 that the parser wouldn't recognise the timestamp unless the timestamp was mapped within properties:
curl -XPUT 'http://localhost:9200/index/container/_mapping' -d'
{
"container" : {
"properties" : {
"#timestamp" : {"type":"date", "format": "dateOptionalTime"}
}
}
}'
Also may not be relavant to you but for a useful article for naming date conventions: http://joelabrahamsson.com/dynamic-mappings-and-dates-in-elasticsearch/
And the elastic search date format list: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-date-format.html
Hope this helps!