Attempting to map dates to index in ElasticSearch - elasticsearch-5

Question

I am using ElasticSearch and attempting to create an index with a class that has a dynamic type property in it. This property may have strings or dates in it. When indexing, I have been using the code below:
dynamic instance = MyObject.GetDynamicJson();
var indexResponse = client.Index((object) instance, i=>i
.Index("myIndex")
.Type("MyObject")
);
Here's the code for GetDynamicJson().
MyObject has only Name and Value as properties. (apologies, I've had issues in the past with Elastic choking on json without all the quotes, which I have escaped with \ characters):
String json = "{ \"Name\":\" + Name + "\",\"DateValue\":\"";
try {
var date = DateTime.parse(Value);
json += DateTime.ToString("yyyy/MM/dd HH:mm:ss Z") + "\", \"Value\":\"\"}";
} catch { //If the DateTime parse fails, DateValue is empty and I'll have text in Value
json += "\",\"Value\":\"" + Value + "\"}";
}
return json;
For some reason it doesn't seem to like the string in DateValue and I definitely don't know why it's leaving out that property entirely in the error:
For whatever reason, ElasticSearch is completely dumping the DateValue property, doesn't seem to see the DateValue property at all.
I'm getting the error:
{"name":"CreatedDate","value":"2017-11-07T13:37:11.4340238-06:00"}
[indices:data/write/bulk[s][p]]"}],"type":"class_cast_exception","reason":"org.elasticsearch.index.mapper.TextFieldMapper cannot be cast to org.elasticsearch.index.mapper.DateFieldMapper"},"status":500
Later note: I have changed the index creator method to update the mapping. I added a third field to the Object, so now it has properties: Name, Value, DateValue:
public static void CreateRecordsIndex(ElasticClient client)
{
client.CreateIndex("myIndex", i => i
.Settings(s => s
.NumberOfShards(2)
.NumberOfReplicas(0)
)
.Mappings(x => x
.Map<MyObject>(m => m.AutoMap())));
}
Now, it is successfully mapping and creating a property each time, but it still seems to drop the property I am sending it in the json. It just sets them all to the default datetime: "dateValue": "0001-01-01T00:00:00". This is strange, because when making the dynamic instance I send to Elastic, I use only the MyObject.GetDynamicJson() method to build it. I no longer get the mapping error, but Elastic still seems oblivious to "dateValue":"some date here" in the object when it is set.

Answer 1

OK, I got rid of the dynamic object type (ultimately I wasn't actually getting data from the json method, I had a typo and Elastic was getting the original object directly - it's a wonder it was still handling it). So I let Elastic do the parse using its mapping. In order to do that, I first updated MyObject to include multiple properties, one for each type the incoming property could be (I am only handling text and dates in this case). For the DateValue property of MyObject, I have this:
public DateTime DateValue {
get
{
try
{
return DateTime.Parse(Value);
} catch
{
return new DateTime();
}
}
set
{
try {
DateValue = value;
} catch
{
DateValue = new DateTime();
}
}
}
Now, if Value is a date, my DateValue field will be set. Otherwise it'll have the default date (a very early date "0001-01-01T00:00:00"). This way, I can later search both for text against that dynamic field, or if a date is set, I can do date and date range queries against it (technically they end up in two different fields, but coming from the same injested data).
Key to this is having the index mapping setup, as you can see in this method from the question:
public static void CreateRecordsIndex(ElasticClient client)
{
client.CreateIndex("myIndex", i => i
.Settings(s => s
.NumberOfShards(2)
.NumberOfReplicas(0)
)
.Mappings(x => x
.Map<MyObject>(m => m.AutoMap())));
}
In order to recreate the index with the updated MyObject, I did this:
if (client.IndexExists("myIndex").Exists)
{
client.DeleteIndex("myIndex");
CreateRecordsIndex(client); //Goes to the method above
}