JsonProvider<...>.Root does not have null as proper value - f#

I am trying to query results of the parsed Json and if I cannot find I want to do something else.
[
{
"orderId": 136,
"quantity": 5,
"price": 3.75
},
{
"orderId": 129,
"quantity": 9,
"price": 3.55
},
{
"orderId": 113,
"quantity": 11,
"price": 3.75
}
]
My code is like:
type OrdersProvider = JsonProvider<"Orders.json">
let orders = OrdersProvider.GetSamples()
let test id =
let res = query{
for i in orders do
where (i.OrderId = id)
select i
headOrDefault
}
if isNull(res)
then NOT_FOUND("")
else OK(res.JsonValue.ToString())
)
However I am getting compiler error "JsonProvider<...>.Root does not have null as proper value". Which is kinda makes sense except I still want to catch the case when id is not in the file. I guess I could change headOrDefault to head and trap the exception but wonder if there is something better.
Update #1:
Following one of the links in comments I was able to get away with
if obj.ReferenceEquals(res,null)
then NOT_FOUND("")
else OK(res.JsonValue.ToString())
)
Update #2:
While mentioned code works but still feels unnatural for the language. Accepted answer looks more natural.

I think the headOrDefault operation was designed for compatibility with LINQ to SQL, which is why it returns null in the default case - this is not something you'd normally want in well behaved F# code, so using it in the way your query does is not a good idea.
Fortunately, headOrDefault will work with F# option type - if you return Some from your select clause then headOrDefault returns None when the value is not available:
let res =
query {
for i in orders do
where (i.OrderId = id)
select (Some i)
headOrDefault }
Now you can handle the missing case with pattern matching:
match res with
| None -> NOT_FOUND("")
| Some order -> OK(order.JsonValue.ToString())

Related

Parse a complex hash and return changes to keys

I'm using json-compare gem to compare two different json files.
Example file 1:
{"suggestions": [
{
"id1": 1,
"title1": "Test",
"body1": "Test"
}
]
}
Example file 2:
{"suggestions": [
{
"id2": 1,
"title2": "Test",
"body2": "Test"
}
]
}
The gem works well and spits out a hash that looks like this:
{:update=>
{"suggestions" =>
{:update=>
{0=>
{:append=>
{"id2"=>1, "title2"=>"Test", "body2"=>"Test"},
:remove=>
{"id1"=>1, "title1"=>"Test", "body1"=>"Test"},
}
}
}
}
}
How can I parse this and return all the places where json Keys were changed? For the sake of simplicity, how would I put to the console:
id1 changed to id2
title1 changed to title2
body1 changed to body2
For the purpose of what I'm building I don't need to know changes to the values. I just need to know that id1 became id2, etc.
Except if you are relaying on key ordering there is no way to tell that id1 got replaced by id2 and title2 by title1, or that id1 became title1 and id2 became title2. Sounds like you would need specific logic related to the actual key names (in this example searching for different integer suffixes).
Maybe this can be enough for the purpose:
def find_what_changed_in(mhash, result = [])
result << mhash
return if mhash.keys == [:append, :remove]
mhash.keys.each { |k| find_what_changed_in(mhash[k], result) }
result.last
end
find_what_changed_in(changes)
#=> {:append=>{"id2"=>1, "title2"=>"Test", "body2"=>"Test"}, :remove=>{"id1"=>1, "title1"=>"Test", "body1"=>"Test"}}
Where:
changes = {:update=>
{"suggestions" =>
{:update=>
{0=>
{:append=>
{"id2"=>1, "title2"=>"Test", "body2"=>"Test"},
:remove=>
{"id1"=>1, "title1"=>"Test", "body1"=>"Test"},
...

iOS Swit 3 - filter array inside filter

I would like to filter array inside a filter. First I have a big array of Staff object (self.bookingSettings.staffs). Inside this array I have multiple object like this :
"staffs": [
{
"id": 1,
"name": "Brian",
"services": [
{
"id": 1
},
{
"id": 2
},
{
"id": 3
},
{
"id": 4
}
],
"pos": 1
},...
I would like to filter this array in order to have only services with id = 3.
I succeed to have if first object is equal to 3 with this code :
self.bookingSettings.staffs.filter({ $0.services.first?.id == self.bookingService.id })
but that takes only the first item.
I think I have to filter inside my filter function, something like this to loop over all object inside services :
self.bookingSettings.staffs.filter({ $0.services.filter({ $0.id == self.bookingService.id }) })
but I've the following error: Cannot convert value of type [BookingService] to closure result type Bool.
Is this a good idea ? How can I achieve this ?
You could use filter, which would look something like this:
self.bookingSettings.staffs.filter {
!$0.services.filter{ $0.id == self.bookingService.id }.isEmpty
}
This code is constructing an entire array of filtered results, only to check if its empty and immediately discard it. Since filter returns all items that match the predicate from the list, it won't stop after it finds a match (which is really what you're looking for). So even if the first element out of a list of a million elements matches, it'll still go on to check 999,999 more elements. If the other 999,999 elements also match, then they will all be copied into filter's result. That's silly, and can use way more CPU and RAM than necessary in this case.
You just need contains(where:):
self.bookingSettings.staffs.filter {
$0.services.contains(where: { $0.id == self.bookingService.id })
}
contains(where:) short-circuits, meaning that it won't keep checking elements after a match is found. It stops and returns true as soon as find a match. It also doesn't both copying matching elements into a new list.

elasticsearch mongoid filter on has_many ids

I'm using ES 1.4, Rails 5, and Mongoid 6. I'm also the mongoid-elasticsearch gem, which I don't think is relevant, but including it in case I'm wrong.
I have a Case model. When I run this query, everything works great, here's the query:
GET _search
{
"query":{
"filtered":{
"query":{
"query_string":{
"query":"behemoth"
}
}
}
}
}
Here's a result, notice the organization_id:
hits": [
{
"_index": "cases",
"_type": "case",
"_id": "57d5e583a46100386987d7f4",
"_score": 0.13424811,
"_source": {
"basic info": {
"first name": "Joe",
"last name": "Smith",
"narrative": "behemoth"
},
"organization_id": {
"$oid": "57d4bc2fa461003841507f83"
},
"case_type_id": {
"$oid": "57d4bc7aa461002854f88441"
}
}
}
See how there's that "$oid" for organzation id? That's because in my as_indexed_json method for Case, I have:
["organization_id"] = self.case_type.organization.id
I think that the filter doesn't work b/c Mongoid somehow adds that subkey $oid. So my next thought was, I'll just make it a string:
["organization_id"] = self.case_type.organization.id.to_s
But that throws an error:
{"error":"MapperParsingException[object mapping for [case] tried to parse as object, but got EOF, has a concrete value been provided to it?]","status":400}
Anyone have any idea how to A) either use a mongo id as a filter, or B) get ES the info it needs so it doesn't complain as above?
Thanks for any help,
Kevin
Turns out that this is because there was already an existing index. When I nuked the index and re-indexed the data, this works fine (really bad error syntax with ES).

Aerospike: how to bulk load a list of integers into a bin?

I'm trying to use the Aerospike bulk loader to seed a cluster with data from a tab-separated file.
The source data looks like this:
set key segments
segment 123 10,20,30,40,50
segment 234 40,50,60,70
The third column, 'segments', contains a comma separated list of integers.
I created a JSON template:
{
"version" : "1.0",
"input_type" : "csv",
"csv_style": { "delimiter": " " , "n_columns_datafile": 3, "ignore_first_line": true}
"key": {"column_name":"key", "type": "integer"},
"set": { "column_name":"set" , "type": "string"},
"binlist": [
{"name": "segments",
"value": {"column_name": "segments", "type": "list"}
}
]
}
... and ran the loader:
java -cp aerospike-load-1.1-jar-with-dependencies.jar com.aerospike.load.AerospikeLoad -c template.json data.tsv
When I query the records in aql, they seem to be a list of strings:
aql> select * from test
+--------------------------------+
| segments |
+--------------------------------+
| ["10", "20", "30", "40", "50"] |
| ["40", "50", "60", "70"] |
+--------------------------------+
The data I'm trying to store is a list of integers. Is there an easy way to convert the objects stored in this bin to a list of integers (possibly a Lua UDF) or perhaps there's a tweak that can be made to the bulk loader template?
Update:
I attempted to solve this by creating a Lua UDF to convert the list from strings to integers:
function convert_segment_list_to_integers(rec)
for i=1, table.maxn(rec['segments']) do
rec['segments'][i] = math.floor(tonumber(rec['segments'][i]))
end
aerospike:update(rec)
end
... registered it:
aql> register module 'convert_segment_list_to_integers.lua'
... and then tried executing against my set:
aql> execute convert_segment_list_to_integers.convert_segment_list_to_integers() on test.segment
I enabled some more verbose logging and notice that the UDF is throwing an error. Apparently, it's expecting a table and it was passed userdata:
Dec 04 2015 23:23:34 GMT: DEBUG (udf): (udf_rw.c:send_result:527) FAILURE when calling convert_segment_list_to_integers convert_segment_list_to_integers ...rospike/usr/udf/lua/convert_segment_list_to_integers.lua:2: bad argument #1 to 'maxn' (table expected, got userdata)
Dec 04 2015 23:23:34 GMT: DEBUG (udf): (udf_rw.c:send_udf_failure:407) Non-special LDT or General UDF Error(...rospike/usr/udf/lua/convert_segment_list_to_integers.lua:2: bad argument #1 to 'maxn' (table expected, got userdata))
It seems that maxn isn't an applicable method to a userdata object.
Can you see what needs to be done to fix this?
To convert your lists with string values to lists of integer values you can run the following record udf:
function convert_segment_list_to_integers(rec)
local list_with_ints = list()
for value in list.iterator(rec['segments']) do
local int_value = math.floor(tonumber(value))
list.append(list_with_ints, int_value)
end
rec['segments'] = list_with_ints
aerospike:update(rec)
end
When you edit your existing lua module, make sure to re-run register module 'convert_segment_list_to_integers.lua'.
The cause of this issue is within the aerospike-loader tool: it will always assume/enforce strings as you can see in the following java code:
case LIST:
/*
* Assumptions
* 1. Items are separated by a colon ','
* 2. Item value will be a string
* 3. List will be in double quotes
*
* No support for nested maps or nested lists
*
*/
List<String> list = new ArrayList<String>();
String[] listValues = binRawText.split(Constants.LIST_DELEMITER, -1);
if (listValues.length > 0) {
for (String value : listValues) {
list.add(value.trim());
}
bin = Bin.asList(binColumn.getBinNameHeader(), list);
} else {
bin = null;
log.error("Error: Cannot parse to a list: " + binRawText);
}
break;
Source on Github: http://git.io/vRAQW
If you prefer, you can modify this code and re-compile to always assume integer list values. Change line 266 and 270 to something like this (untested):
List<Integer> list = new ArrayList<Integer>();
list.add(Integer.parseInt(value.trim());

Emit Tuples From Erlang Views In CouchDB

CouchDB, version 0.10.0, using native erlang views.
I have a simple document of the form:
{
"_id": "user-1",
"_rev": "1-9ccf63b66b62d15d75daa211c5a7fb0d",
"type": "user",
"identifiers": [
"ABC",
"DEF",
"123"
],
"username": "monkey",
"name": "Monkey Man"
}
And a basic javascript design document:
{
"_id": "_design/user",
"_rev": "1-94bd8a0dbce5e2efd699d17acea1db0b",
"language": "javascript",
"views": {
"find_by_identifier": {
"map": "function(doc) {
if (doc.type == 'user') {
doc.identifiers.forEach(function(identifier) {
emit(identifier, {\"username\":doc.username,\"name\":doc.name});
});
}
}"
}
}
}
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"DEF","value":{"username":"monkey","name":"Monkey Man"}},
{"id":"user-1","key":"123","value":{"username":"monkey","name":"Monkey Man"}}
]}
I'm looking into building an Erlang view that does the same thing. Best attempt so far is:
%% Map Function
fun({Doc}) ->
case proplists:get_value(<<"type">>, Doc) of
undefined ->
ok;
Type ->
Identifiers = proplists:get_value(<<"identifiers">>, Doc),
ID = proplists:get_value(<<"_id">>, Doc),
Username = proplists:get_value(<<"username">>, Doc),
Name = proplists:get_value(<<"name">>, Doc),
lists:foreach(fun(Identifier) -> Emit(Identifier, [ID, Username, Name]) end, Identifiers);
_ ->
ok
end
end.
which emits:
{"total_rows":3,"offset":0,"rows":[
{"id":"user-1","key":"ABC","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"DEF","value":["monkey","Monkey Man"]},
{"id":"user-1","key":"123","value":["monkey","Monkey Man"]}
]}
The question is - how can I get those values out as tuples, instead of as arrays? I don't imagine I can (or would want to) use records, but using atoms in a tuple doesn't seem to work.
lists:foreach(fun(Identifier) -> Emit(Identifier, {id, ID, username, Username, name, Name}) end, Identifiers);
Fails with the following error:
{"error":"json_encode","reason":"{bad_term,{<<\"user-1\">>,<<\"monkey\">>,<<\"Monkey Man\">>}}"}
Thoughts? I know that Erlang sucks for this specific kind of thing (named access) and that I can do it by convention (id at first position, username next, real name last), but that makes the client side code pretty ugly.
The JSON object {"foo":"bar","baz":1} is {[{<<"foo">>,<<"bar">>},{<<"baz">>,1}]}
In Erlang lingua it is a proplist wrapped in a tuple.
It's not pretty, but very efficient :)
To get a feel for it you can play with the JSON lib that ships with CouchDB:
Start CouchDB with the -i
(interactive) flag
On the resulting erlang shell, type: couch_util:json_decode(<<"{\"foo\":\"bar\"}">>).
Profit
// in later versions of CouchDB, this is ejson:decode()
For test_suite_reports bd, that has tests field:
[
{
"name": "basics",
"status": "success",
"duration": 21795
},
{
"name": "all_docs",
"status": "success",
"duration": 385
} ...
I have wrote this to get name and status:
fun({Doc}) ->
Name = fun(L) -> proplists:get_value(<<"name">>, L, null) end,
Status = fun(L) -> proplists:get_value(<<"status">>, L, null) end,
Tests = proplists:get_value(<<"tests">>, Doc, null),
lists:foreach(fun({L}) -> Emit(Name(L), Status(L)) end, Tests)
end.
If you like experimental features (that still work...), you might want to have a look to Erlang exprecs.
I found it extremely helpful in creating a sort of dynamic records for Erlang.

Resources