I have a DSL that looks like JSON and I need to support optional ordering of the elements. For example:
{
'id': 'someid',
'position' : 'someposition',
'operand' : 'someoperand'
}
In order to support any ordering of 'id', 'position', and 'operand' tokens with valid commas, I had to do something like this:
Block returns Block:
{Block}
'{'
members+=BlockMember (',' members+=BlockMember)*
'}'
;
BlockMember returns BlockMember:
blockId=BlockId | position=BlockPosition | operand=Operand
;
The problem is that each element (id, position, operand) is both mandatory as well as only once. With this grammar, I have some issues:
You can define the elements more than once (which also screws up LSP
content assist later).
I can't enforce the mandatory requirement.
It creates an inferred EMF model where a Block has a 'members' collection instead of well defined properties. This means later to parse a Block, I have to iterate the member.
Because it needs to be well-formed JSON, the last property cannot have a comma
I know I can do this if I enforce a specific order and look for the comma. What I would like to do however is have the optional ordering with commas between (except for last) without introducing a 'members' that creates the issues listed above.
Is there a technique to do this?
Related
TL;DR I'm wondering what the pros and cons are (or if they are even equivalent) between #> {as_champion, whatever} and using IN ('as_champion', 'whatever') is. Details below:
I'm working with Rails and using Postgres' array column type, but having to use raw sql for my query as the Rails finder methods don't play nicely with it. I found a way that works, but wondering what the preferred method is:
The roles column on the Memberships table is my array column. It was added via rails as so:
add_column :memberships, :roles, :text, array: true
When I examine the table, it shows the type as: text[] (not sure if that is truly how Postgres represents an array column or if that is Rails shenanigans.
To query against it I do something like:
Membership.where("roles #> ?", '{as_champion, whatever}')
From the fine Array Operators manual:
Operator: #>
Description: contains
Example: ARRAY[1,4,3] #> ARRAY[3,1]
Result: t (AKA true)
So #> treats its operand arrays as sets and checks if the right side is a subset of the left side.
IN is a little different and is used with subqueries:
9.22.2. IN
expression IN (subquery)
The right-hand side is a parenthesized subquery, which must return exactly one column. The left-hand expression is evaluated and compared to each row of the subquery result. The result of IN is "true" if any equal subquery row is found. The result is "false" if no equal row is found (including the case where the subquery returns no rows).
or with literal lists:
9.23.1. IN
expression IN (value [, ...])
The right-hand side is a parenthesized list of scalar expressions. The result is "true" if the left-hand expression's result is equal to any of the right-hand expressions. This is a shorthand notation for
expression = value1
OR
expression = value2
OR
...
So a IN b more or less means:
Is the value a equal to any of the values in the list b (which can be a query producing single element rows or a literal list).
Of course, you can say things like:
array[1] in (select some_array from ...)
array[1] in (array[1], array[2,3])
but the arrays in those cases are still treated like single values (that just happen to have some internal structure).
If you want to check if an array contains any of a list of values then #> isn't what you want. Consider this:
array[1,2] #> array[2,4]
4 isn't in array[1,2] so array[2,4] is not a subset of array[1,2].
If you want to check if someone has both roles then:
roles #> array['as_champion', 'whatever']
is the right expression but if you want to check if roles is any of those values then you want the overlaps operator (&&):
roles && array['as_champion', 'whatever']
Note that I'm using the "array constructor" syntax for the arrays everywhere, that's because it is much more convenient for working with a tool (such as ActiveRecord) that knows to expand an array into a comma delimited list when replacing a placeholder but doesn't fully understand SQL arrays.
Given all that, we can say things like:
Membership.where('roles #> array[?]', %w[as_champion whatever])
Membership.where('roles #> array[:roles]', :roles => some_ruby_array_of_strings)
and everything will work as expected. You're still working with little SQL snippets (as ActiveRecord doesn't have a full understanding of SQL arrays or any way of representing the #> operator) but at least you won't have to worry about quoting problems. You could probably go through AREL to manually add #> support but I find that AREL quickly devolves into an incomprehensible and unreadable mess for all but the most trivial uses.
I have the following function for a hacky project:
% The Record variable is some known record with an associated table.
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
ExistingFields = record_to_fields(Existing),
RecordFields = record_to_fields(Record),
ExistingFields == RecordFields
]).
The function record_to_fields/1 simply drops the record name and ID from the tuple so that I can compare the fields themselves. If anyone wants context, it's because I pre-generate a unique ID for a record before attempting to insert it into Mnesia, and I want to make sure that a record with identical fields (but different ID) does not exist.
This results in the following (redacted for clarity) stack trace:
{aborted, {{case_clause, {stuff}},
[{db, '-my_func/2-fun-1-',8, ...
Which points to the line where I declare Query, however there is no case clause in sight. What is causing this error?
(Will answer myself, but I appreciate a comment that could explain how I could achieve what I want)
EDIT: this wouldn't be necessary if I could simply mark certain fields as unique, and Mnesia had a dedicated insert/1 or create/1 function.
For your example, I think your solution is clearer anyway (although it seems you can pull the record_to_fields(Record) portion outside the comprehension so it isn't getting calculated over and over.)
Yes, list comprehensions can only have generators and assignments. But you can cheat a little by writing an assignment as a one-element generator. For instance, you can re-write your expression as this:
RecordFields = record_to_fields(Record),
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
ExistingFields <- [record_to_fields(Existing)],
ExistingFields == RecordFields
]).
As it turns out, the QLC DSL does not allow assignments, only generators and filters; as per the documentation (emphasis mine):
Syntactically QLCs have the same parts as ordinary list
comprehensions:
[Expression || Qualifier1, Qualifier2, ...]
Expression (the template)
is any Erlang expression. Qualifiers are either filters or generators.
Filters are Erlang expressions returning boolean(). Generators have
the form Pattern <- ListExpression, where ListExpression is an
expression evaluating to a query handle or a list.
Which means we cannot variable assignments within a QLC query.
Thus my only option, insofar as I know, is to simply write out the query as:
Query = qlc:q([Existing ||
Existing <- mnesia:table(Table),
record_to_fields(Existing) == record_to_fields(Record)
]).
I want to emulate an "&" operator for searching elements in my mongo db.
there are 4 searchable fields name id tags negative_tags
for a match to be true, any of these could match.
For instance if I search a&b, "a" could be matched in any of the 4 fields and "b" as well. However, they need to both be matched
I tried doing the following
Model.or({:name.all => regexps}, {:id.all =>regexps}, {:tags.all => regexps}, {:negative_tags.all => regexps})
regexps is an array of regexp. For the example given it would be
[ /a/i, /b/i ]
However, this does not behave like I want, because you need the matches to all happen on the same property.
My other try was to run separate mongo queries for each regexps and take the intersection of the sets.
Model.or({:name.in => one_regexp}, {:id.in => one_regexp}, {:tags.in => one_regexp}, {:negative_tags.in => one_regexp})
My problem is that I am not sure how to merge the two hashes. Mongoid lazily evaluates the queries and returns a Mongoid::Criteria object.
I'd like to know how I can do an intersection
There are two distinct ways to handle this. Are you trying to have both regular expressions evaluate per field or can a be true for name and b be true for id?
If it is the latter, I would use a gem for this:
gem 'mongoid_search'
If it is the former, I'd simply join the array into a single regex:
one_regexp.collect {|regexp| "(?=.*#{regexp}" }.join
If what you want to do is to apply two RegEx expressions onto each field, simply put both both in nonconsuming patterns and use one regular expression. This is known as positive lookahead assertion (?=) combined with the .* operator that allows the order to be reversed.
/(?=.*a)(?=.*b)/
How to design REST url for resource collection, which filters resource by attribute not equal to a given value?
For example, to get the students in 8th grade, we use
GET /students?grade=8
How to do the same, if we need to get the students not in 8th grade? And how to design for less than (<) , greater than (>) etc ?
What I am thinking of doing is including the operator as part of the argument, delimited from the value. I would also define non-symbolic operators to avoid the need for ugly URL-encoding or confusion with the equals sign after the parameter name. For your example, this would be something like this for students not in grade 8:
GET /students?grade=neq|8
Students in grades > 8 would be:
GET /students?grade=gt|8
Students between grades 8 and 10 (inclusive):
GET /students?grade=gte|8,lte|10
This approach can be extended to other filterable fields without the need to add additional parameters to modify the way each field is filtered.
Stripe has one of the most respected APIs.
They use a separate parameter for each operator separated with a dot.
For example, to search on created date:
/charges?created.gt=
/charges?created.gte=
/charges?created.lt=
/charges?created.lte=
In your case you could do something like:
/students?grade.gt=8&grade.lt=8
Or even add another operator for not:
/students?grade.not=8
One option would be to add an additional query parameter such as gradeOperator in which you could pass the operator to be used when comparing the value against the grade parameter. E.g.,
GET /students?grade=8&gradeOperator=!%3D
!%3D is the URL-encoded form of !=, so your REST API would de-encode the operator and interpret this as grade != 8.
Another approach would be to pass the value and operator in the HTTP request body. Something like this would potentially work (with the body provided in JSON as an example):
GET /students
Content-Type: application/json
{ "grade": {"value": 8, "operator": "!=" } }
That could be nice since you wouldn't have to repeat the word 'grade' in gradeOperator, the operator is simply nested inside a JSON object as the value of grade.
In either solution, you could potentially define any number of operators, including <, >, >=, <=, etc. Just be sure to properly sanitize any input operators your API receives, especially if used in a DB query, to avoid things like SQL injection attacks.
I have heard that specifying records through tuples in the code is a bad practice: I should always use record fields (#record_name{record_field = something}) instead of plain tuples {record_name, value1, value2, something}.
But how do I match the record against an ETS table? If I have a table with records, I can only match with the following:
ets:match(Table, {$1,$2,$3,something}
It is obvious that once I add some new fields to the record definition this pattern match will stop working.
Instead, I would like to use something like this:
ets:match(Table, #record_name{record_field=something})
Unfortunately, it returns an empty list.
The cause of your problem is what the unspecified fields are set to when you do a #record_name{record_field=something}. This is the syntax for creating a record, here you are creating a record/tuple which ETS will interpret as a pattern. When you create a record then all the unspecified fields will get their default values, either ones defined in the record definition or the default default value undefined.
So if you want to give fields specific values then you must explicitly do this in the record, for example #record_name{f1='$1',f2='$2',record_field=something}. Often when using records and ets you want to set all the unspecified fields to '_', the "don't care variable" for ets matching. There is a special syntax for this using the special, and otherwise illegal, field name _. For example #record_name{record_field=something,_='_'}.
Note that in your example you have set the the record name element in the tuple to '$1'. The tuple representing a record always has the record name as the first element. This means that when you create the ets table you should set the key position with {keypos,Pos} to something other than the default 1 otherwise there won't be any indexing and worse if you have a table of type 'set' or 'ordered_set' you will only get 1 element in the table. To get the index of a record field you can use the syntax #Record.Field, in your example #record_name.record_field.
Try using
ets:match(Table, #record_name{record_field=something, _='_'})
See this for explanation.
Format you are looking for is #record_name{record_field=something, _ = '_'}
http://www.erlang.org/doc/man/ets.html#match-2
http://www.erlang.org/doc/programming_examples/records.html (see 1.3 Creating a record)