What are the best practices in constructing search paths for Falcor queries?
Here an example from a Staffing domain where a staffing coordinator is trying to staff people for project needs:
A need has startDate and an endDate - both are simple string properties of the need
A need is for a project, hence it has a reference to a project
A need is for a skill, so it has a reference to a skill
Here's what a need looks like in a needMap:
needMap: {
101: {
startDate: '2016-07-04T04:00:00.000Z',
endDate: '2016-10-22T03:59:59.999Z',
project: $ref('projectMap[87]'),
skill: $ref('skillMap[3]')
},
...
}
How do I specify a search for all needs that match the following criteria:
minStartDate: minimum start date for the need
maxStartDate: maximum start date for the need
projectId: filter to needs to only this project
skillId: filter to needs to only this skill
Note: I am not looking for the code or ideas to do the actual search, only how to specify the search using Falcor path concepts.
The issues #713 and #775 from the falcor repository in github address this. Two possible solutions appear :
You can use a query string, for your case it would be something like this : model.get(["needs", "search", "minStartDate=2016&maxStartDate=2016&projectId=$ref(projectMap[87])])
You can use another endpoint for the search, and then use falcor only to get more info about the result the other endpoint returned
I guess another option would be to use a call rather than a get, as this allow you to pass multiple parameters. You would have something like this:
model.call(
["model", "search"],
{
minStartDate: "2016",
maxStartDate: "2016",
projectId: $ref('projectMap[87])
},
[],
[]);
Related
I have had to move queries off the main database in order to meet requirements for complex authorization - for example a user with a given authorization role can only view data for individuals in the same institution.
I am using the Breeze .net DocCode sample for guidance, and have copied the premise for the mapping of domain models to DTOs.
get { return ForCurrentUser(Context.Orders).Select(o => new Order {
OrderID = o.OrderID,
....
OrderDetails = o.OrderDetails.Select(od => new OrderDetail
{
ProductID = od.ProductID,
UnitPrice = od.UnitPrice
...
})
The problem is that which mapped properties to .include(entity framework method)/.expand (breeze method) is now a concern of the mapping function (for example, the above code will always return the OrderDetails collection, whether I want them or not). I would like to still only eagerly load/expand properties if the javascript client generated predicate has a .expand directive for that property.
Is this at all possible, or am I stuck with manually defining different mapping functions on the server, depending on what properties I want expanded? (I am happy to use tools such as automapper if that would solve or simplify the problem)
Thank you
You will need to use the ODataQueryOptions as a parameter to your controller method. This gives you the details of the query predicates in your server method, so that you can apply them as needed rather that having them applied automatically. This will let you expand, or not, based upon the query.
See this answer and this answer to see how it works.
Given the following classes:
Simplified example:
class Query {
Institution institution
}
class Institution {
String name
}
With the following parameters being submitted: query.institution.id=20 and query.institution.name=Example
I would like to include only the institution id and ignore the name from being bound to the query's institution instance.
Is it possible with bindData to explicitly include associated instance's that are nested multiple levels like this?
I haven't seen any examples of this, aside from using the prefix for a single level of nesting, and the following does not seem to work:
Simplified example:
bindData(queryInstance, params, [include: [
'institution.id',
]], 'query')
The best practice is to filter your request parameters through a command object, which can then be used to generate any kind of query.
Command cmd = Command.getInstance()
bindData(cmd, request.params)
if (cmd.validate()) Query query = cmd.generateQuery()
This way you get the benefit of binding only to fields you expose on the command object, while validating and transforming incoming data without involving your domain.
I'm trying to get a list of domain objects back, but only those that the user is allowed to see. This is based on what granted role that user has from spring security.
What I'd like to be able to do in my controller is something along the lines of
[reportInstanceList: Report.list(params).sort{it.name}]
but only get the reports where
Report.role = SecurityContextHolder.getContext().getAuthentication().getAuthorities()
I have the Spring security stuff in another service class, but for simplicity's sake, its inline here.
Is there a way to direct GORM to only pull the records where the roles match?
Something like:
def reports = Report.createCriteria().list( params ) {
'in'( "role", SpringSecurityUtils.principalAuthorities )
}
[ reportInstanceList: reports, totalCount: reports.totalCount ]
Should work...
This is what dynamic finders are for. You can use the findAllBy finder to get all the objects with an attribute.
Report.findAllByRoleInList(SecurityContextHolder.getContext().getAuthentication().getAuthorities())
You can also sort at the db level by passing a map of params.
Report.findAllByRoleInList(roles, [sort: 'name'])
My Grails app is using the searchable plugin, which builds on Compass and Lucene to provide search functionality. I have two searchable classes, say Author and Book. I have mapped these classes to the search index, so that only certain fields can be searched.
To perform a search across both classes I simply call
def results = searchableService.search(query)
One of the nice features of doing the search across both class simultaneously, is that the results object includes metadata about number of results included, number of results available, pagination details etc.
I recently added a boolean approved flag to the Book class and I never want unapproved books to appear in the search results. One option is to replace the call above with:
def bookResults = Book.search(query + " approved:1")
def authorResults = Author.search(query)
However, I now need to figure out how to combine the metadata for both results, which is likely to be tricky (particularly pagination).
Is there a way to search across Book and Author with a single query, but only return approved books?
Do you want to be able to find authors or do you want to find books with a given author?
If you want to find books with a given author, you can configure your domain classes in the following way:
class Author {
String name
...
static searchable = {
root false
}
}
this will result in excluding the Author from the searchableService.search(query)-result and you'll find field names like $/Book/Author/name in your index. (use luke to examine your index: http://code.google.com/p/luke/).
You can change the name of those fields by configuring a better prefix in your Book-class:
class Book {
String name
Author author
...
static searchable = {
author component: [prefix: 'author']
}
}
this will change the name of the field in the index to bookauthor.
If you now search with searchableService.search(query), you'll find all books where the name of the book or the name of the author contains the search term. You can even restrict the search to a given author by using the authorname:xyz syntax.
If you really would like to mix the search results, I only know the solution you already mentioned: mixing both results with your own code, but I guess it will be hard to mix the scoring of the hits in a good way.
Update to your response: Here's my pagination code...
.gsp:
<div class="pagination">
<g:paginate total="${resultsTotal}" params="[q: params.q]"/>
</div>
controller:
result = searchableService.search(params.q, params)
[
resultList: result.results,
resultsTotal: result.total
]
So if you just merge the results of your two searches and add the result.totals, this could work for you.
I've created a test app and came to the following solution. maybe it helps...
if the property approved only has the states 0 and 1, the following query will work:
def results = searchableService.search(
"(${query} AND approved:1) OR (${query} -approved:0 -approved:1)"
)
I guess this can be reformulated in a better way if you don't use the QueryParser but the BooleanQueryBuilder.
BTW: if you add a method like
String getType() { "Book" }
and
String getType() { "Author" }
To your domains, you can even configure your search to do it like this
def results = searchableService.search(
"(${query} AND approved:1) OR (${query} AND type:Author)"
)
I have an ASP.NET MVC website. In my backend I have a table called People with the following columns:
ID
Name
Age
Location
... (a number of other cols)
I have a generic web page that uses model binding to query this data. Here is my controller action:
public ActionResult GetData(FilterParams filterParams)
{
return View(_dataAccess.Retrieve(filterParams.Name, filterParams.Age, filterParams.location, . . .)
}
which maps onto something like this:
http://www.mysite.com/MyController/GetData?Name=Bill .. .
The dataAccess layer simply checks each parameter to see if its populated to add to the db where clause. This works great.
I now want to be able to store a user's filtered queries and I am trying to figure out the best way to store a specific filter. As some of the filters only have one param in the queryString while others have 10+ fields in the filter I can't figure out the most elegant way to storing this query "filter info" into my database.
Options I can think of are:
Have a complete replicate of the table (with some extra cols) but call it PeopleFilterQueries and populate in each record a FilterName and put the value of the filter in each of field (Name, etc)
Store a table with just FilterName and a string where I store the actual querystring Name=Bill&Location=NewYork. This way I won't have to keep adding new columns if the filters change or grow.
What is the best practice for this situation?
If the purpose is to save a list of recently used filters, I would serialise the complete FilterParams object into an XML field/column after the model binding has occurred. By saving it into a XML field you're also giving yourself the flexibility to use XQuery and DML should the need arise at a later date for more performance focused querying of the information.
public ActionResult GetData(FilterParams filterParams)
{
// Peform action to get the information from your data access layer here
var someData = _dataAccess.Retrieve(filterParams.Name, filterParams.Age, filterParams.location, . . .);
// Save the search that was used to retrieve later here
_dataAccess.SaveFilter(filterParams);
return View(someData);
}
And then in your DataAccess Class you'll want to have two Methods, one for saving and one for retrieving the filters:
public void SaveFilter(FilterParams filterParams){
var ser = new System.Xml.Serialization.XmlSerializer(typeof(FilterParams));
using (var stream = new StringWriter())
{
// serialise to the stream
ser.Serialize(stream, filterParams);
}
//Add new database entry here, with a serialised string created from the FilterParams obj
someDBClass.SaveFilterToDB(stream.ToString());
}
Then when you want to retrieve a saved filter, perhaps by Id:
public FilterParams GetFilter(int filterId){
//Get the XML blob from your database as a string
string filter = someDBClass.GetFilterAsString(filterId);
var ser = new System.Xml.Serialization.XmlSerializer(typeof(FilterParams));
using (var sr = new StringReader(filterParams))
{
return (FilterParams)ser.Deserialize(sr);
}
}
Remember that your FilterParams class must have a default (i.e. parameterless) constructor, and you can use the [XmlIgnore] attribute to prevent properties from being serialised into the database should you wish.
public class FilterParams{
public string Name {get;set;}
public string Age {get;set;}
[XmlIgnore]
public string PropertyYouDontWantToSerialise {get;set;}
}
Note: The SaveFilter returns Void and there is no error handling for brevity.
Rather than storing the querystring, I would serialize the FilterParams object as JSON/XML and store the result in your database.
Here's a JSON Serializer I regularly use:
using System.IO;
using System.Runtime.Serialization.Json;
using System.Text;
namespace Fabrik.Abstractions.Serialization
{
public class JsonSerializer : ISerializer<string>
{
public string Serialize<TObject>(TObject #object) {
var dc = new DataContractJsonSerializer(typeof(TObject));
using (var ms = new MemoryStream())
{
dc.WriteObject(ms, #object);
return Encoding.UTF8.GetString(ms.ToArray());
}
}
public TObject Deserialize<TObject>(string serialized) {
var dc = new DataContractJsonSerializer(typeof(TObject));
using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(serialized)))
{
return (TObject)dc.ReadObject(ms);
}
}
}
}
You can then deserialize the object and pass it your data access code as per your example above.
You didn't mention about exact purpose of storing the filter.
If you insist to save filter into a database table, I would have following structure of the table.
FilterId
Field
FieldValue
An example table might be
FilterId Field FieldValue
1 Name Tom
1 Age 24
1 Location IL
3 Name Mike
...
The answer is much more simple than you are making it:
Essentially you should store the raw query in its own table and relate it to your People table. Don't bother storing individual filter options.
Decide on a value to store (2 options)
Store the URL Query String
This id be beneficial if you like open API-style apps, and want something you can pass nicely back and forth from the client to the server and re-use without transformation.
Serialize the Filter object as a string
This is a really nice approach if your purpose for storing these filters remains entirely server side, and you would like to keep the data closer to a class object.
Relate your People table to your Query Filters Table:
The best strategy here depends on what your intention and performance needs are. Some suggestions below:
Simple filtering (ex. 2-3 filters, 3-4 options each)
Use Many-To-Many because the number of combinations suggests that the same filter combos will be used lots of times by lots of people.
Complex filtering
Use One-To-Many as there are so many possible individual queries, it less likely they are to be reused often enough to make the extra-normalization and performance hit worth your while.
There are certainly other options but they would depend on more detailed nuances of your application. The suggestions above would work nicely if you are say, trying to keep track of "recent queries" for a user, or "user favorite" filtering options...
Personal opinion
Without knowing much more about your app, I would say (1) store the query string, and (2) use OTM related tables... if and when your app shows a need for further performance profiling or issues with refactoring filter params, then come back... but chances are, it wont.
GL.
In my opinion the best way to save the "Filter" is to have some kind of json text string with each of the "columns names"
So you will have something in the db like
Table Filters
FilterId = 5 ; FilterParams = {'age' : '>18' , ...
Json will provide a lot of capabilities, like the use of age as an array to have more than one filter to the same "column", etc.
Also json is some kind of standard, so you can use this "filters" with other db some day or to just "display" the filter or edit it in a web form. If you save the Query you will be attached to it.
Well, hope it helps!
Assuming that a nosql/object database such as Berkeley DB is out of the question, I would definitely go with option 1. Sooner or later you'll find the following requirements or others coming up:
Allow people to save their filters, label, tag, search and share them via bookmarks, tweets or whatever.
Change what a parameter means or what it does, which will require you to version your filters for backward compatibility.
Provide auto-complete functions over filters, possibly using a user's filter history to inform the auto-complete.
The above will be somewhat harder to satisfy if you do any kind of binary/string serialization where you'll need to parse the result and then process them.
If you can use a NoSql DB, then you'll get all the benefits of a sql store plus be able to model the 'arbitrary number of key/value pairs' very well.
Have thought about using Profiles. This is a build in mechanism to store user specific info. From your description of your problem its seems a fit.
Profiles In ASP.NET 2.0
I have to admit that M$ implementation is a bit dated but there is essentially nothing wrong with the approach. If you wanted to roll your own, there's quite a bit of good thinking in their API.