I'm doing some experimentation with Falcor (falcor-router), and wondering why it sometimes returns 'atom' values.
For example I'm asking for this path: ['items',{from: 0, to: 100},['name', 'value']]
Only 50 items exist for this path, so the only paths items.0..50 are populated.
The resulting jsonGraph contains the correct values for items.0..50, but also contains values for items.51..100 that look like this:
51: {
name: {
type: "$atom"
},
value: {
type: "$atom"
}
}
I would expected that items.51..100 would not be present in the jsonGraph instead of being populated with these atom values.
Why is this happening and what can I do to avoid it?
Presently routes which produce no data are materialized to empty atoms so the client knows that those values truly are undefined (as opposed just not cached yet).
The JSONGraph merge needs to handle the case that the response may be only a subset of the paths that were requested. If you request for three paths, but the JSONGraph response only has data for two of the paths, the JSONGraph merge knows that there's still one left, and retries the request (but only for the missing path). This process is repeated until we've hit our retry limit, or we've received data for all the paths requested.
So materializing is our way of letting the cache know that it shouldn't build missing paths and re-request for this data.
I believe we have a plan to move away from materializing soon, and only communicate the paths for which no data was returned.
https://github.com/Netflix/falcor-router/issues/189#issuecomment-247722403
Related
I'd like to use a personnal API for named entity recognition (NER), and use brat for visualisation. It seems brat offers an Automatic annotation tool, but documentation about its configuration is sparse.
Are there available working examples of this features ?
Could someone explain me what should be the format of the response of the API ?
I finally manage to understand how it works, thanks to this topic in the GoogleGroup diffusion list of BRAT
https://groups.google.com/g/brat-users/c/shX1T2hqzgI
The text is sent to the Automatic Annotator API as a byte string in the body of a POST request, and the format BRAT required in response from this API is in the form of a dictionary of dictionaries, namel(
{
"T1": {
"type": "WhatEverYouWantString", # must be defined in the annotation.conf file
"offsets": [(0, 2), (10, 12)], # list of tuples of integers that correspond to the start and end position of
"texts": ["to", "go"]
}
"T2" : {
"type": "SomeString",
"offsets":[(start1, stop1), (start2, stop2), ...]
"texts":["string[start1:stop1]", "string[start2:stop2]", ...
}
"T3" : ....
}
THEN, you put this dictionary in a JSON format and you send it back to BRAT.
Note :
"T1", "T2", ... are mandatory keys (and corresponds to the Term index in the .ann file that BRAT generates during manual annotation)
the keys "type", "offsets" and "texts" are mandatory, otherwise you get some error in the log of BRAT (you can consult these log as explained in the GoogleGroup thread linked above)
the format of the values are strict ("type" gets a string, "offsets" gets a list of tuple (or list) or integers, "texts" gets a list of strings), otherwise you get BRAT errors
I suppose that the strings in "texts" must corresponds to the "offsets", otherwise there should be an error, or at least a problem with the display of tags (this is already the case if you generate the .ann files from an automatic detection algorithm and have different start and stop than the associated text)
I hope it helps. I managed to make the API using Flask this morning, but I needed to construct a flask.Response object to get the correct output format. Also, the incoming format from BRAT to the Flask API could not be catch until I used a flask.request object with request.get_body() method.
Also, I have to mention that I was not able to use the examples given in the BRAT GitHub :
https://github.com/nlplab/brat/blob/master/tools/tokenservice.py
https://github.com/nlplab/brat/blob/master/tools/randomtaggerservice.py
I mean I could not make them working, but I'm not familiar at all with API and HTTP packages in Python. At least I figured out what was the correct format for the API response.
Finally, I have no idea how to make relations among entities (i.e. BRAT arrows) format from the API, though
https://github.com/nlplab/brat/blob/master/tools/restoataggerservice.py
seems to work with such thing.
The GoogleGroup discussion
https://groups.google.com/g/brat-users/c/lzmd2Nyyezw/m/CMe9FenZAAAJ
seems to mention that it is not possible to send relations between entities back from the Automatic Annotation API and make them work with BRAT.
I may try it later :-)
A url without parameters but with a question mark appended at the end is passed to the client to be parsed and used.
I've been told the client should be robust enough to handle such url and proceed. But shouldn't this be fixed server-side?
thanks
An empty query part is not an error, so it definitely needs to be accepted by the client. (Reference: RFC 3986 section 3.4 which shows the syntax of the query as 0 or more allowable characters.)
An empty query is different from an undefined query (i.e., the URI does not contain a ?). If the base URI contains a query component, merging a relative URI with an empty query will override the base URI's query, whereas merging a relative URI without a query will copy the base URI's query into the merged result.
I'm currently coding a transition from a system that used hand-crafted JSON files to one that can automatically generate the JSON files. The old system works; the new system works; what I need to do is transfer data from the old system to the new one.
The JSON files are used by an iOS app to provide functionality, and have never been read by our server software in Ruby On Rails before. To convert between the original system and the new system, I've started work on parsing the existing JSON files.
The problem is that one of my first two sample files has trailing commas in the JSON:
{ "sample data": [1, 2, 3,] }
This apparently went through just fine with the iOS app, because that file has been in use for a while. Now I need some way to parse the data provided in the file in my Ruby on Rails server, which (quite rightfully) throws an exception over the illegal trailing comma in the JSON file.
I can't just JSON.parse the code, because the parser, quite rightfully, rejects it as invalid JSON. Is there some way to parse it -- either an option I can pass to JSON.parse, or a gem that adds something, etc etc? Or do I need to report back that we're going to have to hand-fix the broken files before the automated process can process them?
Edit:
Based on comments and requests, it looks like some additional data is called for. The JSON files in question are stored in .zip files on S3, stored via ActiveStorage. The process I'm writing needs to download, unpack, and parse the zip files, using the 'manifest.json' file as a key to convert the archived file into a database structure with multiple, smaller files stored on S3 instead of a single zip that contains everything. A (very) long term goal is for clients to stop downloading a unitary zip file, and instead download the files individually. The first step towards that is to break the zip files up on the server, which means the server needs to read in the zip files. A more detailed sample of the data follows. (Note that the structure contains several design decisions I later came to regret; one of the original ideas was to be able to re-use files rather than pack multiple copies of the same identical file, but YAGNI bit me in the rear there)
The following includes comments that are not legal in JSON format:
{
"defined_key": [
{
"name": "Object_with_subkeys",
"key": "filename",
"subkeys": [
{
"id":"1"
},
{
"id":"2"
},
{
"id":"3" // references to identifier on another defined key
}, // Note trailing comma
]
}
],
"another_defined_key":[
{
"identifier": "should have made parent a hash with id as key instead of an array",
"data":"metadata",
"display_name":"Names: Can be very arbitrary",
"user text":"Wait for the right {moment}", // I actually don't expect { or } in the strings, but they're completely legal and may have been used
"thumbnail":"filename-2.png",
"video-1":"filename-3.mov"
}
]
}
The problem is that your are trying to parse something that looks a lot like JSON but is not actually JSON as defined by the spec.
Arrays- An array structure is a pair of square bracket tokens surrounding zero or more values. The values are separated by commas.
Since you have a trailing comma another value is also expected and most JSON parsers will raise an error due to this violation
All that being said json-next will parse this appropriately maybe give that a shot.
It can parse JSON like representations that completely violate the JSON spec depending on the flavor you use. (HanSON, SON, JSONX as defined in the gem)
Example:
json = "{ \"sample data\": [1, 2, 3,] }")
require 'json/next'
HANSON.parse(json)
#=> {"sample data"=>[1, 2, 3]}
but the following is equivalent and completely violates spec
JSONX.parse("{ \"sample data\": [1 2 3] }")
#=> {"sample data"=>[1, 2, 3]}
So if you choose this route do not expect to use this to validate the JSON data or structure in any fashion and you could end up with unintended results.
How do I clear/remove query string parameters, which my MVC action, doesn't require/support?
For instance, my action requires, say an id and a bool flag, so the url would be something like: http://localhost:someport/controller/action/?id=1&remove=true
But, if a user types in something like, http://localhost:someport/controller/action/?id=1&remove=true&some-junk-param=0
Then, I want the some-junk-param to be removed and not shown in the address bar, when the request is processed.
Any thoughts?
If you need to get rid of unwanted query string parameters, you have two general options:
Do it on server-side. You can achive this only with redirection, that means when browser asks URL with bad query string, server redirects browser to URL with good query string.
Caveats:
In this case we have redundant query just for cleaning query string.
User will have trash in browser history.
Do it on client-side. ASP.NET MVC Model binder will get only expected parameters from query string, so it's nothing bad with having other values in query string. You can check your URL on client-side with javascript and rewrite it with or without changing history using History API (IE10+).
Caveats:
In this case you will have to support consistency about allowed parameters between JS and C# code
Of course every way is suitable for it's own cases, but looking at caveats the second way is better, because it affects developer expirience whereas first way affects user expirience.
As we all know, file uploading is most often accomplished using POST method. So, why can't the GET method be used for file uploads instead? Is there a specific prohibition against HTTP GET uploads?
GET requests may contain an entity body
RFC 2616 does not prevent an entity body as part of a GET request. This is often misunderstood because PHP muddies the waters with its poorly-named $_GET superglobal. $_GET technically has nothing to do with the HTTP GET request method -- it's nothing more than a key-value list of url-encoded parameters from the request URI query string. You can access the $_GET array even if the request was made via POST/PUT/etc. Weird, right? Not a very good abstraction, is it?
Why a GET entity body is a bad idea
So what does the spec say about the GET method ... well:
In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered "safe."
So the important thing with GET is to make sure any GET request is safe. Still, the prohibition is
only "SHOULD NOT" ... technically HTTP still allows a GET requests to result in an action that isn't
strictly based around "retrieval."
Of course, from a semantic standpoint using a method named GET to perform an action other than
"getting" a resource doesn't make very much sense either.
When a GET entity body is flat-out wrong
Regarding idempotence, the spec says:
Methods can also have the property of "idempotence" in that (aside from error or expiration issues)
the side-effects of N > 0 identical requests is the same as for a single request. The methods GET,
HEAD, PUT and DELETE share this property.
This means that a GET method must not have differing side-effects for multiple requests for the
same resource. So, regardless of the entity body present as part of a GET request, the side-effects
must always be the same. In layman's terms this means that if you send a GET with an entity body
100 times the server cannot create 100 new resources. Whether sent once or 100 times the request must
have the same result. This severely limits the usefulness of the GET method for sending entity bodies.
When in doubt, always fall back to the safety/idempotence tests when evaluating the efficacy
of a method and its resulting side-effects.
In case of GET Method
Appends form-data into the URL in name/value pairs and length of URL is limited(3000 characters).
File content can't be put inside a URL parameter using a form.So use POST
In Get method, the value of action, appends a `?' to it, then appends the form data set, encoded using the "application/x-www-form-urlencoded" content type. The user agent then traverses the link to this URI. In this scenario, form data are restricted to ASCII codes.
So, that file upload is not possible in GET Method