I am using two different HTTP POST utilities (poster out of Firefox as well as Python requests API) to post a simple SPARQL insert to Virtuoso.
My URL is: http://localhost:8890/sparql
My request parameters are:
default-graph-uri: <MY_GRAPH>
should-sponge: soft
debug: on
timeout:
format: application/xml
save: display
fname:
I put the actual SPARQL (INSERT DATA { GRAPH...) in the content of the message.
I tried different content types, none of which worked. I do get 200 but the response is in HTML even though the above parameter set specifies application/xml, however, no data is inserted. When I try content type of text/turtle, I get 409 Invalid Path, which is also referenced in this post.
I can successfully do HTTP GET, however, that has a payload length limitation which I would like to exceed for performance reasons. The only difference with the GET is that the SPARQL goes in the URL under query parameter and the POST should enable a much larger payload in the message content, by including multiple triples in the same request, not just one (I have 100s of 1000s of inserts). I was trying to follow this documentation page.
I stopped by this question days ago trying to achieve the same with curl. Since it is a powerful (and far more convenient) alternative to browser extensions, here is the formulation that eventually proved successful:
curl -X POST \
-H "Content-Type:application/sparql-update" \
-H "Accept:text/html" \
--data "select distinct ?Concept where {[] a ?Concept} LIMIT 100" http://localhost:8890/sparql
More details on the headers in this thread.
If you are using python, I would avoid using the requests library. There are some dedicated libraries for RDF which abstract the process and make your life easier.
Try:
SPARQLWrapper
RDFLib
They are both form the same family of packages from rdflib
Based on experience, I find the SPARQLWrapper significantly simpler and easier to use for your use case. It's an abstracted version of RDFLib. The docs suggest something like this could work:
from SPARQLWrapper import SPARQLWrapper, POST
sparql = SPARQLWrapper("https://example.org/sparql")
sparql.setCredentials("some-login", "some-password") # if required
sparql.setMethod(POST) # this is the crucial option
sparql.setQuery("""
<QUERY GOES HERE>
""".format(PARSE SOME VARS INTO THE QUERY HERE IF YOU WANT)
)
results = sparql.query()
print results.response.read()
Make sure you add the option for POST. You should be doing bulk I/O in no time :).
There are many aspects to this "question" making it difficult to provide a simple answer, suitable to this site. This is one of the reasons I suggested the mailing list, which is better suited to conversational and/or multi-facet assistance.
Have you tried using curl as most of our examples do?
Looking at the Poster page on Mozilla Add-Ons, I see that you may need to manually add a ? to the end of your target URI -- so http://localhost:8890/sparql? rather than http://localhost:8890/sparql -- and it's not clear whether you've done that in your testing. On the project page, I also note its last commit was in 2012, and there are a great many open issues.
I'm not at all familiar with Python, so I've not dug in there.
Have you tried setting an Accept: header? This can have significant impact on the content returned by the server.
If I understand your described efforts correctly, your format: query parameter should be output-format:, and its value should not be application/xml but one of the supported formats listed in the documentation.
Neither the virtuoso-users post you referenced nor this question have enough detail to analyze the cause of the 409 Invalid Path error. Explicit details that allow us to reproduce this result would be helpful, optimally in a distinct thread.
This seems to be a Virtuoso specific issue. You can only post a query by using content type "application/sparql-update" instead of "application/sparql-query" which is common.
The request is done as follows with Python:
headers = {
'Content-Type': 'application/sparql-update',
'Accept': 'application/json'
}
s = Session();
s.mount(server_url, HTTPAdapter(max_retries=1))
response: Response = s.post(server_url, data=<sparql_string>, headers=headers, timeout=100)
return response.json();
Related
Whenever searching for this I find resources on how to specify the media type of a resource that the schema defines, but I can't see an answer on what the actual media type of the schema itself is.
Given the way HTTP works, it makes sense to me that if I request the right content type with the Accept header, my server can respond appropriately.
Thus, if I request /products with Accept: application/json I would get products in JSON format, but if I requested openapi-whatever I would get the OpenAPI schema.
I think I can probably use either application/openapi+json or application/openapi+yaml, but I can't see anything about it in the actual specification.
I'm not sure whether or not I actually want to use the Accept header for this difference, but I certainly want to respond with the correct Content-Type header in any case.
The OpenAPI Initiative's Technical Steering Committee (TSC) approved the following media types:
application/vnd.oai.openapi (YAML variant)
application/vnd.oai.openapi+json (JSON only variant)
with an optional version parameter:
application/vnd.oai.openapi;version=2.0
However, these media types are not yet registered with IANA.
This seems to be newer (Sept. 2021):
application/openapi+yaml
application/openapi+json
https://www.ietf.org/archive/id/draft-polli-rest-api-mediatypes-00.html
First you must know I'm a total beginner, I'm trying to learn so I almost don't know anything.
On the basic page of the API, there is a curl command used as an example to show us how to make requests.
I'm using Ruby on Rails so I used "curl-to-ruby" website to translate it, but it did not work as expected.
I wanted it to show me this :
uri = URI.parse("REQUEST_URL")
response = JSON.parse(Net::HTTP.get(uri))
Instead I got this :
uri = URI.parse("REQUEST_URL")
response = Net:HTTP.get_response(uri)
I don't understand any of this, I thought I wouldn't need to and just use "curl-to-ruby", but apparently I really need to get this.
Would you please try to explain me ?
Or give me links ?
Or matters to read (curl, API, http) ?
Thank you very much, have a nice day.
It's because that command doesn't return just the content, it returns the whole HTTP response object including headers and body. You need to extract the response body and parse that using JSON.parse(), e.g.
JSON.parse(response.body)
See documentation here: https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html#method-c-get_response
(Also, there is nothing in the cURL command which would hint to the converter that the content-type of the response was expected to be JSON (e.g. perhaps an "accepts" header or something), so even if it were able to produce extra code adding the JSON.parse part, it has no way of knowing that it would be appropriate to do so in this case.)
I see a lot of useful methods in the API, but I don't find any method to list all my Posts, or all the posts from within a publication. Is this intentional?
Thought it would be something really obvious to exist in the API. Or am I missing something?
Got it, just use the RSS feed instead.
I wrapped a Github package by #mark-fasel into a Clay microservice that enables you to do exactly this:
Simplified Return Format:
https://clay.run/services/nicoslepicos/medium-get-users-posts-simple
What Medium actually returns at the endpoint
https://clay.run/services/nicoslepicos/medium-get-users-posts
I put together a little fiddle, since a user was asking how to use the endpoint in HTML to get the titles for their last 3 points:
https://jsfiddle.net/h405m3ma/1/
You can call the API as:
curl -i -H "Content-Type: application/json" -X POST -d '{"username":"nicolaerusan"}' https://clay.run/services/nicoslepicos/medium-get-users-posts-simple
You can also use it easily in your node code using the clay-client npm package and just write:
Clay.run('nicoslepicos/medium-get-users-posts-simple', {"username":"usernameValue"}) .then((result) => {
// Do what you want with returned result console.log(result);
})
If you need to generally pull down an RSS feed, here's a microservice for that:
https://clay.run/services/nicoslepicos/rss-to-json
Can someone point me to good resource for Net::HTTP? I'm trying to understand why certain code functions the way it does. For example:
def url_check(domain)
parsed = URI.parse(domain).host
check = Net::HTTP.new(parsed).head('/').kind_of? Net::HTTPOK
( check == true ? "up" : "down" )
end
I understand 95% of the above code, but I can't find any resources that explain what .head('/') is doing. I'm hoping someone can point me to a good resource that is beginner friendly.
HEAD is an HTTP command that returns just the http headers.
head("/") probably just returns the http headers sent by the server in response to request uri "/", ie the root of the website. It is commonly used to do a quick check to see if the page and/or site exists without fetching the entire html page.
You probably also need to learn something about HTTP protocol as well.
GET, POST, HEAD, SET, PUT, DELETE, TRACE are some common ones that come to my head right now there are couple more. You will have better understanding of the code once you understand the basics of HTTP.
I have a relatively simple (I think) use-case but I can't find any examples where someone has done this. We are using Varnish as a cache and reverse proxy in front of two different applications and would like to make things a bit more unified across both as they both do similar things. I was hoping Varnish could help rewrite the URLs as shown below.
Original application URL for pagination (get first 10 items):
http://myapplication.com/products/?startindex=1&endindex=10
Desired URL:
http://myapplication.com/products/?paginate=1:10
This is just one example (the most complex because it combines two parameters), but in all cases the input values for the parameters stay the same, it is just that the parameter names will change.
Another example would be:
http://myapplication.com/search/?query=something
to:
http://myapplication.com/search/?q=something
Does anyone have any experience with varnish and how this could be done?
Thanks
Apparently you can. The answer is that regsub is your friend.
For example:
if (req.url ~ "(.*)(id=)") {
set req.url = regsub(req.url, "(feeds/[a-zA-Z]*/)(.*)([\?|&])(id=)([a-zA-Z0-9]*)(.*)", "\1\2\3byGuid=\5\6");
}
This will convert and incoming "id" parameter into a "byGuid" parameter on the backend. t also does a bunch of stuff with the rest of the URL string but the basics are there. SO if anyone wants to do something similar this is a good starting point.