I'm trying to post a twitter status update with clojure... but this probably really is a question about oAuth headers, and using it through the wonderful clj-http library.
I've used clj-http before for basic-auth and other type of headers and it was fairly straightforward. The Authorization: Bearer ... was a bit confusing but I got there in the end.
For twitter though, I am supposed to pass a lot of variables in Authorization and I am not quite sure how I'd go about doing that. Here's the curl command, according to twitter, that I'll need to post the tweet:
curl --request 'POST' 'https://api.twitter.com/1.1/statuses/update.json'
--data 'status=this+is+sparta'
--header 'Authorization: OAuth oauth_consumer_key="xxxxx",
oauth_nonce="xxxxx",
oauth_signature="xxxxx",
oauth_token="xxxxx", oauth_version="1.0"'
--verbose
So I am not sure how I would append all the oAuth... things in the header. trying with (str ..) isn't really working out. Here's what I've got so far:
(client/post "https://api.twitter.com/1.1/statuses/update.json")
{:headers {:Authorization (str "OAuth oauth_consumer_key='xxxxx', oauth_nonce='xxxxx', oauth_signature='xxxxx', oauth_token='xxxxx', oauth_version='1.0'" )}
:form-params {:status "This is sparta"})
This returns 403. permission error when I try.
Any ideas on how I'd construct that query?
ps: I do have the oAuth token and token_secret for the account... but I notice the token_secret value isn't being passed? and what does oauth_nonce for? I'm for now passing the value that twitter gave me for curl... looking around it seems that it is just a random value so not that fussed about it.
It might be worth taking a look at Matt Revelle's OAuth client library for Clojure. https://github.com/mattrepl/clj-oauth
OAuth is sufficiently non-trivial to make it tiresome to hack together something from bare bones. If nothing else and you still want to go the minimal route, you might get some ideas.
I've had difficulty in the past trying to use clj-oauth (mostly because of my own understanding of clojure) but I found twitter-api pretty straightforward to use. It makes use of clj-oauth internally.
Related
First you must know I'm a total beginner, I'm trying to learn so I almost don't know anything.
On the basic page of the API, there is a curl command used as an example to show us how to make requests.
I'm using Ruby on Rails so I used "curl-to-ruby" website to translate it, but it did not work as expected.
I wanted it to show me this :
uri = URI.parse("REQUEST_URL")
response = JSON.parse(Net::HTTP.get(uri))
Instead I got this :
uri = URI.parse("REQUEST_URL")
response = Net:HTTP.get_response(uri)
I don't understand any of this, I thought I wouldn't need to and just use "curl-to-ruby", but apparently I really need to get this.
Would you please try to explain me ?
Or give me links ?
Or matters to read (curl, API, http) ?
Thank you very much, have a nice day.
It's because that command doesn't return just the content, it returns the whole HTTP response object including headers and body. You need to extract the response body and parse that using JSON.parse(), e.g.
JSON.parse(response.body)
See documentation here: https://docs.ruby-lang.org/en/2.0.0/Net/HTTP.html#method-c-get_response
(Also, there is nothing in the cURL command which would hint to the converter that the content-type of the response was expected to be JSON (e.g. perhaps an "accepts" header or something), so even if it were able to produce extra code adding the JSON.parse part, it has no way of knowing that it would be appropriate to do so in this case.)
I am using two different HTTP POST utilities (poster out of Firefox as well as Python requests API) to post a simple SPARQL insert to Virtuoso.
My URL is: http://localhost:8890/sparql
My request parameters are:
default-graph-uri: <MY_GRAPH>
should-sponge: soft
debug: on
timeout:
format: application/xml
save: display
fname:
I put the actual SPARQL (INSERT DATA { GRAPH...) in the content of the message.
I tried different content types, none of which worked. I do get 200 but the response is in HTML even though the above parameter set specifies application/xml, however, no data is inserted. When I try content type of text/turtle, I get 409 Invalid Path, which is also referenced in this post.
I can successfully do HTTP GET, however, that has a payload length limitation which I would like to exceed for performance reasons. The only difference with the GET is that the SPARQL goes in the URL under query parameter and the POST should enable a much larger payload in the message content, by including multiple triples in the same request, not just one (I have 100s of 1000s of inserts). I was trying to follow this documentation page.
I stopped by this question days ago trying to achieve the same with curl. Since it is a powerful (and far more convenient) alternative to browser extensions, here is the formulation that eventually proved successful:
curl -X POST \
-H "Content-Type:application/sparql-update" \
-H "Accept:text/html" \
--data "select distinct ?Concept where {[] a ?Concept} LIMIT 100" http://localhost:8890/sparql
More details on the headers in this thread.
If you are using python, I would avoid using the requests library. There are some dedicated libraries for RDF which abstract the process and make your life easier.
Try:
SPARQLWrapper
RDFLib
They are both form the same family of packages from rdflib
Based on experience, I find the SPARQLWrapper significantly simpler and easier to use for your use case. It's an abstracted version of RDFLib. The docs suggest something like this could work:
from SPARQLWrapper import SPARQLWrapper, POST
sparql = SPARQLWrapper("https://example.org/sparql")
sparql.setCredentials("some-login", "some-password") # if required
sparql.setMethod(POST) # this is the crucial option
sparql.setQuery("""
<QUERY GOES HERE>
""".format(PARSE SOME VARS INTO THE QUERY HERE IF YOU WANT)
)
results = sparql.query()
print results.response.read()
Make sure you add the option for POST. You should be doing bulk I/O in no time :).
There are many aspects to this "question" making it difficult to provide a simple answer, suitable to this site. This is one of the reasons I suggested the mailing list, which is better suited to conversational and/or multi-facet assistance.
Have you tried using curl as most of our examples do?
Looking at the Poster page on Mozilla Add-Ons, I see that you may need to manually add a ? to the end of your target URI -- so http://localhost:8890/sparql? rather than http://localhost:8890/sparql -- and it's not clear whether you've done that in your testing. On the project page, I also note its last commit was in 2012, and there are a great many open issues.
I'm not at all familiar with Python, so I've not dug in there.
Have you tried setting an Accept: header? This can have significant impact on the content returned by the server.
If I understand your described efforts correctly, your format: query parameter should be output-format:, and its value should not be application/xml but one of the supported formats listed in the documentation.
Neither the virtuoso-users post you referenced nor this question have enough detail to analyze the cause of the 409 Invalid Path error. Explicit details that allow us to reproduce this result would be helpful, optimally in a distinct thread.
This seems to be a Virtuoso specific issue. You can only post a query by using content type "application/sparql-update" instead of "application/sparql-query" which is common.
The request is done as follows with Python:
headers = {
'Content-Type': 'application/sparql-update',
'Accept': 'application/json'
}
s = Session();
s.mount(server_url, HTTPAdapter(max_retries=1))
response: Response = s.post(server_url, data=<sparql_string>, headers=headers, timeout=100)
return response.json();
I see a lot of useful methods in the API, but I don't find any method to list all my Posts, or all the posts from within a publication. Is this intentional?
Thought it would be something really obvious to exist in the API. Or am I missing something?
Got it, just use the RSS feed instead.
I wrapped a Github package by #mark-fasel into a Clay microservice that enables you to do exactly this:
Simplified Return Format:
https://clay.run/services/nicoslepicos/medium-get-users-posts-simple
What Medium actually returns at the endpoint
https://clay.run/services/nicoslepicos/medium-get-users-posts
I put together a little fiddle, since a user was asking how to use the endpoint in HTML to get the titles for their last 3 points:
https://jsfiddle.net/h405m3ma/1/
You can call the API as:
curl -i -H "Content-Type: application/json" -X POST -d '{"username":"nicolaerusan"}' https://clay.run/services/nicoslepicos/medium-get-users-posts-simple
You can also use it easily in your node code using the clay-client npm package and just write:
Clay.run('nicoslepicos/medium-get-users-posts-simple', {"username":"usernameValue"}) .then((result) => {
// Do what you want with returned result console.log(result);
})
If you need to generally pull down an RSS feed, here's a microservice for that:
https://clay.run/services/nicoslepicos/rss-to-json
Can someone point me to good resource for Net::HTTP? I'm trying to understand why certain code functions the way it does. For example:
def url_check(domain)
parsed = URI.parse(domain).host
check = Net::HTTP.new(parsed).head('/').kind_of? Net::HTTPOK
( check == true ? "up" : "down" )
end
I understand 95% of the above code, but I can't find any resources that explain what .head('/') is doing. I'm hoping someone can point me to a good resource that is beginner friendly.
HEAD is an HTTP command that returns just the http headers.
head("/") probably just returns the http headers sent by the server in response to request uri "/", ie the root of the website. It is commonly used to do a quick check to see if the page and/or site exists without fetching the entire html page.
You probably also need to learn something about HTTP protocol as well.
GET, POST, HEAD, SET, PUT, DELETE, TRACE are some common ones that come to my head right now there are couple more. You will have better understanding of the code once you understand the basics of HTTP.
I'm trying to get Twitter authentication working on my ASP.NET site. When you create the app on the Twitter website, you have to specify a callback URL, which for sake of argument, I have set to http://mydomain.com
I've read the oAuth 1.0a spec, and to override this callback URL with your own custom one you have to send the oauth_callback parameter in the request_token phase (url-encoded of course).
So my request URL looks like this:
http://twitter.com/oauth/request_token?oauth_callback=http%3A%2F%2Fmydomain.com%2Ftwittercallback
Supposedly, if all goes to plan, in your response data, you are supposed to receive a new parameter of oauth_callback_confirmed=true in addition to your token and token secret parameters.
However, my response comes through as:
oauth_token=MYTOKEN&oauth_token_secret=MYTOKENSECRET
I know I haven't given you guys the greatest amount to go on, but I'm at my wits end as to why I am not receiving the oauth_callback_confirmed parameter. Without this, my application keeps defaulting back to the callback URL hard-coded on the Twitter website. Please if anyone could help me out, I will be eternally grateful!
Thanks,
A.
I've read the oAuth 1.0a spec, and to
override this callback URL with your
own custom one you have to send the
oauth_callback parameter in the
request_token phase (url-encoded of
course).
So my request URL looks like this:
http://twitter.com/oauth/request_token?oauth_callback=http%3A%2F%2Fmydomain.com%2Ftwittercallback
just because YOU read the spec doesn't mean that TWITTER read it. :P
kidding - this is essentially correct - but the way twitter likes to receive this data is a little different (and not well documented).
the way i've found to get the oauth_callback to confirm is as follows: specify the oauth_callback in the parameters of the request function, NOT within the URL.
python example (using oauth2):
''' Create our client.'''
client = oauth.Client(consumer)
''' do the request '''
resp, content = client.request(request_token_url,"POST",body=urllib.urlencode({'oauth_callback':callbackURL}))
''' note that it's called "body" in this particular OAuth function for Client but in OAuth Request object it's called "parameters." YMMV depending on programming language/ library of course. '''
this is ALSO the only way i've managed to get an oauth verifier back. supposedly one should not have to specify the callback URL every time, since we provide it in app settings...but experience seems to indicate otherwise.
finally, please be aware that at leg 3 you have to do the same thing AGAIN - this time including the oauth_verifier as well as the callback URL in the parameters.
hope this helps - can't begin to tell you how much effort i put into figuring this out.
good luck!
J
I've used this guide to set up my PC to be used as the callback location. Basically you set up your hosts file in a certain way, clear your cache and add a couple of Firefox registry values. At the end when you are debugging an oauth call the redirect comes back to your local PC.
As I said it worked for me.
<?php
// oauth-php example
$token = OAuthRequester::requestRequestToken(
$consumer_key,
$user_id,
array('oauth_callback'=> urlencode($callback_uri))
);
?>