I have a remote TIKA server set up and I'm trying to use it from within a RoR application. I need to pull a file from a remote location and send it on to the Tika server. The wiki for TikaJAXRS gives an example using curl, but I have not been able to get that to work. What does work is this:
curl https://mydomain.s3.amazonaws.com/uploads/testdocument.docx | curl -v -i -X PUT -T - ec2...154.uswest2.compute.amazonaws.com:9998/tika
How do I render this in my Rails app using net::http? I've successfully written a GET request with net::http to the Tika server from the Rails app and gotten back the expected result, but the documentation on PUT is a bit sparse. (The server does require a PUT rather than POST.)
BTW, if anyone knows how to make that last example in that wiki work and render it in net::http, that would be even better!
Addendum:
Here's what I have in the RoR app that doesn't work:
ENDPOINT = "http://ec2...154.us-west-2.compute.amazonaws.com:9998"
file = "https://mydomain.s3.amazonaws.com/uploads/testdocument.docx"
uri = URI.parse(endpoint)
#http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Put.new("/tika")
request.body = URI.parse(file).read
#response = #http.request(request)
and I get back a code 415
I need to know how to change this code to do what the curl commands (curl remote_file piped to curl PUT) are doing successfully.
Update
After a couple of days of fruitless attempts on this, I have a workaround:
gem 'curb'
#response = Curl.put("http://ec2...154.us-west-2.compute.amazonaws.com:9998/tika",
Curl.get("https://mydomain.s3.amazonaws.com/uploads/testdocument.docx").body_str)
While this does provide a solution to my immediate problem, I still want to know how to implement this same functionality more directly by using Net::HTTP.
Related
I have an api method in a Rails controller like the following:
def login
if !request || !request.headers
render :json => {:error => I18n.t('error_must_provide_api_key_or_token')}, :status => :unauthorized and return
end
api_key = params['X-Api-Key']
if api_key.nil?
api_key = request.headers['X-Api-Key']
end
... rest of method ...
The method first checks to see if we have a request, and whether that request has headers. Then, what I want to do is check for a header variable called X-Api-Key. I first check the params hash, and, if there isn't one found there, I then check the request.headers hash.
What I don't understand is why I have to check both of these. Previously, I had:
api_key = request.headers['X-Api-Key']
This works when I'm debugging on my local machine, but it doesn't work once I push to my production server and run in production mode. Conversely, the following:
api_key = params['X-Api-Key']
Works when I push to the production server, but doesn't work when running locally.
My local machine is running MacOS, and rbenv 1.1.0 with ruby 2.4.0p0 and rails 5.1.1.
My server is Ubuntu 16.04 and running ruby 2.1.4p265 with rails 4.2.5.
Request headers are coming from the http headers.
While params are from the body of the HTTP (if its not GET method), or from the encoded url (e.g: something like http://localhost:3000/cars?a=b, then params[:a] would give 'b').
e.g (using curl, -H stands for header)
using url encoded for the params,
curl -H 'X-Api-Key: 1234' http://localhost:3000/cars?X-Api-Key=abc
or using http request body, with application/json as content-type.
curl -X GET -H 'X-Api-Key: 1234' -H "Content-Type: application/json" -d '{"X-Api-Key": "abc" }' http://localhost:3000/cars
Then, request.headers['X-Api-Key] would returns 1234
while params['X-Api-Key'] would returns abc.
So what happened here is likely, in local you are sending the X-Api-Key, through http header, while in production you are sending it through url encoded or http request body as content-type as json.
To solve this,
if you want to support both ways, to feed X-Api-Key
through http header
http encoded params or json params in the http body
Then what you wrote above make sense.
if you want to only support http header, then change your production client code, to send X-Api-Key through http request header.
if you want to only support params, then change your local code, to send X-Api-Key, through http request body or encoded url.
I have a very simple API, that I would like to make a POST to using ruby and NOT using a GEM just the built in libraries net/http, uri, and openssl if needed.
Anyway, I am using the code below to make a very simple POST request but am getting some VERY strange results and was hoping someone else has seen this.
I have also tested the same request below in POSTMAN and NodeJS and BOTH work as expected, the only one I can not get to work is Ruby.
require 'uri'
require 'net/http'
require 'openssl'
url = URI("https://somesite.dev/devices")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
request["key"] = '1234567'
response = http.request(request)
puts response.read_body
The result is something I have not seen before: I am getting the header key twice... So the log to the API shows a query like this:
SELECT * FROM device where key = '1234567, 1234567' LIMIT ...
As stated above I can make the same request via POSTMAN or NodeJS and get the correct result.
NOTE: Because I have a local copy of the API I can test locally BUT it's not SSL it's all over http. When making the request locally it works just fine. So from what I can tell this issue only presents it self when SSL is introduced.
Any help would be amazing! Thanks!!
Ruby Version 2.2.1
The issue was with something I did not list in my initial question. The API was using AWS API Gateway, and HTTP_PROXY was enabled on the method causing this strange issue. After I removed HTTP_PROXY the issue cleared up and the above code worked!
I will replace my command line
`curl -XPUT 'host:port/url' -d '{"val": "some_json"}'̀
by a Rails command, and get the result...
Somewhere like this :
response = call('put', 'host:port/url', '{"val" : "some_json"}')
Is there any predefined method to do this in Rails, or some gem ?
I know the command get of HTTP, but I will do a 'PUT' method.
Net::HTTP.get(URI.parse('host:port/url'))
Thanks for your replies
You can use Net::HTTP to send any standard http requests.
Here is a way, you can connect to any url ( http / https ), with any valid http methods with or without parameters.
def universal_connector(api_url, api_parameters={}, method="Get")
# Do raise Error, if url is invalid and Method is invalid
uri = URI(api_url)
req = eval("Net::HTTP::#{method.capitalize}.new('#{uri}')")
req.set_form_data(api_parameters)
Net::HTTP.start(uri.host, uri.port,:use_ssl => uri.scheme == 'https') do |http|
response = http.request(req)
return response.body
end
end
There are many alternatives available as well. Specifically, Faraday. Also, read this before making a choice.
#get is just a simple shortcut for the whole code (Net::HTTP Ruby library tends to be very verbose). However, Net::HTTP perfectly supports PUT requests.
Another alternative is to use an HTTP client as a wrapper. The most common alternatives are HTTParty and Faraday.
HTTParty.put('host:port/url', { body: {"val" : "some_json"} })
As a side note, please keep in mind that Rails is a framework, not a programming language. Your question is about how to perform an HTTP PUT request in Ruby, not Rails. It's important to understand the difference.
I'm trying to verify if there is a remote url with following code:
endpoint_uri = URI.parse(#endpoint.url)
endpoint_http = Net::HTTP.new(endpoint_uri.host, endpoint_uri.port)
endpoint_request = Net::HTTP::Head.new(endpoint_uri.request_uri)
endpoint_response = endpoint_http.request(endpoint_request)
I'm still getting 405 Method not allowed. When I use Get instead Head in Net::HTTP::Head.new I'm getting 200 Success but also with whole remote document in response what results in bigger response time (0.3s => 0.9s).
Any ideas why this is happening? Thx
There's a chance that the #endpoint url you're trying to interact with doesn't support HEAD requests (which would be really weird, but still may be the case). Your code works fine for me with a handful of urls (google.com, stackoverflow.com, etc.)
Have you tried a curl request to see what it returns?
curl -I http://www.the_website_you_want_to_test.com
I built a web service in using Spring framework in Java and have it run on a tc server on localhost. I tested the web service using curl and it works. In other words, this curl command will post a new transaction to the web service.
curl -X POST -H 'Accept:application/json' -H 'Content-Type: application/json' http://localhost:8080/BarcodePayment/transactions/ --data '{"id":5,"amount":5.0,"paid":true}'
Now, I am building a web app using RoR and would like to do something similar. How can I build that? Basically, the RoR web app will be a client that posts to the web service.
Searching SO and the web, I found some helpful links but I cannot get it to work. For example, from this post, he/she uses net/http.
I tried but it doesn't work. In my controller, I have
require 'net/http'
require "uri"
def post_webservice
#transaction = Transaction.find(params[:id])
#transaction.update_attribute(:checkout_started, true);
# do a post service to localhost:8080/BarcodePayment/transactions
# use net/http
url = URI.parse('http://localhost:8080/BarcodePayment/transactions/')
response = Net::HTTP::Post.new(url_path)
request.content_type = 'application/json'
request.body = '{"id":5,"amount":5.0,"paid":true}'
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request) }
assert_equal '201 Created', response.get_fields('Status')[0]
end
It returns with error:
undefined local variable or method `url_path' for #<TransactionsController:0x0000010287ed28>
The sample code I am using is from here
I am not attached to net/http and I don't mind using other tools as long as I can accomplish the same task easily.
Thanks much!
url = URI.parse('http://localhost:8080/BarcodePayment/transactions/')
response = Net::HTTP::Post.new(url_path)
Your problem is exactly what the interpreter told you it is: url_path is undeclared. what you want is to call the #path method on the url variable you declared in the previous line.
url = URI.parse('http://localhost:8080/BarcodePayment/transactions/')
response = Net::HTTP::Post.new(url.path)
should work.