How does one create a POST request using TCPSocket in Ruby? Is there a special format to making a post? I have the following but I get a parse error (it's for a rails server):
require 'socket'
s = TCPSocket.open("localhost", 3000)
s.puts("POST /<controller>/<action> HTTP/1.1")
s.puts("Host: localhost:3000")
s.puts("Content-Type: application/x-www-form-urlencoded")
s.puts("Content-Length: 103\r\n\r\n")
The Host: field should not include the port number.
Found this article that may be of some use to you. I especially like Eric Hodel's comment about how to do it with Net::HTTP. I know you specified that you wanted to do TCPSocket.send (presumably because you're working on something slightly more interesting than just sending POSTs), but if you aren't doing something more complicated you may be able to use Net::HTTP and rejoice at how easy it is.
Related
I have a very simple number crunching Ruby function that I want to make available via a web API. The API is essentially a single endpoint, e.g. http://example.com/crunch/<number> and it returns JSON output.
I can obviously install Rails and implement this quickly. I require no more help from a 'framework' other than to handle HTTP for me. No ORM, MVC and other frills.
On the far end, I can write some Ruby code to listen on a port and accept GET request and parse HTTP headers etc. etc. I don't want to re-invent that wheel either.
What can I use to expose a minimal API to the web using something with the least footprint/dependencies. I read about Sinatra, Ramaze, etc., but I believe there can be a way to do something even simpler. Can I just hack some code on top of Rack to do what I am trying to do?
Or in other words, what will be the simplest Ruby equivalent of the following code in nodejs:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
var ans = crunch(number);
res.end(ans);
}).listen(1337, "127.0.0.1");
console.log('Server running at http://127.0.0.1:1337/');
You seem like you want to use Rack directly. "Rack from the Beginning" is a decent tutorial that should get you started.
It'll probably look something like this:
class CrunchApp
def self.crunch(crunchable)
# top-secret crunching
end
def self.call(env)
crunchy_stuff = input(env)
[200, {}, crunch(crunchy_stuff)]
end
private
def self.input(env)
request = Rack::Request.new(env)
request.params['my_input']
end
end
Rack::Server.start app: CrunchApp
But I must say, using that instead of something like Sinatra seems silly unless this is just a fun project to play with things. See their 'Hello World':
require 'sinatra'
get '/hi' do
"Hello World!"
end
Ruby-Grape is a good option for your use case. It has a minimal implementation over Rack that allow the creation of simple REST-API endpoints.
Cuba is another good option with a thin layer over Rack itself.sample post
If you are familiar with Rails you can use the Rails API gem which is very well documented with minor overhead. Remember also that Rails-API will be part of Rails 5.
Last, but not last you can implement it on Rack directly.
I have the website on Ruby on Rails 3.2.11 and Ruby 1.9.3.
What can cause the following error:
(JSON::ParserError) "{N}: unexpected token at 'alihack<%eval request(\"alihack.com\")%>
I have several errors like this in the logs. All of them tries to eval request(\"alihack.com\").
Part of the log file:
"REMOTE_ADDR" => "10.123.66.198",
"REQUEST_METHOD" => "PUT",
"REQUEST_PATH" => "/ali.txt",
"PATH_INFO" => "/ali.txt",
"REQUEST_URI" => "/ali.txt",
"SERVER_PROTOCOL" => "HTTP/1.1",
"HTTP_VERSION" => "HTTP/1.1",
"HTTP_X_REQUEST_START" => "1407690958116",
"HTTP_X_REQUEST_ID" => "47392d63-f113-48ba-bdd4-74492ebe64f6",
"HTTP_X_FORWARDED_PROTO" => "https",
"HTTP_X_FORWARDED_PORT" => "443",
"HTTP_X_FORWARDED_FOR" => "23.27.103.106, 199.27.133.183".
199.27.133.183 - is CLoudFlare IP.
"REMOTE_ADDR" => "10.93.15.235" and "10.123.66.198" and others, I think, are fake IPs of proxy.
Here's a link guy has the same issue with his web site from the same ip address(23.27.103.106).
To sum up, the common ip from all errors is 23.27.103.106 and they try to run the script using ruby's eval.
So my questions are:
What type of vulnerability they try to find?
What to do? Block the ip?
Thank you in advance.
Why it happens?
It seems like an attempt to at least test for, or exploit, a remote code execution vulnerability. Potentially a generic one (targeting a platform other than Rails), or one that existed in earlier versions.
The actual error however stems from the fact that the request is an HTTP PUT with application/json headers, but the body isn't a valid json.
To reproduce this on your dev environment:
curl -D - -X PUT --data "not json" -H "Content-Type: application/json" http://localhost:3000
More details
Rails action_dispatch tries to parse any json requests by passing the body to be decoded
# lib/action_dispatch/middleware/params_parser.rb
def parse_formatted_parameters(env)
...
strategy = #parsers[mime_type]
...
case strategy
when Proc
...
when :json
data = ActiveSupport::JSON.decode(request.body)
...
In this case, it's not a valid JSON, and the error is raised, causing the server to report a 500.
Possible solutions
I'm not entirely sure what's the best strategy to deal with this. There are several possibilities:
you can block the IP address using iptables
filter (PUT or all) requests to /ali.txt within your nginx or apache configs.
use a tool like the rack-attack gem and apply the filter there. (see this rack-attack issue )
use the request_exception_handler gem to catch the error and handle it from within Rails (See this SO answer and this github issue)
block PUT requests within Rails' routes.rb to all urls but those that are explicitly allowed. It looks like that in this case, the error is raised even before it reaches Rails' routes - so this might not be possible.
Use the rack-robustness middleware and catch the json parse error with something like this configuration in config/application.rb
Write your own middleware. Something along the lines of the stuff on this post
I'm currently leaning towards options #3, #4 or #6. All of which might come in handy for other types of bots/scanners or other invalid requests that might pop-up in future...
Happy to hear what people think about the various alternative solutions
I saw some weird log entries on my own site [which doesn't use Ruby] and Google took me to this thread. The IP on my entries was different. [120.37.236.161]
After poking around a bit more, here is my mostly speculation/educated guess:
First, in my own logs I saw a reference to http://api.alihack.com/info.txt - checked this link out; looked like an attempt at a PHP injection.
There's also a "register.php" page there - submitting takes you to an "invite.php" page.
Further examination of this domain took me to http://www.alihack.com/2014/07/10/168.aspx (page is in Chinese but Google Translate helped me out here)
I expect this "Black Spider" tool has been modified by script kiddies and is being used as a carpet bomber to attempt to find any sites which are "vulnerable."
It might be prudent to just add an automatic denial of any attempt including the "alihack" substring to your configuration.
I had a similar issue show up in my Rollbar logs, a PUT request to /ali.txt
Best just to block that IP, I only saw one request on my end with this error. The request I received came from France -> http://whois.domaintools.com/37.187.74.201
If you use nginx, add this to your nginx conf file;
deny 23.27.103.106/32;
deny 199.27.133.183/32;
For Rails-3 there is a special workaround-gem: https://github.com/infopark/robust_params_parser
I'm using FLEX 3 to make XML request to Rails 3. Since FLEX 3 only offers POST and GET, I have to use the "?_method=PUT" hack to maintain proper RESTfulness:
http://127.0.0.1:3000/locator/locator_users/1.xml?_method=PUT
On the server side its showing up as a POST and I'm getting an ActionController::RoutingError (No Route Matches).
I did a rake routes and the route is there, properly namespaced and all.
This worked fine with Rails 2, so I have reason to believe it must be Rails 3 that changed. After doing some searching, people seemed to have indicated that it should still work. But, it's not for me. Can anyone confirm or deny Rails 3 compatibility?
UPDATE
OK, after some more tinkering, I think this is actually a problem of Flash Player 10. Flash PLayer 9 seems to work fine with "_method=" hack, 10 does not. See this new post I wrote (Flash Player 9 vs Flash Player 10 with FLEX 3, ?_method=PUT/DELETE not working?).
Partly this is due to Rack::MethodOverride's behavior. It does not check the query params for _method, so a call to http://127.0.0.1:3000/locator/locator_users/1.xml?_method=PUT
will not get overridden properly due to that.
I wrote a piece of Rack middleware that replaces it to fix this particular issue.
All you have to do is
add it to the Gemfile
gem 'rack-methodoverride-with-params'
swap Rack::MethodOverride out in config/environment.rb
config.middleware.swap Rack::MethodOverride, Rack::MethodOverrideWithParams
If you can add _method=PUT in your request body, then no need to swap out the rack middleware.
If you cannot do that, then I've come across another (lower-impact) solution, which is to simply define a custom route that accomplishes what you're looking for, for example:
# config/routes.rb
post '/locator/locator_users/:id', to: 'locator_users#update', constraints: {_method: 'POST'} # allow http method override
Granted, you would have to add this route for every resource you need HTTP method override on, but that might be a good thing if you want to limit your exposure to potential weirdness since this breaks HTTP semantics.
EDIT: You can do the same thing with GET requests if you need to, just swap out post for get (this can be useful if you need to support REST over JSONP).
You might have to rewrite how you're making your request to the server. I'm using Rails 3, Flex 4, and Flash 10 together (but with an app developed in Flex 3) and use _method as a parameter in my HTTPService object (leaving the content-type as the default application/x-www-form-url).
HTTPService only supports GET and POST requests. If you use set useProxy property to true on the HTTPService object, you can use HEAD, OPTIONS, TRACE, and DELETE but only if you are also using a server-based proxy service. If this doesn't work, then you may want to try URLLoader or URLRequest or implementing your own custom solution instead.
Hey all, I've got a client that is integrating a Rails 1.2.6 site with another site that exposes services RESTfully. Upgrading to Rails 2.x is not an option at this time. Does anyone have recommendations for methods other than direct Net::HTTP calls to communicate with the REST service? Techniques or Gem recommendations are welcome, but most of the gems I've seen seem to have a dependency on ActiveSupport 2.x, which I understand to be incompatible with Rails 1.x.
Thanks in advance for any input you can provide.
Try HTTParty. It's very light on the dependencies, and makes it braindead-easy to add consumption of JSON or XML resources to an application.
Thanks Chris Heald for your response. I did end up using Net::HTTP because it was more straightforward than I thought it was in the end. HTTParty looks like it could make this easier still, but for the benefit of future people with this problem, here's what I did.
# Assume #user_name and #password were previously declared to be the
# appropriate basic auth values and that the connection is open as #connection
def put(path, body, header={})
request = Net::HTTP::Put.new(path, header.merge({'Accept' => 'application/xml,application/json', 'Content-type'=>'application/json'}))
request.basic_auth(#user_name, #password)
#connection.request(request, body).body
end
def post(path, body, header={})
request = Net::HTTP::Post.new(path, header.merge({'Accept' => 'application/xml,application/json', 'Content-type'=>'application/json'}))
request.basic_auth(#user_name, #password)
#connection.request(request, body).body
end
def get(path, header={})
request = Net::HTTP::Get.new(path)
request.basic_auth(#user_name, #password)
#connection.request(request).body
end
I then called JSON::parse() on the output of these methods and got a hash representing the JSON that I could use as I saw fit.
I am trying to debug an ActiveResource call that is not working.
What's the best way to view the HTTP response to the request ActiveResource is making?
Monkey patch the connection to enable Net::HTTP debug mode. See https://gist.github.com/591601 - I wrote it to solve precisely this problem. Adding this gist to your rails app will give you Net::HTTP.enable_debug! and Net::HTTP.disable_debug! that you can use to print debug info.
Net::HTTP debug mode is insecure and shouldn't be used in production, but is extremely informative for debugging.
Add a new file to config/initializers/ called 'debug_connection.rb' with the following content:
class ActiveResource::Connection
# Creates new Net::HTTP instance for communication with
# remote service and resources.
def http
http = Net::HTTP.new(#site.host, #site.port)
http.use_ssl = #site.is_a?(URI::HTTPS)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE if http.use_ssl
http.read_timeout = #timeout if #timeout
# Here's the addition that allows you to see the output
http.set_debug_output $stderr
return http
end
end
This will print the whole network traffic to $stderr.
It's easy. Just look at the response that comes back. :)
Two options:
You have the source file on your computer. Edit it. Put a puts response.inspect at the appropriate place. Remember to remove it.
Ruby has open classes. Find the right method and redefine it to do exactly what you want, or use aliases and call chaining to do this. There's probably a method that returns the response -- grab it, print it, and then return it.
Here's a silly example of the latter option.
# Somewhere buried in ActiveResource:
class Network
def get
return get_request
end
def get_request
"I'm a request!"
end
end
# Somewhere in your source files:
class Network
def print_request
request = old_get_request
puts request
request
end
alias :old_get_request :get_request
alias :get_request :print_request
end
Imagine the first class definition is in the ActiveRecord source files. The second class definition is in your application somewhere.
$ irb -r openclasses.rb
>> Network.new.get
I'm a request!
=> "I'm a request!"
You can see that it prints it and then returns it. Neat, huh?
(And although my simple example doesn't use it since it isn't using Rails, check out alias_method_chain to combine your alias calls.)
I like Wireshark because you can start it listening on the web browser client end (usually your development machine) and then do a page request. Then you can find the HTTP packets, right click and "Follow Conversation" to see the HTTP with headers going back and forth.
This only works if you also control the server:
Follow the server log and fish out the URL that was called:
Completed in 0.26889 (3 reqs/sec) | Rendering: 0.00036 (0%) | DB: 0.02424 (9%) | 200 OK [http://localhost/notifications/summary.xml?person_id=25738]
and then open that in Firefox. If the server is truely RESTful (ie. stateless) you will get the same response as ARes did.
Or my method of getting into things when I don't know the exact internals is literally just to throw in a "debugger" statement, start up the server using "script/server --debugger" and then step through the code until I'm at the place I want, then start some inspecting right there in IRB.....that might help (hey Luke btw)
Maybe the best way is to use a traffic sniffer.
(Which would totally work...except in my case the traffic I want to see is encrypted. D'oh!)
I'd use TCPFlow here to watch the traffic going over the wire, rather than patching my app to output it.
the firefox plugin live http headers (http://livehttpheaders.mozdev.org/) is great for this. Or you can use a website tool like http://www.httpviewer.net/