I am having trouble making a proxy call. How on earth do you make this happen?
Here is what I have so far:
proxy_addr = '162.243.105.128'
proxy_port = 6170
Net::HTTP::Proxy(proxy_addr, proxy_port).start('www.google.com') {|http| http}
I get:
#<#<Class:0x007f85d8a092d0> www.google.com:80 open=false>
When:
Net::HTTP::Proxy(proxy_addr, proxy_port).start('www.google.com') {|http| http.get('www.google.com')}
I get
#<Net::HTTPNotImplemented 501 Tor is not an HTTP Proxy readbody=true>
How do I make this work!
Tor is a SOCKS proxy, not an HTTP proxy.
I ran into the same issue. I don't think it is possible with plain Net::HTTP. Install http://socksify.rubyforge.org/
require 'socksify/http'
http = Net::HTTP::SOCKSProxy(addr, port)
puts http.get(URI('http://echoip.com'))
Related
I am constantly getting Failed to open TCP connection to :80 (Cannot assign requested address - connect(2) for nil port 80) (Errno::EADDRNOTAVAIL) while using the ruby faraday gem. I don't have alot of experience with ruby on rails.
I have a docker ruby on rails service running on elastic beanstalk that is using puma with ssl. CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"] I have a network load balancer configured with it to forward 443 -> 8443 (i've experimented with both self signed certs and real wild card certs).
ssl_bind '0.0.0.0', '8443', {
key: '/var/app/ssl/something.key',
cert: '/var/app/ssl/something.crt'
}
This configuration works as expected
Puma starting in single mode...
* Version 3.12.0 (ruby 2.5.1-p57), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: staging
* Listening on tcp://0.0.0.0:3000
* Listening on ssl://0.0.0.0:8443?cert=/var/app/ssl/something.crt&key=/var/app/ssl/something.key&verify_mode=none
and I can get the healthz status page with both types of certs. Using httpie and --verify=no for the self signed.
$ http https://backend.something.com/healthz
HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
{
"name": "my-backend-service",
"version": "0.0.1"
}
I have another ruby on rails backend service that makes api requests to this service using faraday. Ive removed some of the request/response code from my actual.
def connection(baseUrl, options = {})
conn = Faraday.new(url: baseUrl) do |c|
# dont really know if this is needed or not
# http.use_ssl? is always false
c.adapter :net_http do |http|
http.verify_mode = OpenSSL::SSL::VERIFY_NONE # if http.use_ssl?
end
end
I don't know if this http.verify_mode is actually working. I cant really find that method anywhere around here on ruby-doc.org
If you try to make a request it will just end up faiing.
conn = connection(https://backend.something.com)
response = conn.post '/foo', params[:foo].to_json
The logs show from starting to parameters and then the http.rb:939 error. I realize the parameters aren't valid here but they aren't my problem.
Started POST "/foo"
Processing by FooController#create as */*
Parameters: {"paramter"=>"something", "paramter"=>"something", "paramter"=>"something"}
ERROR -- : /usr/local/lib/ruby/2.5.0/net/http.rb:939:in `rescue in block in connect': Failed to open TCP connection to :80 (Cannot assign requested address - connect(2) for nil port 80) (Errno::EADDRNOTAVAIL)
If I make the same request from httpie or curl to this service I get the expected results over both http/https.
$ http POST https://backend.something.com parameter="something" parameter="something" parameter="something"
HTTP/1.1 201 Created
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
If you inspect the conn object it kinda still seems like it is sitting on its default value for #url_prefix = http:/. Found in the docs above, but I don't know if I'm looking at the correct thing the correct ruby way. I imagined that Faraday.new(url: baseUrl) would parse the correct schema, which is https.
#<Faraday::Connection:0x0000565351701430 #parallel_manager=nil, #headers={"User-Agent"=>"Faraday v0.15.4"}, #params={}, #options=#<Faraday::RequestOptions (empty)>, #ssl=#<Faraday::SSLOptions (empty)>, #default_parallel_manager=nil, #builder=#<Faraday::RackBuilder:0x0000565351700f58 #handlers=[FaradayMiddleware::EncodeJson, FaradayMiddleware::ParseJson, Faraday::Adapter::NetHttp]>, #url_prefix=#<URI::HTTP http:/>, #manual_proxy=false, #proxy=nil, #temp_proxy=nil>
It seems that the the address you are trying to connect to is invalid. From what I can see in the error:
ERROR -- : /usr/local/lib/ruby/2.5.0/net/http.rb:939:in `rescue in block in connect': Failed to open TCP connection to :80 (Cannot assign requested address - connect(2) for nil port 80) (Errno::EADDRNOTAVAIL)
it is nil. Ensure that the host is properly set.
I have a ruby class in which I am making Net::HTTP.start call inside perform method.This is the code:
class Poller
def self.perform(args)
uri = URI('http://localhost:8081/adhoc/dummy_poll?noAuth=true')
begin
Net::HTTP.start(uri.host, uri.port, :read_timeout=>30) do |http|
request = Net::HTTP::Get.new uri
#response = http.request request
#responsecode = #response.code
end
rescue Exception => e
Rails.logger.error "error mess==>"+e.message()
#responsecode = "408"
end
When I enqueue this within resque from another class using this line:
Resque.enqueue_in_with_queue(:data_workflow_manager,10,Poller,args)
I get this error:
Connection refused - connect(2) for "::1" port 8081.
However, HTTP.start works fine without any errors when the perform method is called in some another class using:
Poller.perform(args)
Please help me understand, why is this happening?
try explicitly connecting to the loop back address. There may be resolution issues with localhost.
uri = URI('http://127.0.0.1:8081/adhoc/dummy_poll?noAuth=true')
It's likely you do not have any server running on port 8081 which is why the connection will be refused.
Check it with lsof -i and look for programs bound to 8081 on a linux machine.
def authenticate(username, password)
require 'net-ldap'
ldap = Net::LDAP.new
ldap.host = 'server.local'
ldap.port = 389
ldap.base = 'cn=users, dc=server, dc=local'
ldap.auth username, password
if ldap.bind
puts "authentication succeeded"
else
puts "authentication failed"
end
The above is the code i use in my method and i am not sure why my attempts fail. I am trying to authenticate user. I could not find where i am going wrong? It puts authentication failed every time. why? please help me.
First up see if the computer you are using can talk to the LDAP server
telnet server.local 389
Obviously you want to be replacing server.local with your actual server details. If you can't log into the server this way then port 389 isn't open and you may need to be running on the SSL port 636. Try the previous command with 636 rather than 389 to see if that is the case.
If you are unable to telnet into the server on either of those ports you've either got a firewall rule blocking you from talking to it, LDAP is configured on a non standard port or something else is seriously wrong.
A working port 636 will probably mean you need to run something like the following.
require "net-ldap"
ldap = Net::LDAP.new(
host: "server.local"
port: 636
encryption: :simple_tls
)
ldap.auth username, password
Failing all of that an error message is going to be pretty useful so try running
if ldap.bind
# Yay!
else
puts ldap.get_operation_result
end
With some results from this maybe we can help you a bit more.
I'm running an app which accepts data from GPS loggers, being this data sent via SMS, HTTP or TCP. I was using the HTTP connection to transmit the data, which was straightforward to receive through a GET request on my Rails app.
For battery saving purposes, we changed the connection to pure TCP and nginx is not accepting these requests for now:
From nginx's access.log:
HTTP:
xx.xx.xx.xxx - - [03/Mar/2013:20:17:45 -0500] "GET /?imei=123456789012345&rmc=$GPRMC,035106.000,A,4145.1451,N,08506.8784,W,1.56,176.49,010213,,*13,3656mV,AUTO HTTP/1.1" 302 247 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22"
TCP:
xx.xx.xx.xxx - - [03/Mar/2013:18:31:22 -0500] "imei=123456789012345&rmc=$GPRMC,233012.000,A,4221.6614,N,07106.1014,W,17.25,218.94,030313,,*21,4102mV,AUTO,310,260,ADB7,13EF,057,310,260,0000,0000,044,310,260,0000,0000,055,310,260,0000,0000,05A,310,260,0000,0000,059,310,260,0000,0000,05E,0,0,0000,0000,000,2" 400 172 "-" "-"
Does anyone know a way to filter these TCP requests on Nginx and process them as HTTP GET requests to be available through a Rails app?
Thanks in advance
Answer:
I was able to achieve this by creating a custom proxy, using em-proxy and faraday:
require 'em-proxy'
require 'faraday'
Proxy.start(:host => "0.0.0.0", :port => 8080, :debug => false) do |conn|
conn.on_data do |data|
http = Faraday.new(:url => 'http://example.com') do |faraday|
faraday.request :url_encoded # form-encode POST params
faraday.response :logger # log requests to STDOUT
faraday.adapter Faraday.default_adapter # make requests with Net::HTTP
end
http.get "/upload?#{data}"
end
end
You should look at websockets. Nginx now has native support for them, but there is an older module for Nginx that does the work as well.
Here's an older question about this with both solutions provided.
Nginx is a HTTP server so it expects clients to speak HTTP. It is not possible to have a client speaking a non-HTTP protocol to nginx in the manner you're describing (ok, maybe you could write a custom module to do it, but that seems silly).
I think what you need to do is write your own proxy server which will convert the raw TCP protocol into HTTP requests. Your GPS loggers would talk to the proxy which would then talk to nginx. This ought to be fairly simple since the TCP payload appears to just be a urlencoded string, so you could just use those directly.
I keep getting the two following errors from my server, I assumed they were just bots looking for potential targets, but does anyone know specifically why I'm getting these? I'm using the SslRequirement plugin to make sure all hits to the login/signup page are redirected to SSL, so all of these weird https requests to root should just be redirected to regular http.
A ActionController::UnknownHttpMethod occurred in application#index:
quit, accepted HTTP methods are get, head, put, post, delete, and options
/usr/local/lib/ruby/gems/1.9.1/gems/actionpack-2.3.4/lib/action_controller/request.rb:35:in `request_method'
PATH_INFO : /
REMOTE_ADDR : 99.19.208.249
REMOTE_PORT : 6376
REQUEST_METHOD : CONNECT
REQUEST_URI : /
SERVER_PORT : 443
SERVER_PROTOCOL : HTTP/1.0
SERVER_SOFTWARE : Apache
A ActionController::UnknownHttpMethod occurred in application#index:
CONNECT, accepted HTTP methods are get, head, put, post, delete, and options
/usr/local/lib/ruby/gems/1.9.1/gems/actionpack-2.3.4/lib/action_controller/request.rb:35:in `request_method'
HTTPS : on
HTTP_X_FORWARDED_PROTO : https
PATH_INFO : /
REMOTE_ADDR : 91.209.196.76
REMOTE_PORT : 50751
REQUEST_METHOD : quit
REQUEST_URI : /
SERVER_PORT : 443
SERVER_PROTOCOL : HTTP/0.9
The CONNECT command is used by HTTP proxy servers to indicate that the client wants to just connect a socket directly to another server; this is usually used for tunneling TLS over an HTTP proxy, but could be used for tunneling almost any protocol.
QUIT is not an HTTP command, but it is an SMTP command. It is possible that you are getting these commands from a bot that is trying to find open relays for sending spam; it's trying to figure out if you have an open SMTP relay, or an open HTTP proxy that allows the CONNECT command which could also be used to tunnel SMTP traffic.
So, likely you're just being hit by a spam botnet trying to find open relays. My advice would be to drop such requests as early as possible, and not worry about them.
CONNECT: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.9
QUIT usually means 'close the connection'.
# Avoid annoying ActionController::UnknownHttpMethod exceptions like:
#
# ActionController::UnknownHttpMethod) "CONNECT, accepted HTTP methods are get, head, put, post, delete, and options"
#
# Install this file in app/metal and these requests will receive a 405
# "Method Not Allowed" status and will be logged under `info'.
class IgnoreUnknownHttpMethod
def self.call(env)
[
if ActionController::Request::HTTP_METHODS.include?(env["REQUEST_METHOD"].downcase)
404 # Not Found
else
Rails.logger.info("Ignoring unknown HTTP method; #{env.inspect}")
405 # Method Not Allowed
end, {"Content-Type" => "text/plain"}, []]
end
end
Credit https://gist.github.com/remvee/600569
I just noticed a few attempt to connect to my servers as follow, we see that it ends with a QUIT...
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET / HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET /robots.txt HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET /sitemap.xml HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:58 -0800] "quit" 405 5461 "-" "-"
As a side note, my server returns 444 which is not a legal HTTP code. It means NO RESPONSE and I do that because their "Agent String" is empty.
Looking at the IP address, I found a search engine that searches for things on the internet. Not a hacker trying to break in per se. (There intend is not evil, it seems.) I suppose that some of the applications they are checking for must understand the QUIT. So as a robot, it makes sense that you would want to try what would otherwise looks like weird commands such as CONNNECT and QUIT. There is no real limit to the commands an application can support, even if those commands are not official HTTP commands.
If your application does not understand these methods, then nothing will happen so you should not bother too much about these.
If your application receives those methods, then you may want to write a little something at the start to get the method and compare to GET and POST (and whatever else you support, like DELETE and PUT) and if it matches none of these, then reply with a 405 error code: "Method Not Allowed".
https://www.rfc-editor.org/rfc/rfc7231#section-6.5.5
If you cannot change your application and you are not sure whether it could react to a CONNECT / QUIT hit, then you could look into using a setup such as mod_security for Apache2.
as per jturkel at https://gist.github.com/remvee/600569.
In Rails 3.2 for non http actions I added the following at the end of config/application.rb and solved the problem for quit.
# silence ActionController::UnknownHttpMethod exceptions
ActionDispatch::ExceptionWrapper.rescue_responses.merge!( 'ActionController::UnknownHttpMethod' => :method_not_allowed )