Does anyone know of a plugin / gem that will log any HTTP requests your rails app may be making when responding to a request? For example if you are using HTTParty to hit an API, how can you see what outbound requests are coming out of your rails app?
You have to tell the outbound HTTP client to use a proxy.
For HTTParty it's fairly simple (from the docs),
class Twitter
include HTTParty
http_proxy 'http://myProxy', 1080
If you're looking for a proxy to set up, personally I like Paros proxy (Java so cross platform and does SSL).
Try also http_logger gem:
require 'http_logger'
Net::HTTP.logger = Logger.new(...) # defaults to Rails.logger if Rails is defined
Net::HTTP.colorize = true # Default: true
This will log all requests that goes through Net::HTTP library.
https://github.com/railsware/http_logger
If you're doing development on your own machine, Charles Proxy is a good option.
In production, you'd probably be better off creating your own logger.debug() messages.
The only way I got this to work was to specify only the IP as the first parameter to the http_proxy call:
http_proxy '10.2.2.1', 8888
The example above, with the http:// prefix, did not work, I got a SocketError: getaddrinfo: nodename nor servname provided
Try my httplog gem, you can customize it to log requests, responses, headers etc.
Related
I am parsing a web page using nokogiri gem like the code below:
require 'open-uri'
url = 'https://www.google.com/' # example url
Nokogiri::HTML(URI.open(url))
# ... some other codes
I would like to use proxy with open-uri. I have looked through the documentation, but the example code is without using proxy.
How can I use proxy when using URI.open? I would appreciate any help with syntax explanation or some example code. Thank you!
URI.open is designed to be an easy to use wrapper for Net::Http but I think you have at least a couple of options:
URI.open (and URI generically) supports the use of an HTTP_PROXY environment variable, like: Automatically adding proxy to all HTTP connections in ruby
Net:Http has options that would allow you to fine tune this and use different types of authentication with the proxy, and it's not particularly hard see: How to set a proxy in rubys net/http?
I have a Rails application that has changed from user authentication via devise to accessing a remote API for authentication. For the test suite, this requires that I mock the https request for authentication and return a valid response. We are using Cucumber and Capybara for testing and I am attempting to use the webmock Gem to mock the login response.
The login is initiated by a button click on a form (Capybara action 'click_button').
Unfortunately, although webmock appears to be installed correctly (I am able to make a Net::HTTP POST request and this is recognized by Webmock), the POST to the remote authorization facility is not being captured by webmock. I know that the form POST is making its way to the controller, because the stacktrace shows that the POST is being executed in the controller as it should and the error message is "Errno::ECONNREFUSED at ... Failed to open TCP connection".
I have tried:
WebMock.stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
and
WebMock::stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
and
stub_request(:any, '127.0.0.1').to_return(body: {STATUSCODE: 1}.to_json)
I have tried putting the stub_request call in the devise_steps.rb file, just before the "click_button" command, as well as in the features/support/env.rb and features/support/webmock.rb file.
I assume that what I am doing is so common that it has to be possible, but I have found nothing that indicates why the stub_request is not successful.
So the domain of the remote API for authentication is localhost? So it would be running on the same server with a different port? Then you have to mock the address with the port.
For instance your Rails app is running on port 80 and your auth API is running on 8080 then you have to do this.
stub_request(:any, '127.0.0.1:8080').to_return(body: {STATUSCODE: 1}.to_json)
I think Webmock should support different ports but I'm not 100% sure. Also have you set WebMock.disable_net_connect!(allow_localhost: true)?
https://github.com/bblimke/webmock#external-requests-can-be-disabled-while-allowing-localhost
Edit:
If this is not working, I suggest you make the API URL configurable and e.g. set it in tests to auth-api.com and then mock this.
The root cause of the problem was that the path on the stub_request was not complete. I had misread the webmock documentation....
I use rack-reverse-proxy to setup my Blogger.com to a subdomain of my Ruby on Rails app: pulpoludo.com/blog
It's work, but I have an issue with the link of Blogger which returns to blog.pulpoludo.com (where my Blogger blog is host).
I would like to rewrite this link. But I don't know-how. Can you help me?
(I have found someone who does this in PHP: https://matt-stannard.blogspot.com/2013/02/blogger-in-subdirectory-of-my-domain.html
But I would like to do the same thing with Rails and a gem maybe)
Indeed, you cannot use rack-reverse-proxy because it does not allow you to change the response (you need to rewrite the page you retrieve using a regular expression replacement, as in the example you link to.
Also, you should probably avoid using rack-reverse-proxy in production, as it will keep your ruby processes busy waiting for the backend responses, that might fail or be slow. And:
It is not meant for production systems
You should instead proxy from your front HTTP acceptor (nginx or other). For nginx you can see a very thorough response, using a combination of proxy_pass and sub_filter, at https://stackoverflow.com/a/32543398/384417.
edit: If it's not possible to use nginx or another reverse proxy, you can still do it in ruby.
rack-reverse-proxy supports transformers, you can build one yourself, and register it so it's run on the response. This (closed) issue will help, it is exactly what you need: https://github.com/waterlink/rack-reverse-proxy/issues/65. The caveat (as always when changing responses) is that you have to update the Content-Length response header to match the updated size of the body.
I am running a rails app on elastic beanstalk, my problem is that I cannot use the load balancer health check, as it checks via http, not https. I have config.force_ssl set to true, and don't really want to change it, since it means I have 100% ssl coverage, HSTS, and secure cookies.
I have setup a controller to check at /health and can access this via curl.
There seems to be a contradiction between what is shown in elastic beanstalk (EC2 Instance Health Check), where I can only change the path and some timings, and the health check in the load balancer in EC2, where I can change the ping protocol, port and path to what I want, but this seems to have no effect other than causing a 503 error.
Ping Protocol: https
Ping Port: 443
Ping Path: /health
The load balancer has ports 80 and 443 open but redirects from 80 to 443 due to force_ssl.
Does anyone know of the correct settings to get around this, as I won't be able to scale the application without it?
Thank you
Eamon
The way to do this is to add this to your config, like this.
config.force_ssl
config.ssl_options = { redirect: { exclude: -> request { request.path =~ /health/ } } }
This is as per the rails docs and is valid for Rails 5:
http://api.rubyonrails.org/classes/ActionDispatch/SSL.html
Small hint for the accepted answer:
In case you have updated from Rails 4 to Rails 5 and have therefore a new_frameworks_default.rb in your initializers directory. You need to remove (or out-comment) the following line:
Rails.application.config.ssl_options = { hsts: { subdomains: true } }
Otherwise ssl_options is already set.
The accepted answer kept not working for me, but after hours of debugging I found this gem:
https://github.com/lserman/aws-healthcheck
It returns 200 on /healthcheck which works like a charm.
If, like me you are not using rails 5, the other way you can do this is to disable force SSL, and use NGINX to force all traffic onto https... You can use an ebextension file to do this. The exact version you need comes direct from AWS:
https://github.com/awslabs/elastic-beanstalk-docs/blob/master/.ebextensions/aws_provided/security%20configuration/https-redirect-ruby-puma.config
If you need more information on ebextensions, you can find them in the docs http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html
EDIT: The original link no longer works...
Try this:
https://github.com/awsdocs/elastic-beanstalk-samples/blob/master/configuration-files/aws-provided/security-configuration/https-redirect/ruby-puma/https-redirect-ruby-puma.config
Full disclaimer; I'm not a strong Ruby dev, but I am learning quickly :)
I've set up a simple Ruby script on a Heroku dyno that listens for calls from our donation platform.
When a donation is made, it hits a webhook endpoint within my app, which then sends a donation receipt via Mandrill (which works fine), and updates/inserts a record in a Mailchimp list, via the 'upsert' method of the wonderful Gibbon gem.
That all works fine; except when the Heroku box happens to come up on an IP address that has done something bad in the past, and Mailchimp's API drops with a 403 (Forbidden) error.
I've had this confirmed by the Mailchimp API team; they suggest using something like QuotaGuard Static to tunnel the API requests to Mailchimp through, removing the issue of API calls from inconsistent (and sometimes untrusted) IP addresses.
I'd love some advice on how to make this happen. I can see that Gibbon uses Faraday to handle HTTP requests, but I'm not an advanced enough Ruby dev to fork the code and add in HTTP proxy functionality.
If there's a way to globally force the Faraday calls to use a HTTP proxy (ie QuotaGuard Static), that's what I'm looking for. A config setting for Faraday, for example.
Or perhaps there's a tweak I can make to my Procfile:
web: bundle exec ruby webhooks.rb -p $PORT
...that will force the outbound traffic to go via the QuotaGuard Static proxy. I know Proximo has this functionality, but it also blocks inbound access to the app, which doesn't work for this app.
Appreciate any ideas the community can offer. Thanks!
Gibbon Author here. You can simply set the proxy value to the proxy URL in Gibbon 2.2.0 and later.
From the Faraday documentation (here) the Connectionclass uses the proxy specified in the http_proxy environment variable. I have never tried it, but looking at the source code it should work.
I wanted to provide a bit more information, since the two answers pointed me on the right track but still required me to do some digging. I solved this issue by first adding the QuotaGuard Static add-on in Heroku (free for up to 250 uses per month) and then initializing Gibbon like so:
g = Gibbon::Request.new
g.proxy = ENV["QUOTAGUARDSTATIC_URL"]
And here is the relevant section from the Gibbon docs: https://github.com/amro/gibbon#other