I'm attempting to create a rake tasks that visits a series of URLs, logs the status code, and then prints them to console. My code is as follows:
require 'csv'
require 'net/http'
require "open-uri"
namespace :routes do
desc "check status codes"
task check_301s: :environment do
# open_csv
# edit_urls
generate_routes
# visit_urls
# log_status_codes
visit_and_log_routes
# give_results
end
def generate_routes
csv = CSV.open('lib/better.csv', headers: true)
#urls = []
csv.each do |row|
#urls << row['url'].gsub('foo', 'localhost:3000')
end
end
def visit_and_log_routes
responses = []
#urls.each do |url|
http = Net::HTTP.new(url, 3000)
response = http.request_get('/')
responses << response
end
puts responses
end
end
I'm receiving the following error code when running this from terminal:
SocketError: Failed to open TCP connection to localhost:3000/children-centre/london/greenwich/eltham-childrens-centre/join:3000 (getaddrinfo: nodename nor servname provided, or not known)
I'm not sure if there is an easier way to visit URLs and log their status code, if so please let me know how to do it, and if not then how I can correct this error message. I'm assuming I'm simply leaving out the relevant options but am not sure how / where to add them.
For anyone making the same impressive error I was... the route needs http:// at the start instead of just localhost:3000
Related
When debugging failing integration tests, I keep running into the same problem where the exceptions raised in my code are suppressed and not shown in the testing output.
For example, for the following controller and test:
class RegistrationController::ApplicationController
def create
# some code that raises an exception
end
end
class RegistrationFlowTest < ActionDispatch::IntegrationTest
test 'user registers successfully' do
post sign_up_path, params: { username: 'arnold', password: '123' }
assert_response :success
end
end
The output is something like
Minitest::Assertion: Expected response to be a <2XX: success>, but was a <500: Internal Server Error>
Is there a way to see the exact raised exception? Instead of just the difference of HTTP response code?
Thanks!
Simon
My recommended approach to this issue would be to actually parse the response provided by Rails (at least by default in test and development environments) which includes the stacktrace for the error and handle that in the case that your test fails. This has the advantage that it won't output the stacktrace when errors are raised that don't result in failing tests (e.g. scenarios where you are intentionally testing how failures are handled).
This little module that I've crafted will allow you to call assert_response_with_errors to assert the response to a call but output the exception message and stack trace in a readable format when the response is not what you expected.
module ActionDispatch
module Assertions
module CustomResponseAssertions
# Use this method when you want to assert a response body but also print the exception
# and full stack trace to the test console.
# Useful when you are getting errors in integration tests but don't know what they are.
#
# Example:
# user_session.post create_gene_path, params: {...}
# user_session.assert_response_with_errors :created
def assert_response_with_errors(type, message = nil)
assert_response(type, message)
rescue Minitest::Assertion => e
message = e.message
message += "\nException message: #{#response.parsed_body[:exception]}"
stack_trace = #response.parsed_body[:traces][:'Application Trace'].map { |line| line[:trace] }.join "\n"
message += "\nException stack trace start"
message += "\n#{stack_trace}"
message += "\nException stack trace end"
raise Minitest::Assertion, message
end
end
end
end
To use this, you need to include it into ActionDispatch::Assertions before Rails has loaded its stack in your test_helper.rb. So just prepend the include into your test_helper.rb, like this:
ActionDispatch::Assertions.include ActionDispatch::Assertions::CustomResponseAssertions
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
...
This will happen because Rails controllers by default handle exceptions and raise the 500 status, making the exceptions invisible to the test suite (which is very helpful if errors are raised in unit tests of a model). Options for disabling this in your test suite, or alternative workarounds, are discussed here.
The key lines of code from that link, which should be added to test/integration/integration_test_helper.rb:
ActionController::Base.class_eval do
def perform_action
perform_action_without_rescue
end
end
Dispatcher.class_eval do
def self.failsafe_response(output, status, exception = nil)
raise exception
end
end
EDIT: I've noticed that that link is quite old now. I'm really familiar with Rack, so whilst the first block looks ok to me, I'm not sure if the second will still be current. You might need to have a look at the relevant current Rails guide if it needs bringing up to date.
I am trying to log the output of a rake task to a new logger. Ideally the solution would work in development and production.
Here is my task:
task :clients, [:field] => [:setup_logger] do |t, args|
clients = Client.all
problems = []
group = clients.group_by { |client| client[args[:field]] }
unique_ids = group.keys
unique_ids.each do |unique_id|
problems << [unique_id, group[unique_id].length] if group[unique_id].length != 1
end
if !problems.blank?
logger = Logger.new("db_issues.log")
logger.error "look at me"
logger.close
end
p problems
end
When I run this, even though problems is not blank, no new log file is created. How am I supposed to accomplish this?
The problem is the logger.close, look after boot you server for a message like this:
log writing failed. closed stream
try removing logger.close from your code and restarting your server.
I'm building an app that supports realtime bid using Faye-websocket. But I got this 200 error and I have no idea what problem it is.
Error:
WebSocket connection to 'ws://localhost/auctions/3' failed: Error during WebSocket handshake: Unexpected response code: 200
SocketConnection.rb
require 'faye/websocket'
require 'websocket/extensions'
require 'thread'
require 'json'
class SocketConnection
KEEPALIVE_TIME = 15 # in seconds
def initialize app
#app = app
end
def call env
#env = env
if Faye::WebSocket.websocket?(env)
socket = Faye::WebSocket.new env
socket.ping 'Mic check, one, two' do
p [:ping, socket.object_id, socket.url]
end
socket.on :open do |event|
p [:open, socket.object_id, socket.url]
p [:open, socket.url, socket.version, socket.protocol]
end
socket.rack_response
else
#app.call(env)
end
end
end
I firgured out the problem. It requires a server to support socket connection. In my case, I use thin server. All errors are fixed
I am looking for an example of Faraday Middleware that handles http (status code) errors on requests and additionally network timeouts.
After reading the docs for Faraday and it's middleware it's my understanding that this is one of middleware's use cases… I just have no idea what an implementation is supposed to look like.
Thanks
Faraday has an error handling middleware in by default:
faraday.use Faraday::Response::RaiseError
For example:
require 'faraday'
conn = Faraday.new('https://github.com/') do |c|
c.use Faraday::Response::RaiseError
c.use Faraday::Adapter::NetHttp
end
response = conn.get '/cant-find-me'
#=> gems/faraday-0.8.8/lib/faraday/response/raise_error.rb:6:in `on_complete': the server responded with status 404 (Faraday::Error::ResourceNotFound)
If you want to write your own middleware to handle HTTP status code responses, heres a basic example:
require 'faraday'
class CustomErrors < Faraday::Response::Middleware
def on_complete(env)
case env[:status]
when 404
raise RuntimeError, 'Custom 404 response'
end
end
end
conn = Faraday.new('https://github.com/') do |c|
c.use CustomErrors
c.use Faraday::Adapter::NetHttp
end
response = conn.get '/cant-find-me' #=> `on_complete': Custom 404 response (RuntimeError)
For your code, you'll probably want to put it in a separate file, require it, modularise it etc.
If you want to see a good live example, the new Instagram gem has a pretty good setup to raise custom errors: GitHub link
I have a rake task to load car's image from the table websites using paperclip. The image stored in database as a remote link.
Here is my code and i'm using ruby 1.8.7, rails 2.3.8 and DB mysql.
namespace :db do
task :load_photo => :environment do
require 'rubygems'
require 'open-uri'
require 'net/http'
require 'paperclip'
begin
images =Website.find(:all,:conditions=>["image_url is not null"])
images.each do |photo|
url = URI.parse(photo.image_url)
Net::HTTP.start(url.host, url.port) do |http|
if http.head(url.request_uri).code == "200"
Car.update_attribute(:photo,open(url))
end
end
end
rescue Exception => e
end
end
end
Run above rake task by db:load_photo. In my table (Website) has 60,000 rows. Rake task running upto 10000 rows only and execution terminated with an error message "execution expired".
Can any one help me to figure this out?
Thanks in advance.
You may find it more performant to run it in batches, active record has a find_in_batches method which stops loading all the records into memory at one time.
http://ryandaigle.com/articles/2009/2/23/what-s-new-in-edge-rails-batched-find
You could change your code to look like:
namespace :db do
task :load_photo => :environment do
require 'rubygems'
require 'open-uri'
require 'net/http'
require 'paperclip'
Website.find_in_batches(:conditions=>["image_url is not null"]) do |websites|
websites.each do |website|
begin
url = URI.parse(website.image_url)
Net::HTTP.start(url.host, url.port) do |http|
if http.head(url.request_uri).code == "200"
Car.update_attribute(:photo,open(url))
end
end
rescue Exception => e
end
end
end
end
end
I can only guess, but it looks like you're making a little DoS attack to the server you're pulling images from.
You can try to play with a little delay between sequential requests (like "sleep 1").
Also, if your "execution expired" is a Timeout::Error exception, then you can't catch it with
rescue Exception => e
because Timeout::Error is not a subclass of StandardError, it is a subclass of the Interrupt class. You have to catch it explicitly, like so:
rescue Timeout::Error => e