I'm using the following function in Ruby on Rails:
def isGoogleEmailAddress?(email_domain)
Resolv::DNS.open({:nameserver=>["8.8.8.8"]}) do |r|
mx = r.getresources(email_domain,Resolv::DNS::Resource::IN::MX)
if mx.any? {|server| server.exchange.to_s.downcase.include? "google"} then
return true
end
return false
end
end
Is there a way to handle the issue where Resolv fails, timeouts, errors etc?
Look through the documentation for the Resolv class and add exception handlers for the various errors/exceptions the class can raise.
They're easy to pick out. Look for classes ending in error and timeout.
Related
I have a Rails 5 application using raven-ruby to send exceptions to Sentry which then sends alerts to our Slack.
Raven.configure do |config|
config.dsn = ENV['SENTRY_DSN']
config.environments = %w[ production development ]
config.excluded_exceptions += []
config.async = lambda { |event|
SentryWorker.perform_async(event.to_hash)
}
end
class SentryWorker < ApplicationWorker
sidekiq_options queue: :default
def perform(event)
Raven.send_event(event)
end
end
It's normal for our Sidekiq jobs to throw exceptions and be retried. These are mostly intermittent API errors and timeouts which clear up on their own in a few minutes. Sentry is dutifully sending these false alarms to our Slack.
I've already added the retry_count to the jobs. How can I prevent Sentry from sending exceptions with a retry_count < N to Slack while still alerting for other exceptions? An example that should not be alerted will have extra context like this:
sidekiq: {
context: Job raised exception,
job: {
args: [{...}],
class: SomeWorker,
created_at: 1540590745.3296254,
enqueued_at: 1540607026.4979043,
error_class: HTTP::TimeoutError,
error_message: Timed out after using the allocated 13 seconds,
failed_at: 1540590758.4266324,
jid: b4c7a68c45b7aebcf7c2f577,
queue: default,
retried_at: 1540600397.5804272,
retry: True,
retry_count: 2
},
}
What are the pros and cons of not sending them to Sentry at all vs sending them to Sentry but not being alerted?
Summary
An option that has worked well for me is by configuring Sentry's should_capture alongside Sidekiq's sidekiq_retries_exhausted with a custom attribute on the exception.
Details
1a. Add the custom attribute
You can add a custom attribute to an exception. You can define this on any error class with attr_accessor:
class SomeError
attr_accessor :ignore
alias ignore? ignore
end
1b. Rescue the error, set the custom attribute, & re-raise
def perform
# do something
rescue SomeError => e
e.ignore = true
raise e
end
Configure should_capture
should_capture allows you to capture exceptions when they meet a defined criteria. The exception is passed to it, on which you can access the custom attribute.
config.should_capture { |e| !e.ignore? }
Flip the custom attribute when retries are exhausted
There are 2 ways to define the behaviour you want to happen when a job dies, depending on the version of Sidekiq being used. If you want to apply globally & have sidekiq v5.1+, you can use a death handler. If you want to apply to a particular worker or have less than v5.1, you can use sidekiq_retries_exhausted.
sidekiq_retries_exhausted { |_job, ex| ex.ignore = false }
You can filter out the entire event if the retry_count is < N (can be done inside that sidekiq worker you posted). You will loose the data on how often this happens without alerting, but the alerts themselves will not be too noisy.
class SentryWorker < ApplicationWorker
sidekiq_options queue: :default
def perform(event)
retry_count = event.dig(:extra, :sidekiq, :job, retry_count)
if retry_count.nil? || retry_count > N
Raven.send_event(event)
end
end
end
Another idea is to set a different fingerprint depending on whether this is a retry or not. Like this:
class MyJobProcessor < Raven::Processor
def process(data)
retry_count = event.dig(:extra, :sidekiq, :job, retry_count)
if (retry_count || 0) < N
data["fingerprint"] = ["will-retry-again", "{{default}}"]
end
end
end
See https://docs.sentry.io/learn/rollups/?platform=javascript#custom-grouping
I didn't test this, but this should split up your issues into two, depending on whether sidekiq will retry them. You can then ignore one group but can still look at it whenever you need the data.
A much cleaner approach if you are trying to ignore exceptions belonging to a certain class is to add them to your config file
config.excluded_exceptions += ['ActionController::RoutingError', 'ActiveRecord::RecordNotFound']
In the above example, the exceptions Rails uses to generate 404 responses will be suppressed.
See the docs for more configuration options
From my point of view, the best option is Sentry holds all the exceptions and you could modify Sentry and set alerts to send or not the exceptions to the Slack.
In order to configure the Alerts in Sentry: In the sentry account, you could go to the ALerts option in the main menu.
In the following picture I configure an alert to only send to slack a notification if occurs an Exception of type ControllerException more than 10 times
Using this alert we only receive the notification in Slack when all conditions are accomplished
I want to test a method which makes a call to a service, if the call to that service times out I am displaying a negative feedback to user. How to unittest the timeout use case ??
My method looks like:
def method
x = callservice()
if x[:value]
display_positve_feedback("positive")
else
display_negative_feedback("negative")
end
rescue Timeout::Error => e
display_negative_feedback("Timeout, please wait for 5 mins and check again")
end
end
I have mocked callservice but how to I make that service Timeout to check the timeout use case???
You don't use a return code. If you add it, you could test for the method result.
Example:
def method
begin
x = callservice()
if x[:value]
display_positve_feedback("positive")
return true
else
display_negative_feedback("negative")
return false
end
rescue Timeout::Error => e
display_negative_feedback("Timeout, please wait for 5 mins and check again")
return nil
end
raise "This should never happen"
end
Now you can test on true, falseor nil.
Suppressing exceptions is usually not a good practice that I would recommend to anyone.
When you catch (rescue) the timeout exception, re-raise it (or your own application-specific one) in the rescue code and have the caller (your test) check that an exception is thrown for the timeout.
You could use assert_raise or expect raise_error, for example, depending on what testing framework you use.
Using an exception, the calling code only needs to know about a potential exception rather than having to check for specific return values, which makes the code simpler and more clear.
Stub the callservice method to raise a Timeout::Error. How you'll do that depends on what testing framework you're using. For example, in RSpec it might look something like this:
my_obj.stub(:callservice) { raise Timeout::Error }
expect(my_obj).to receive(:display_negative_feedback)
.with("Timeout, please wait for 5 mins and check again")
my_obj.method
I want to listening SerialPort and when message occurs then get or create Log model with id received from my device.
How to load once automatically SerialPort and keep established connection and if key_detected? in listener deal with Log model?
This is my autoloaded module in lib:
module Serialport
class Connection
def initialize(port = "/dev/tty0")
port_str = port
baud_rate = 9600
data_bits = 8
stop_bits = 1
parity = SerialPort::NONE
#sp = SerialPort.new(port_str, baud_rate, data_bits, stop_bits, parity)
#key_parts = []
#key_limit = 16 # number of slots in the RFID card.
while true do
listener
end
#sp.close
end
def key_detected?
#key_parts << #sp.getc
if #key_parts.size >= #key_limit
self.key = #key_parts.join()
#key_parts = []
true
else
false
end
end
def listener
if key_detected?
puts self.key
# log = Log.find(rfid: self.key).first_or_create(rfid: self.key)
end
end
end
end
Model:
class Log < ActiveRecord::Base
end
I would have written this in a comment, but it's a bit long... But I wonder if you could clarify your question, and I will update my answer as we go:
With all due respect to the Rails ability to "autoload", why not initialize a connection in an initialization file or while setting up the environment?
i.e., within a file in you_app/config/initializers called serial_port.rb:
SERIAL_PORT_CONNECTION = Serialport::Connection.new
Implementing an infinite loop within your Rails application will, in all probability, hang the Rails app and prevent it from being used as a web service.
What are you trying to accomplish?
If you just want to use active_record or active_support, why not just include these two gems in a separate script?
Alternatively, consider creating a separate thread for the infinite loop (or better yet, use a reactor (They are not that difficult to write, but there are plenty pre-written in the wild, such as Iodine which I wrote for implementing web services)...
Here's an example for an updated listener method, using a separate thread so you call it only once:
def listener
Thread.new do
loop { self.key while key_detected? }
# this will never be called - same as in your code.
#sp.close
end
end
I want to implement an action that calls remote service with RabbitMQ and presents returned data. I implemented this (more as a proof of concept so far) in similar way to example taken from here: https://github.com/baowen/RailsRabbit and it looks like this:
controller:
def rpc
text = params[:text]
c = RpcClient.new('RPC server route key')
response = c.call text
render text: response
end
RabbitMQ RPC client:
class RpcClient < MQ
attr_reader :reply_queue
attr_accessor :response, :call_id
attr_reader :lock, :condition
def initialize()
# initialize exchange:
conn = Bunny.new(:automatically_recover => false)
conn.start
ch = conn.create_channel
#x = ch.default_exchange
#reply_queue = ch.queue("", :exclusive => true)
#server_queue = 'rpc_queue'
#lock = Mutex.new
#condition = ConditionVariable.new
that = self
#reply_queue.subscribe do |_delivery_info, properties, payload|
if properties[:correlation_id] == that.call_id
that.response = payload.to_s
that.lock.synchronize { that.condition.signal }
end
end
end
def call(message)
self.call_id = generate_uuid
#x.publish(message.to_s,
routing_key: #server_queue,
correlation_id: call_id,
reply_to: #reply_queue.name)
lock.synchronize { condition.wait(lock) }
response
end
private
def generate_uuid
# very naive but good enough for code
# examples
"#{rand}#{rand}#{rand}"
end
end
A few tests indicate that this approach works. On the other hand, this approach assumes creating a client (and subscribing to the queue) for every request on this action, which is inefficient according to the RabbitMQ tutorial. So I've got two questions:
Is it possible to avoid creating a queue for every Rails request?
How will this approach (with threads and mutex) interfere with my whole Rails environment? Is it safe to implement things this way in Rails? I'm using Puma as my web server, if it's relevant.
Is it possible to avoid creating a queue for every Rails request?
Yes - there is no need for every single request to have it's own reply queue.
You can use the built-in direct-reply queue. See the documentation here.
If you don't want to use the direct-reply feature, you can create a single reply queue per rails instance. You can use a single reply queue, and have the correlation id help you figure out where the reply needs to go within that rails instance.
How will this approach (with threads and mutex) interfere with my whole Rails environment? Is it safe to implement things this way in Rails?
what's the purpose of the lock / mutex in this code? doesn't seem necessary to me, but i'm probably missing something since i haven't done ruby in about 5 years :)
I've got a question about how to share rspec-mocks' double between examples. I'm writing a new rails app with rspec-mocks 3.1.3. I'm used to using the old (< 2.14 and and trying to update my knowledge if current rspec usage.
I have a model method:
def self.from_strava(activity_id, race_id, user)
#client ||= Strava::Api::V3::Client.new(access_token: 'abc123')
activity = #client.retrieve_an_activity(activity_id)
result_details = {race_id: race_id, user: user}
result_details[:duration] = activity['moving_time']
result_details[:date] = Date.parse(activity['start_date'])
result_details[:comment] = activity['description']
result_details[:strava_url] = "http://www.strava.com/activities/#{activity_id}"
Result.create!(result_details)
end
And here is the spec:
describe ".from_strava" do
let(:user) { FactoryGirl.build(:user) }
let(:client) { double(:client) }
let(:json_response) { JSON.parse(File.read('spec/support/strava_response.json')) }
before(:each) do
allow(Strava::Api::V3::Client).to receive(:new) { client }
allow(client).to receive(:retrieve_an_activity) { json_response }
allow(Result).to receive(:create!)
end
it "sets the duration" do
expect(Result).to receive(:create!).with(hash_including(duration: 3635))
Result.from_strava('123', 456, user)
end
it "sets the date" do
expect(Result).to receive(:create!).with(hash_including(date: Date.parse("2014-11-14")))
Result.from_strava('123', 456, user)
end
end
When I run a single test on it's own it's fine, but when I run the whole describe ".from_strava" block it fails with the message
Double :client was originally created in one example but has leaked into another example and can no longer be used. rspec-mocks' doubles are designed to only last for one example, and you need to create a new one in each example you wish to use it for.
I understand what it's saying, but surely this is an appropriate use of a double being used in 2 examples. After all, the client double isn't important to the example, it's just a way for me to load the canned response. I guess I could use WebMock but that seems very low-level and doesn't translate well to the actual code written. We should only be asserting one thing per example after all.
I had thought about replacing the client double with a call to
allow(Strava::Api::V3::Client).to receive_message_chain(:new, :retrieve_an_activity) { json_response }
but that doesn't seem to be the right approach either, given that the documentation states that receive_message_chain should be a code smell.
So if I shouldn't use receive_message_chain, shared client double and also follow the standard DRY principle then how should I fix this?
I would love some feedback on this.
Thanks,
Dave
Caching clients for external components can often be really desired (keeping alive connections/any SSL setup that you might need, etc.) and removing that for the sake of fixing an issue with tests is not a desirable solution.
In order to fix your test (without refactoring your code), you can do the following to clear the instance variable after each of your tests:
after { Result.instance_variable_set("#client", nil) }
While admittedly, this is not the cleanest solution, it seems to be the simplest and achieves both, lets you have a clear setup with no state shared in between tests, and keep your client cached in "normal" operation mode.
surely this is an appropriate use of a double being used in 2 examples.
No, it's not. :) You're trying to use a class variable; do not do that because the variable doesn't span examples. The solution is to set the client each time i.e. in each example.
Bad:
#client ||= Strava::Api::V3::Client.new(access_token: 'abc123')
Good:
#client = Strava::Api::V3::Client.new(access_token: 'abc123')
I had the same use case in an app of mine, and we solved it by extracting the cacheing into a private method and then stubbing that method to return the double (instead of stubbing the new method directly).
For example, in the class under test:
def self.from_strava(activity_id, race_id, user)
activity = strava_client.retrieve_an_activity(activity_id)
...
end
private
def self.strava_client
#client ||= Strava::Api::V3::Client.new(access_token: 'abc123')
end
And in the spec:
let(:client) { double(:client) }
before { allow(described_class).to receive(:strava_client).and_return(client) }
...
TLDR: Add after { order.vendor_service = nil } to balance the before block. Or read on...
I ran into this, and it was not obvious where it was coming from. In order_spec.rb model tests, I had this:
describe 'order history' do
before do
service = double('VendorAPI')
allow(service).to receive(:order_count).and_return(5)
order.vendor_service = service
end
# tests here ..
end
And in my Order model:
def too_many_orders?
##vendor_service ||= VendorAPI.new(key: 'abc', account: '123')
return ##vendor_service.order_count > 10
end
This worked fine when I only ran rspec on order_spec.rb
I was mocking something completely different in order_controller_spec.rb a little differently, using allow_any_instance_of() instead of double and allow:
allow_any_instance_of(Order).to receive(:too_many_orders?).and_return(true)
This, too, tested out fine.
The confounding trouble is that when I ran the full suite of tests, I got the OP's error on the controller mock -- the one using allow_any_instance. This was very hard to track down, as the problem (or at least my solution) lay in the model tests where I use double/allow.
To fix this, I added an after block clearing the class variable ##vendor_service, balancing the before block's action:
describe 'order history' do
before do
service = double('VendorAPI')
allow(service).to receive(:order_count).and_return(5)
order.vendor_service = service
end
after do
order.vendor_service = nil
end
# tests here ..
end
This forced the ||= VendorAPI.new() to use the real new function in later unrelated tests, not the mock object.