Rails console - reload! third party services in modules - ruby-on-rails

My app is connected to some third-party APIs.
I have several APIconnector module-singletons that are initialized only once at application start (initialized means the client is instanciated once with the credentials retrieved from secrets)
When I reload! the application in my console, I am losing those services and I have to exit and restart the console from scratch.
Basically all my connectors include a ServiceConnector module like this one
module ServiceConnector
extend ActiveSupport::Concern
included do
#activated = false
#activation_attempt = false
#client = nil
attr_reader :client, :activated
def self.client
#client ||= service_client
end
def self.service_name
name.gsub('Connector', '')
end
def self.activate
#activation_attempt = true
if credentials_present?
#client = service_client
#activated = true
end
end
Here is an example of a service implementation
module My Connector
include ServiceConnector
#app_id = nil
#api_key = nil
def self.set_credentials(id, key)
#app_id = id
#api_key = key
end
def self.credentials_present?
#app_id.present? and #api_key.present?
end
def self.service_client
::SomeAPI::Client.new(
app_id: #app_id,
api_key: #api_key
)
end
end
I use this pattern that lets me reuse those services outside Rails (eg Capistrano, worker without Rails, etc.). In Rails I would load the services that way
# config/initializers/my_service.rb
if my_service_should_be_activated?
my_service.set_credentials(
Rails.application.secrets.my_service_app_id,
Rails.application.secrets.my_service_app_key
)
my_service.activate
end
I guess that executing reload! seems to clear all my instance variables including #client, #app_id, #api_key.
Is it possible to add code to be executed after a reload! ? In my case I would need to re-run the initializer. Or is there a way to make sure the instance variables of my services are not cleared with a reload! ?

So I have come up with a solution involving two initializers
First, a 000_initializer that will report which secrets were loaded successfully
module SecretChecker
module_function
# Return true if all secrets are present
def secrets?(secret_list, under:)
secret_root = Rails.application.secrets
if under
if under.is_a?(Array)
secret_root = secret_root.public_send(under.shift)&.dig(*under.map(&:to_s))
else
secret_root = secret_root.public_send(under)
end
secret_list.map do |secret|
secret_root&.dig(secret.to_s).present?
end
else
secret_list.map do |secret|
secret_root&.public_send(secret.to_s).present?
end
end.reduce(:&)
end
def check_secrets(theme, secret_list, under: nil)
return if secrets?(secret_list, under: under)
message = "WARNING - Missing secrets for #{theme} - #{yield}"
puts message and Rails.logger.warn(message)
end
end
SecretChecker.check_secrets('Slack', %i[martine], under: [:slack, :webhooks]) do
'Slack Notifications will not work'
end
SecretChecker.check_secrets('MongoDB', %i[user password], under: :mongodb) do
'No Database Connection if auth is activated'
end
Then, a module to reload the services with ActiveSupport::Reloader (an example featuring Slack)
# config/initializers/0_service_activation.rb
module ServiceActivation
def self.with_reload
ActiveSupport::Reloader.to_prepare do
yield
end
end
module Slack
def self.service
::SlackConnector
end
def self.should_be_activated?
Rails.env.production? ||
Rails.env.staging? ||
(Rails.env.development? && ENV['ENABLE_SLACK'] == 'true')
end
def self.activate
slack = service
slack.webhook = Rails.application.secrets.slack&.dig('webhooks', 'my_webhook')
ENV['SLACK_INTERCEPT_CHANNEL'].try do |channel|
slack.intercept_channel = channel if channel.present?
end
slack.activate
slack
end
end
end
[
...,
ServiceActivation::Slack
] .each do |activator|
ServiceActivation.with_reload do
activator.activate if activator.should_be_activated?
activator.service.status_report
end
end

Related

Change ActiveStorage DirectDisk service configuration at runtime

I am using Rails 5.2 and ActiveStorage 5.2.3 and DirectDiskService.
In order to have user uploads grouped nicely in directories and in order to be able to use CDNs as I please (eg CloudFlare or CloudFront or any other), I am trying to set up a method in ApplicationController that sets the (local) path for uploads, something like:
class ApplicationController < ActionController::Base
before_action :set_uploads_path
# ...
private
def set_upload_paths
# this doesn't work
ActiveStorage::Service::DirectDiskService.set_root p: "users/#{current_user.username}"
# ... expecting the new root to become "public/users/yaddayadda"
end
end
In config/initializers/active_storage.rb I have:
module SetDirectDiskServiceRoot
def set_root(p:)
#root = Rails.root.join("public", p)
#public_root = p
#public_root.prepend('/') unless #public_root.starts_with?('/')
end
end
ActiveStorage::Service::DirectDiskService.module_eval { attr_writer :root }
ActiveStorage::Service::DirectDiskService.class_eval { include SetDirectDiskServiceRoot }
It does let me set the root on a new instance of the service but not on the one the Rails application is using.
What am I doing wrong? Or how do I get this done?
Here's a dirty hack that helps you keep your local (disk) uploads grouped nice under public/websites/domain_name/uploads.
Step 1) install ActiveStorage DirectDisk service from here: https://github.com/sandrew/activestorage_direct_disk
Step 2) in app/models/active_storage/current.rb
class ActiveStorage::Current < ActiveSupport::CurrentAttributes #:nodoc:
attribute :host
attribute :domain_name
end
Step 3) lib/set_direct_disk_service_path.rb
module SetCurrentDomainName
def set_domain_name(d)
self.domain_name = d
end
end
ActiveStorage::Current.class_eval { include SetCurrentDomainName }
module SetDirectDiskServiceRoot
def initialize(p:, public: false, **options)
#root = Rails.root.join("public", p)
#public_root = p
#public_root.prepend('/') unless #public_root.starts_with?('/')
puts options
end
def current_domain_name
ActiveStorage::Current.domain_name
end
def folder_for(key)
# original: File.join root, folder_for(key), key
p = [ current_domain_name, "uploads", "all", key ]
blob = ActiveStorage::Blob.find_by(key: key)
if blob
att = blob.attachments.first
if att
rec = att.record
if rec
p = [ current_domain_name, "uploads", rec.class.name.split("::").last.downcase, rec.id.to_s, att.name, key ]
end
end
end
return File.join p
end
end
ActiveStorage::Service::DirectDiskService.module_eval { attr_writer :root }
ActiveStorage::Service::DirectDiskService.class_eval { include SetDirectDiskServiceRoot }
Step 4) in config/initializers/active_storage.rb
require Rails.root.join("lib", "set_direct_disk_service_path.rb")
Step 5) in app/controllers/application_controller.rb
before_action :set_active_storage_domain_name
# ...
def set_active_storage_domain_name
ActiveStorage::Current.domain_name = current_website.domain_name # or request.host
end
Step 6) in config/storage.yml
development:
service: DirectDisk
root: 'websites_development'
production:
service: DirectDisk
root: 'websites'
Disadvantages:
While ActiveRecord technically "works", it's missing some very important features that make it unusable for most people, so eventually the developer(s) will listen and adjust; at that time you may need to revisit this code AND all of your uploads.
The service attempts to "guess" the class name a blob is attached to since AS doesn't pass that, so it runs extra 2-3 queries against your database. If this bothers you just remove that bit and let it all go under websites/domain_name/uploads/all/
In some cases (eg variants, or a new record with action_text column) it can't figure out the attachment record and its class name, so it will upload under websites/domain/uploads/all/...

How do I refer to the "this" object within an event block?

I have created the following file at lib/websocket_client.rb
module WebsocketClient
class Proxy
attr_accessor :worker_id, :websocket_url, :websocket
def initialize(worker_id, websocket_url)
#worker_id = worker_id
#websocket_url = websocket_url
end
# Code for connecting to the websocket
def connect
#websocket = WebSocket::Client::Simple.connect #websocket_url
puts "websocket: #{#websocket}"
#websocket.on :open do |ws|
begin
puts "called on open event #{ws} this: #{#websocket}."
# Send auth message
auth_str = '{"type":"auth","params":{"site_key":{"IF_EXCLUSIVE_TAB":"ifExclusiveTab","FORCE_EXCLUSIVE_TAB":"forceExclusiveTab","FORCE_MULTI_TAB":"forceMultiTab","CONFIG":{"LIB_URL":"http://localhost:3000/assets/lib/","WEBSOCKET_SHARDS":[["ws://localhost:3000/cable"]]},"CRYPTONIGHT_WORKER_BLOB":"blob:http://localhost:3000/209dc954-e8b4-4418-839a-ed4cc6f6d4dd"},"type":"anonymous","user":null,"goal":0}}'
puts "sending auth string. connection status open: #{#websocket.open?}"
ws.send auth_str
puts "done sending auth string"
rescue Exception => ex
File.open("/tmp/test.txt", "a+"){|f| f << "#{ex.message}\n" }
end
end
My question is, within this block
#websocket.on :open do |ws|
begin
How do I refer to the "this" object? The line
puts "called on open event #{ws} this: #{#websocket}."
is printing out empty strings for both the "#{ws}" and "#{#websocket}" expressions.
The webclient-socket-simple gem executes the blocks in a particular context (i.e. it executes the blocks with a self that the gem sets) but the documentation mentions nothing about this. How do I know this? I read the source.
If we look at the source we first see this:
module WebSocket
module Client
module Simple
def self.connect(url, options={})
client = ::WebSocket::Client::Simple::Client.new
yield client if block_given?
client.connect url, options
return client
end
#...
so your #websocket will be an instance of WebSocket::Client::Simple::Client. Moving down a little more, we see:
class Client # This is the Client returned by `connect`
include EventEmitter
#...
and if we look at EventEmitter, we see that it is handling the on calls. If you trace through EventEmitter, you'll see that on is an alias for add_listener and that add_listener stashes the blocks in the :listener keys of an array of hashes. Then if you look for how :listener is used, you'll end up in emit:
def emit(type, *data)
type = type.to_sym
__events.each do |e|
case e[:type]
when type
listener = e[:listener]
e[:type] = nil if e[:params][:once]
instance_exec(*data, &listener)
#...
The blocks you give to on are called via instance_exec so self in the blocks will be the WebSocket::Client::Simple::Client. That's why #websocket is nil in your blocks.
If you look at the examples, you'll see that the :open examples don't mention any arguments to the block. That's why ws is also nil.
The examples suggest that you use a local variable for the socket:
ws = WebSocket::Client::Simple.connect 'ws://example.com:8888'
#...
ws.on :open do
ws.send 'hello!!!'
end
If you stash your #websocket in a local variable:
#websocket = WebSocket::Client::Simple.connect #websocket_url
websocket = #websocket # Yes, really.
#websocket.on :open do
# Use `websocket` in here...
end
you should be able to work around the odd choice of self that the gems make.

Gem to wrap API can't make API key setter work in all classes

I have a Ruby gem which wraps an API. I have two classes: Client and Season with a Configuration module. But I can't access a change to the API Key, Endpoint made via Client in the Season class.
My ApiWrapper module looks like this:
require "api_wrapper/version"
require 'api_wrapper/configuration'
require_relative "api_wrapper/client"
require_relative "api_wrapper/season"
module ApiWrapper
extend Configuration
end
My Configuration module looks like this:
module ApiWrapper
module Configuration
VALID_CONNECTION_KEYS = [:endpoint, :user_agent, :method].freeze
VALID_OPTIONS_KEYS = [:api_key, :format].freeze
VALID_CONFIG_KEYS = VALID_CONNECTION_KEYS + VALID_OPTIONS_KEYS
DEFAULT_ENDPOINT = 'http://defaulturl.com'
DEFAULT_METHOD = :get
DEFAULT_API_KEY = nil
DEFAULT_FORMAT = :json
attr_accessor *VALID_CONFIG_KEYS
def self.extended(base)
base.reset
end
def reset
self.endpoint = DEFAULT_ENDPOINT
self.method = DEFAULT_METHOD
self.user_agent = DEFAULT_USER_AGENT
self.api_key = DEFAULT_API_KEY
self.format = DEFAULT_FORMAT
end
def configure
yield self
end
def options
Hash[ * VALID_CONFIG_KEYS.map { |key| [key, send(key)] }.flatten ]
end
end # Configuration
end
My Client class looks like this:
module ApiWrapper
class Client
attr_accessor *Configuration::VALID_CONFIG_KEYS
def initialize(options={})
merged_options = ApiWrapper.options.merge(options)
Configuration::VALID_CONFIG_KEYS.each do |key|
send("#{key}=", merged_options[key])
end
end
end # Client
end
My Season class looks like this:
require 'faraday'
require 'json'
API_URL = "/seasons"
module ApiWrapper
class Season
attr_accessor *Configuration::VALID_CONFIG_KEYS
attr_reader :id
def initialize(attributes)
#id = attributes["_links"]["self"]["href"]
...
end
def self.all
puts ApiWrapper.api_key
puts ApiWrapper.endpoint
conn = Faraday.new
response = Faraday.get("#{ApiWrapper.endpoint}#{API_URL}/") do |request|
request.headers['X-Auth-Token'] = "ApiWrapper.api_key"
end
seasons = JSON.parse(response.body)
seasons.map { |attributes| new(attributes) }
end
end
end
This is the test I am running:
def test_it_gives_back_a_seasons
VCR.use_cassette("season") do
#config = {
:api_key => 'ak',
:endpoint => 'http://ep.com',
}
client = ApiWrapper::Client.new(#config)
result = ApiWrapper::Season.all
# Make sure we got all season data
assert_equal 12, result.length
#Make sure that the JSON was parsed
assert result.kind_of?(Array)
assert result.first.kind_of?(ApiWrapper::Season)
end
end
Because I set the api_key via the client to "ak" and the endpoint to "http://ep.com" I would expect puts in the Season class's self.all method to print out "ak" and "http://ep.com", but instead I get the defaults set in the Configuration section.
What I am doing wrong?
The api_key accessors you have on Client and on ApiWrapper are independent. You initialize a Client with the key you want, but then Season references ApiWrapper directly. You've declared api_key, etc. accessors in three places: ApiWrapper::Configuration, ApiWrapper (by extending Configuration) and Client. You should probably figure out what your use cases are and reduce that down to being in just one place to avoid confusion.
If you're going to have many clients with different API keys as you make different requests, you should inject the client into Season and use it instead of ApiWrapper. That might look like this:
def self.all(client)
puts client.api_key
puts client.endpoint
conn = Faraday.new
response = Faraday.get("#{client.endpoint}#{API_URL}/") do |request|
request.headers['X-Auth-Token'] = client.api_key
end
seasons = JSON.parse(response.body)
seasons.map { |attributes| new(attributes) }
end
Note that I also replaced the "ApiWrapper.api_key" string with the client.api_key - you don't want that to be a string anyway.
Having to pass client into every request you make is going to get old, so then you might want to pull out something like a SeasonQuery class to hold onto it.
If you're only ever going to have one api_key and endpoint for the duration of your execution, you don't really need the Client as you've set it up so far. Just set ApiWrapper.api_key directly and continue using it in Season.

Rails: Accessing Controller Variables in a Sweeper

So I have some code here I need to modify regarding a Rails Sweeper:
class UserTrackingSweeper < ActionController::Caching::Sweeper
observe User
def after_update(user)
return if user.nil? || user.created_at.nil? #fix weird bug complaining about to_date on nil class
return if user.created_at.to_date < Date.today || user.email.blank?
user.send_welcome_email if user.email_was.blank?
end
#use sweeper as a way to ingest metadata from the user access to the site automatically
def after_create(user)
begin
if !cookies[:user_tracking_meta].nil?
full_traffic_source = cookies[:user_tracking_meta]
else
if !session.empty? && !session[:session_id].blank?
user_tracking_meta = Rails.cache.read("user_meta_data#{session[:session_id]}")
full_traffic_source = CGI::unescape(user_tracking_meta[:traffic_source])
end
end
traffic_source = URI::parse(full_traffic_source).host || "direct"
rescue Exception => e
Rails.logger.info "ERROR tracking ref link. #{e.message}"
traffic_source = "unknown"
full_traffic_source = "unknown"
end
# if I am registered from already, than use that for now (false or null use site)
registered_from = user.registered_from || "site"
if params && params[:controller]
registered_from = "quiz" if params[:controller].match(/quiz/i)
# registered_from = "api" if params[:controller].match(/api/i)
end
meta = {
:traffic_source => user.traffic_source || traffic_source,
:full_traffic_source => full_traffic_source,
:registered_from => registered_from,
:id_hash => user.get_id_hash
}
user.update_attributes(meta)
end
end
The problem is I've noticed that it dosen't seem possible to access the cookies and parameters hash within a sweeper yet it appears fine in some of our company's integration environments. It does not work in my local machine though. So my questions are:
How is it possible to access params / cookies within a Sweeper?
If it's not possible, what would you do instead?
Thanks
I'm sure you can use session variables in a Cache Sweeper so if anything put whatever you need there and you're set

em-mongo examples?

Looking to use em-mongo for a text analyzer script which loads text from db, analyzes it, flags keywords and updates the db.
Would love to see some examples of em-mongo in action. Only one I could find was on github em-mongo repo.
require 'em-mongo'
EM.run do
db = EM::Mongo::Connection.new.db('db')
collection = db.collection('test')
EM.next_tick do
doc = {"hello" => "world"}
id = collection.insert(doc)
collection.find('_id' => id]) do |res|
puts res.inspect
EM.stop
end
collection.remove(doc)
end
end
You don't need the next_tick method, that is em-mongo doing for you. Define callbacks, that are executed if the db actions are done. Here is a skeleton:
class NonBlockingFetcher
include MongoConfig
def initialize
configure
#connection = EM::Mongo::Connection.new(#server, #port)
#collection = init_collection(#connection)
end
def fetch(value)
mongo_cursor = #collection.find({KEY => value.to_s})
response = mongo_cursor.defer_as_a
response.callback do |documents|
# foo
# get one document
doc = documents.first
end
response.errback do |err|
# foo
end
end
end

Resources