Looking to use em-mongo for a text analyzer script which loads text from db, analyzes it, flags keywords and updates the db.
Would love to see some examples of em-mongo in action. Only one I could find was on github em-mongo repo.
require 'em-mongo'
EM.run do
db = EM::Mongo::Connection.new.db('db')
collection = db.collection('test')
EM.next_tick do
doc = {"hello" => "world"}
id = collection.insert(doc)
collection.find('_id' => id]) do |res|
puts res.inspect
EM.stop
end
collection.remove(doc)
end
end
You don't need the next_tick method, that is em-mongo doing for you. Define callbacks, that are executed if the db actions are done. Here is a skeleton:
class NonBlockingFetcher
include MongoConfig
def initialize
configure
#connection = EM::Mongo::Connection.new(#server, #port)
#collection = init_collection(#connection)
end
def fetch(value)
mongo_cursor = #collection.find({KEY => value.to_s})
response = mongo_cursor.defer_as_a
response.callback do |documents|
# foo
# get one document
doc = documents.first
end
response.errback do |err|
# foo
end
end
end
Related
Question:
I need to know the records' attributes that have been called inside a block (say I need something like the following):
def my_custom_method(&block)
some_method_that_starts_tracking
block.call
some_method_that_stops_tracking
puts some_method_that_returns_called_records_attributes
do_something_about(some_method_that_returns_called_records_attributes)
end
my_custom_method { somecodethatcallsauthorofbook1andemailandfirstnameofuser43 }
# this is the `puts` output above (just as an example)
# => {
# #<Book id:1...> => [:author],
# #<User id:43...> => [:email, :first_name]
# }
code inside the block can be anything
Specifically, I meant to track any instance of a subclass of ApplicationRecord, so it can be instance of any models like Book, User, etc...
Attempts:
From my understanding, this is similar to how rspec works when a method is expected to be called. That it somehow tracks any calls of that method. So, my initial attempt is to do something like the following (which does not yet fully work):
def my_custom_method(&block)
called_records_attributes = {}
ApplicationRecord.descendants.each do |klass|
klass.class_eval do
attribute_names.each do |attribute_name|
define_method(attribute_name) do
called_records_attributes[self] ||= []
called_records_attributes[self] << attribute_name
self[attribute_name]
end
end
end
end
block.call
# the above code will work but at this point, I don't know how to clean the methods that were defined above, as the above define_methods should only be temporary
puts called_records_attributes
end
my_custom_method { Book.find_by(id: 1).title }
# => {
# #<Book id: 1...> => ['title']
# }
the .descendants above probably is not a good idea because Rails use autoload if I'm not mistaken
as already said above in the comment, I do not know how to remove these "defined_methods" that are just supposed to be only temporary for the duration of this "block".
furthermore, my code above would probably have overriden the "actual" attribute getters of the models, if ever any has been already defined, which is bad.
Background:
I am writing a gem live_record which I am adding a new feature that will allow a developer to just simply write something like
<!-- app/views/application.html.erb -->
<body>
<%= live_record_sync { #book.some_custom_method_about_book } %>
</body>
... which will render #book.some_custom_method_about_book as-is on the page, but at the same time the live_record_sync wrapper method would take note of all the attributes that have been called inside that block (i.e. inside some_custom_method_about_book the #book.title is called), and then it sets these attributes as the block's own "dependencies", in which later when that specific book's attribute has been updated, I can already also update directly the HTML page of which this attribute is a "dependency" as like specified just above. I am aware that this is not an accurate solution, but I'd like to open up my chances by experimenting on this first.
-- Rails 5
Disclaimer: I believe this is just a mediocre solution, but hopefully helps anyone with the same problem.
I tried reading rspec source code, but because I couldn't easily comprehend what is happening under the hood, and that it occurred to me that rspec's (i.e.) expect(Book.first).to receive(:title) is different from what I really want because the methods there are already specified (i.e. :title), while what I want is to track ANY methods that are attributes, so because of these two reasons I skipped reading further, and attempted my own solution, which hopefully did somehow work; see below.
Note that I am using Thread local-storage here, so this code should be thread-safe (untested yet).
# lib/my_tracker.rb
class MyTracker
Thread.current[:my_tracker_current_tracked_records] = {}
attr_accessor :tracked_records
class << self
def add_to_tracked_records(record, attribute_name)
Thread.current[:my_tracker_current_tracked_records][{model: record.class.name.to_sym, record_id: record.id}] ||= []
Thread.current[:my_tracker_current_tracked_records][{model: record.class.name.to_sym, record_id: record.id}] << attribute_name
end
end
def initialize(block)
#block = block
end
def call_block_while_tracking_records
start_tracking
#block_evaluated_value = #block.call
#tracked_records = Thread.current[:my_tracker_current_tracked_records]
stop_tracking
end
def to_s
#block_evaluated_value
end
# because I am tracking record-attributes, and you might want to track a different object / method, then you'll need to write your own `prepend` extension (look for how to use `prepend` in ruby)
module ActiveRecordExtensions
def _read_attribute(attribute_name)
if Thread.current[:my_tracker_current_tracked_records] && !Thread.current[:my_tracker_is_tracking_locked] && self.class < ApplicationRecord
# I added this "lock" to prevent infinite loop inside `add_to_tracked_records` as I am calling the record.id there, which is then calling this _read_attribute, and then loops.
Thread.current[:my_tracker_is_tracking_locked] = true
::MyTracker.add_to_tracked_records(self, attribute_name)
Thread.current[:my_tracker_is_tracking_locked] = false
end
super(attribute_name)
end
end
module Helpers
def track_records(&block)
my_tracker = MyTracker.new(block)
my_tracker.call_block_while_tracking_records
my_tracker
end
end
private
def start_tracking
Thread.current[:my_tracker_current_tracked_records] = {}
end
def stop_tracking
Thread.current[:my_tracker_current_tracked_records] = nil
end
end
ActiveSupport.on_load(:active_record) do
prepend MyTracker::ActiveRecordExtensions
end
ActiveSupport.on_load(:action_view) do
include MyTracker::Helpers
end
ActiveSupport.on_load(:action_controller) do
include MyTracker::Helpers
end
Usage Example
some_controller.rb
book = Book.find_by(id: 1)
user = User.find_by(id: 43)
my_tracker = track_records do
book.title
if user.created_at == book.created_at
puts 'same date'
end
'thisisthelastlineofthisblockandthereforewillbereturned'
end
puts my_tracker.class
# => #<MyTracker ... >
puts my_tracker.tracked_records
# => {
# {model: :Book, record_id: 1} => ['title', 'created_at'],
# {model: :User, record_id: 43} => ['created_at']
# }
puts my_tracker
# => 'thisisthelastlineofthisblockandthereforewillbereturned'
# notice that `puts my_tracker` above prints out the block itself
# this is because I defined `.to_s` above.
# I need this `.to_s` so I can immediately print the block as-is in the views.
# see example below
some_view.html.erb
<%= track_records { current_user.email } %>
P.S. Maybe it's better that I wrap this up as a gem. If you're interested, let me know
Looking for advice on how to fix this error and refactor this code to improve it.
require 'mechanize'
require 'pry'
require 'pp'
module Mymodule
class WebBot
agent = Mechanize.new { |agent|
agent.user_agent_alias = 'Windows Chrome'
}
def form(response)
require "addressable/uri"
require "addressable/template"
template = Addressable::Template.new("http://www.domain.com/{?query*}")
url = template.expand({"query" => response}).to_s
page = agent.get(url)
end
def get_products
products = []
page.search("datatable").search('tr').each do |row|
begin
product = row.search('td')[1].text
rescue => e
p e.message
end
products << product
end
products
end
end
end
Calling the module:
response = {size: "SM", color: "BLUE"}
t = Mymodule::WebBot.new
t.form(response)
t.get_products
Error:
NameError: undefined local variable or method `agent'
Ruby has a naming convention. agent is a local variable in the class scope. To make it visible to other methods you should make it a class variable by naming it ##agent, and it'll be shared among all the objects of WebBot. The preferred way though is to make it an instance variable by naming it #agent. Every object of WebBot will have its own #agent. But you should put it in initialize, initialize will be invoked when you create a new object with new
class WebBot
def initialize
#agent = Mechanize.new do |a|
a.user_agent_alias = 'Windows Chrome'
end
end
.....
And the same error will occur to page. You defined it in form as a local variable. When form finishes execution, it'll be deleted. You should make it an instance variable. Fortunately, you don't have to put it in initialize. You can define it here in form. And the object will have its own #page after invoking form. Do this in form:
def form(response)
require "addressable/uri"
require "addressable/template"
template = Addressable::Template.new("http://www.domain.com/{?query*}")
url = template.expand({"query" => response}).to_s
#page = agent.get(url)
end
And remember to change every occurrence of page and agent to #page and #agent. In your get_products for example:
def get_products
products = []
#page.search("datatable").search('tr').each do |row|
.....
These changes will resolve the name errors. Refactoring is another story btw.
I have a Ruby gem which wraps an API. I have two classes: Client and Season with a Configuration module. But I can't access a change to the API Key, Endpoint made via Client in the Season class.
My ApiWrapper module looks like this:
require "api_wrapper/version"
require 'api_wrapper/configuration'
require_relative "api_wrapper/client"
require_relative "api_wrapper/season"
module ApiWrapper
extend Configuration
end
My Configuration module looks like this:
module ApiWrapper
module Configuration
VALID_CONNECTION_KEYS = [:endpoint, :user_agent, :method].freeze
VALID_OPTIONS_KEYS = [:api_key, :format].freeze
VALID_CONFIG_KEYS = VALID_CONNECTION_KEYS + VALID_OPTIONS_KEYS
DEFAULT_ENDPOINT = 'http://defaulturl.com'
DEFAULT_METHOD = :get
DEFAULT_API_KEY = nil
DEFAULT_FORMAT = :json
attr_accessor *VALID_CONFIG_KEYS
def self.extended(base)
base.reset
end
def reset
self.endpoint = DEFAULT_ENDPOINT
self.method = DEFAULT_METHOD
self.user_agent = DEFAULT_USER_AGENT
self.api_key = DEFAULT_API_KEY
self.format = DEFAULT_FORMAT
end
def configure
yield self
end
def options
Hash[ * VALID_CONFIG_KEYS.map { |key| [key, send(key)] }.flatten ]
end
end # Configuration
end
My Client class looks like this:
module ApiWrapper
class Client
attr_accessor *Configuration::VALID_CONFIG_KEYS
def initialize(options={})
merged_options = ApiWrapper.options.merge(options)
Configuration::VALID_CONFIG_KEYS.each do |key|
send("#{key}=", merged_options[key])
end
end
end # Client
end
My Season class looks like this:
require 'faraday'
require 'json'
API_URL = "/seasons"
module ApiWrapper
class Season
attr_accessor *Configuration::VALID_CONFIG_KEYS
attr_reader :id
def initialize(attributes)
#id = attributes["_links"]["self"]["href"]
...
end
def self.all
puts ApiWrapper.api_key
puts ApiWrapper.endpoint
conn = Faraday.new
response = Faraday.get("#{ApiWrapper.endpoint}#{API_URL}/") do |request|
request.headers['X-Auth-Token'] = "ApiWrapper.api_key"
end
seasons = JSON.parse(response.body)
seasons.map { |attributes| new(attributes) }
end
end
end
This is the test I am running:
def test_it_gives_back_a_seasons
VCR.use_cassette("season") do
#config = {
:api_key => 'ak',
:endpoint => 'http://ep.com',
}
client = ApiWrapper::Client.new(#config)
result = ApiWrapper::Season.all
# Make sure we got all season data
assert_equal 12, result.length
#Make sure that the JSON was parsed
assert result.kind_of?(Array)
assert result.first.kind_of?(ApiWrapper::Season)
end
end
Because I set the api_key via the client to "ak" and the endpoint to "http://ep.com" I would expect puts in the Season class's self.all method to print out "ak" and "http://ep.com", but instead I get the defaults set in the Configuration section.
What I am doing wrong?
The api_key accessors you have on Client and on ApiWrapper are independent. You initialize a Client with the key you want, but then Season references ApiWrapper directly. You've declared api_key, etc. accessors in three places: ApiWrapper::Configuration, ApiWrapper (by extending Configuration) and Client. You should probably figure out what your use cases are and reduce that down to being in just one place to avoid confusion.
If you're going to have many clients with different API keys as you make different requests, you should inject the client into Season and use it instead of ApiWrapper. That might look like this:
def self.all(client)
puts client.api_key
puts client.endpoint
conn = Faraday.new
response = Faraday.get("#{client.endpoint}#{API_URL}/") do |request|
request.headers['X-Auth-Token'] = client.api_key
end
seasons = JSON.parse(response.body)
seasons.map { |attributes| new(attributes) }
end
Note that I also replaced the "ApiWrapper.api_key" string with the client.api_key - you don't want that to be a string anyway.
Having to pass client into every request you make is going to get old, so then you might want to pull out something like a SeasonQuery class to hold onto it.
If you're only ever going to have one api_key and endpoint for the duration of your execution, you don't really need the Client as you've set it up so far. Just set ApiWrapper.api_key directly and continue using it in Season.
I have this test:
describe 'check_account_status' do
it 'should send the correct reminder email one week prior to account disablement' do
# Three weeks since initial email
reverification = create(:reverification)
initial_notification = reverification.twitter_reverification_sent_at.to_datetime
ActionMailer::Base.deliveries.clear
Timecop.freeze(initial_notification + 21) do
Reverification.check_account_status
ActionMailer::Base.deliveries.size.must_equal 1
ActionMailer::Base.deliveries.first.subject.must_equal I18n.t('.account_mailer.one_week_left.subject')
reverification.reminder_sent_at.class.must_equal ActiveSupport::TimeWithZone
reverification.notification_counter.must_equal 1
must_render_template 'reverification.html.haml'
end
end
This test produces this error:
check_account_status#test_0001_should send the correct reminder email one week prior to account disablement [/Users/drubio/vms/ohloh-ui/test/models/reverification_test.rb:67]:
Expected: ActiveSupport::TimeWithZone
Actual: NilClass
Here is my code:
class Reverification < ActiveRecord::Base
belongs_to :account
FIRST_NOTIFICATION_ERROR = []
class << self
def check_account_status
Reverification.where(twitter_reverified: false).each do |reverification|
calculate_status(reverification)
one_week_left(reverification)
end
end
private
def calculate_status(reverification)
#right_now = Time.now.utc.to_datetime
#initial_email_date = reverification.twitter_reverification_sent_at.to_datetime
#notification_counter = reverification.notification_counter
end
def one_week_left(reverification)
# Check to see if three weeks have passed since the initial email
# and check to see if its before the one day notification before
# marking an account as spam.
if (#right_now.to_i >= (#initial_email_date + 21).to_i) && (#right_now.to_i < (#initial_email_date + 29).to_i)
begin
AccountMailer.one_week_left(reverification.account).deliver_now
rescue
FIRST_NOTIFICATION_FAILURE << account.id
return
end
update_reverification_fields(reverification)
end
end
def update_reverification_fields(reverification)
reverification.notification_counter += 1
reverification.reminder_sent_at = Time.now.utc
reverification.save!
reverification.reload
end
end
Forgive the indentation, but what seems to be the problem, is that my reverification object doesn't update when it leaves the check_account_status method. I've placed puts statements through out the code and I can see without a doubt that the reverification object is indeed updating. However after it leaves the update_reverification_fields and returns to the test block, the fields are not updated. Why is that? Has anyone encountered this?
I believe you have a scope issue, the methods you call from check_account_status certainly don't return the updated object back to your method and Ruby only passes parameters by value.
Try something like this instead:
def check_account_status
Reverification.where(twitter_reverified: false).each do |reverification|
reverification = calculate_status(reverification)
reverification = one_week_left(reverification)
end
end
private
def calculate_status(reverification)
# ...
reverification
end
def one_week_left(reverification)
# ...
reverification = update_reverification_fields(reverification)
reverification
end
def update_reverification_fields(reverification)
# ...
reverification
end
The problem is that reverification object in your test and objects inside of check_account_status are different instances of the same model.
def update_reverification_fields(reverification)
reverification.notification_counter += 1
reverification.reminder_sent_at = Time.now.utc
reverification.save!
reverification.reload
end
This reload here, it's doing nothing. Let's walk through your test.
# check_account_status runs a DB query, finds some objects and does things to them
Reverification.check_account_status
# this expectation succeeds because you're accessing `deliveries` for the
# first time and you don't have it cached. So you get the actual value
ActionMailer::Base.deliveries.size.must_equal 1
# this object, on the other hand, was instantiated before
# `check_account_status` was called and, naturally, it doesn't see
# the database changes that completely bypassed it.
reverification.reminder_sent_at.class.must_equal ActiveSupport::TimeWithZone
So, before making expectations on reverification, reload it, so that it pulls latest data from the DB.
reverification.reload # here
reverification.reminder_sent_at.class.must_equal ActiveSupport::TimeWithZone
I've got a gem where in one of classes is sth similiar:
class Test
TESTING = {
:sth1 => 'foo',
:sth2 => 'bar'
}
# p Test.new.show
# should print 'cat'
def show
p TESTING[:sth3]
end
end
I extended in other file
# in other file
class Test
TESTING = {
:sth3 => 'cat'
}
end
But i need to use :sth3 in first file, as the first part of code stands.
Thx in advance.
You didn't extend it, you replaced the hash with a new one. Here's how to fix it:
# in the other file
Test::TESTING[:sth3] = 'cat'
I recommend using methods with lazy initialization, so that you can arrange the assignments in any order:
class Test
def self.testing
#testing ||= {}
end
testing[:sth1] = 'foo'
testing[:sth2] = 'bar'
end
# in the other file
Test.testing[:sth3] = 'cat'