I have two tables (nodes and agents). nodes are belong_to agents. I have a script that is pulling the Rails project into it and I'm trying to pull in values from the ActiveRecord. I'm assuming what I'm asking should work whether it's in a controller or view -or- in a cli script. So, my script looks thusly:
#!/usr/bin/env ruby
require '/Users/hseritt/devel/rails_projects/monitor_app/config/environment'
banner = "Banner running..."
script_dir = '/devel/rails_projects/monitor_app/monitor/bin'
class Runner
attr_accessor :banner, :agents, :agent_module, :nodes
def initialize(banner, script_dir)
#banner = banner
#agents = Agent.all
#nodes = Node.all
#script_dir = script_dir
end
def run
puts #banner
#agents.each do |agent|
if agent.active?
agent_module = '%s/%s' % [#script_dir, agent.name]
require agent_module
#nodes.each do |node|
if node.agent == agent
puts node.name
end
end
#
# HERE IS THE ISSUE:
# ns = Node.find_by_agent_id(agent.id)
# ns.each do |node|
# puts node.name
# end
#
# yields this error:
#`method_missing': undefined method `each' for #<Node:0x007fe4dc4beba0> (NoMethodError)
# I would think `ns` here would be itterable but it doesn't seem that way.
end
end
end
end
if $0 == __FILE__
runner = Runner.new(banner, script_dir)
runner.run
end
So, this is in the run method. The block that is not commented out but of course this is not a good solution since each time you iterate through agents you'll have to iterate through nodes each time. The block that is commented out seemed logical to me but throws an error. I'm having trouble probably googling the right thing here I think. What am I missing here?
Node.find_all_by_agent_id
if you don't use "all", it takes the first element only
Related
I'm using the Mongoid in a Rails project. To improve the performance of large queries, I'm using the includes method to eager load the relationships.
I would like to know if there is an easy way to count the real number of queries performed by a block of code so that I can check if my includes really reduced the number of DB accesses as expected. Something like:
# It will perform a large query to gather data from companies and their relationships
count = Mongoid.count_queries do
Company.to_csv
end
puts count # Number of DB access
I want to use this feature to add Rspec tests to prove that my query remains efficient after changes (e.g; when adding data from a new relationship). In python's Django framework, for instance, one may use the assertNumQueries method to this end.
Checking on rubygems.org didn't yield anything that seems to do what you want.
You might be better off looking into app performance tools like New Relic, Scout, or DataDog. You may be able to get some out of the gate benchmarking specs with
https://github.com/piotrmurach/rspec-benchmark
I just implemented this feature to count mongo queries in my rspec suite in a small module using mongo Command Monitoring.
It can be used like this:
expect { code }.to change { finds("users") }.by(3)
expect { code }.to change { updates("contents") }.by(1)
expect { code }.not_to change { inserts }
Or:
MongoSpy.flush
# ..code..
expect(MongoSpy.queries).to match(
"find" => { "users" => 1, "contents" => 1 },
"update" => { "users" => 1 }
)
Here is the Gist (ready to copy) for the last up-to-date version: https://gist.github.com/jarthod/ab712e8a31798799841c5677cea3d1a0
And here is the current version:
module MongoSpy
module Helpers
%w(find delete insert update).each do |op|
define_method(op.pluralize) { |ns = nil|
ns ? MongoSpy.queries[op][ns] : MongoSpy.queries[op].values.sum
}
end
end
class << self
def queries
#queries ||= Hash.new { |h, k| h[k] = Hash.new(0) }
end
def flush
#queries = nil
end
def started(event)
op = event.command.keys.first # find, update, delete, createIndexes, etc.
ns = event.command[op] # collection name
return unless ns.is_a?(String)
queries[op][ns] += 1
end
def succeeded(_); end
def failed(_); end
end
end
Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, MongoSpy)
RSpec.configure do |config|
config.include MongoSpy::Helpers
end
What you're looking for is command monitoring. With Mongoid and the Ruby Driver, you can create a custom command monitoring class that you can use to subscribe to all commands made to the server.
I've adapted this from the Command Monitoring Guide for the Mongo Ruby Driver.
For this particular example, make sure that your Rails app has the log level set to debug. You can read more about the Rails logger here.
The first thing you want to do is define a subscriber class. This is the class that tells your application what to do when the Mongo::Client performs commands against the database. Here is the example class from the documentation:
class CommandLogSubscriber
include Mongo::Loggable
# called when a command is started
def started(event)
log_debug("#{prefix(event)} | STARTED | #{format_command(event.command)}")
end
# called when a command finishes successfully
def succeeded(event)
log_debug("#{prefix(event)} | SUCCEEDED | #{event.duration}s")
end
# called when a command terminates with a failure
def failed(event)
log_debug("#{prefix(event)} | FAILED | #{event.message} | #{event.duration}s")
end
private
def logger
Mongo::Logger.logger
end
def format_command(args)
begin
args.inspect
rescue Exception
'<Unable to inspect arguments>'
end
end
def format_message(message)
format("COMMAND | %s".freeze, message)
end
def prefix(event)
"#{event.address.to_s} | #{event.database_name}.#{event.command_name}"
end
end
(Make sure this class is auto-loaded in your Rails application.)
Next, you want to attach this subscriber to the client you use to perform commands.
subscriber = CommandLogSubscriber.new
Mongo::Monitoring::Global.subscribe(Mongo::Monitoring::COMMAND, subscriber)
# This is the name of the default client, but it's possible you've defined
# a client with a custom name in config/mongoid.yml
client = Mongoid::Clients.from_name('default')
client.subscribe( Mongo::Monitoring::COMMAND, subscriber)
Now, when Mongoid executes any commands against the database, those commands will be logged to your console.
# For example, if you have a model called Book
Book.create(title: "Narnia")
# => D, [2020-03-27T10:29:07.426209 #43656] DEBUG -- : COMMAND | localhost:27017 | mongoid_test_development.insert | STARTED | {"insert"=>"books", "ordered"=>true, "documents"=>[{"_id"=>BSON::ObjectId('5e7e0db3f8f498aa88b26e5d'), "title"=>"Narnia", "updated_at"=>2020-03-27 14:29:07.42239 UTC, "created_at"=>2020-03-27 14:29:07.42239 UTC}], "lsid"=>{"id"=><BSON::Binary:0x10600 type=uuid data=0xfff8a93b6c964acb...>}}
# => ...
You can modify the CommandLogSubscriber class to do something other than logging (such as incrementing a global counter).
From the health_check official site, we know that it can add a config.add_custom_check block in the config file:
https://github.com/ianheggie/health_check
# Add one or more custom checks that return a blank string if ok, or an error message if there is an error
config.add_custom_check do
CustomHealthCheck.perform_check # any code that returns blank on success and non blank string upon failure
end
# Add another custom check with a name, so you can call just specific custom checks. This can also be run using
# the standard 'custom' check.
# You can define multiple tests under the same name - they will be run one after the other.
config.add_custom_check('sometest') do
CustomHealthCheck.perform_another_check # any code that returns blank on success and non blank string upon failure
end
But about the CustomHealthCheck class, how to define it?
For okcomputer gem, it offers a way like this:
https://github.com/sportngin/okcomputer
# config/initializers/okcomputer.rb
class MyCustomCheck < OkComputer::Check
def check
if rand(10).even?
mark_message "Even is great!"
else
mark_failure
mark_message "We don't like odd numbers"
end
end
end
OkComputer::Registry.register "check_for_odds", MyCustomCheck.new
Didn't find the usage about health_check gem.
Update
I have tried:
Add these source in the config/initializers/health_check.rb file:
class CustomHealthCheck
def perform_check
if rand(10).even?
p "Even is great!"
else
p "We don't like odd numbers"
end
end
end
HealthCheck.setup do |config|
...
Run curl -v localhost:3000/health_check.json, got:
{"healthy":false,"message":"health_check failed: undefined method `perform_check' for CustomHealthCheck:Class"}%
Update 2
Edited source in config/initializers/health_check.rb:
class CustomHealthCheck
def self.perform_check
p 'OK'
end
end
HealthCheck.setup do |config|
...
Got:
{"healthy":false,"message":"health_check failed: OK"}%
Success is defined by returning an empty or blank string. Right now your perform_check always returns the string "OK" which will be seen as failure.
Try this to get a passing health check:
class CustomHealthCheck
def self.perform_check
everything_is_good = true # or call some method to do more elaborate checking
return everything_is_good ? "" : "We've got Problems"
end
end
Question:
I need to know the records' attributes that have been called inside a block (say I need something like the following):
def my_custom_method(&block)
some_method_that_starts_tracking
block.call
some_method_that_stops_tracking
puts some_method_that_returns_called_records_attributes
do_something_about(some_method_that_returns_called_records_attributes)
end
my_custom_method { somecodethatcallsauthorofbook1andemailandfirstnameofuser43 }
# this is the `puts` output above (just as an example)
# => {
# #<Book id:1...> => [:author],
# #<User id:43...> => [:email, :first_name]
# }
code inside the block can be anything
Specifically, I meant to track any instance of a subclass of ApplicationRecord, so it can be instance of any models like Book, User, etc...
Attempts:
From my understanding, this is similar to how rspec works when a method is expected to be called. That it somehow tracks any calls of that method. So, my initial attempt is to do something like the following (which does not yet fully work):
def my_custom_method(&block)
called_records_attributes = {}
ApplicationRecord.descendants.each do |klass|
klass.class_eval do
attribute_names.each do |attribute_name|
define_method(attribute_name) do
called_records_attributes[self] ||= []
called_records_attributes[self] << attribute_name
self[attribute_name]
end
end
end
end
block.call
# the above code will work but at this point, I don't know how to clean the methods that were defined above, as the above define_methods should only be temporary
puts called_records_attributes
end
my_custom_method { Book.find_by(id: 1).title }
# => {
# #<Book id: 1...> => ['title']
# }
the .descendants above probably is not a good idea because Rails use autoload if I'm not mistaken
as already said above in the comment, I do not know how to remove these "defined_methods" that are just supposed to be only temporary for the duration of this "block".
furthermore, my code above would probably have overriden the "actual" attribute getters of the models, if ever any has been already defined, which is bad.
Background:
I am writing a gem live_record which I am adding a new feature that will allow a developer to just simply write something like
<!-- app/views/application.html.erb -->
<body>
<%= live_record_sync { #book.some_custom_method_about_book } %>
</body>
... which will render #book.some_custom_method_about_book as-is on the page, but at the same time the live_record_sync wrapper method would take note of all the attributes that have been called inside that block (i.e. inside some_custom_method_about_book the #book.title is called), and then it sets these attributes as the block's own "dependencies", in which later when that specific book's attribute has been updated, I can already also update directly the HTML page of which this attribute is a "dependency" as like specified just above. I am aware that this is not an accurate solution, but I'd like to open up my chances by experimenting on this first.
-- Rails 5
Disclaimer: I believe this is just a mediocre solution, but hopefully helps anyone with the same problem.
I tried reading rspec source code, but because I couldn't easily comprehend what is happening under the hood, and that it occurred to me that rspec's (i.e.) expect(Book.first).to receive(:title) is different from what I really want because the methods there are already specified (i.e. :title), while what I want is to track ANY methods that are attributes, so because of these two reasons I skipped reading further, and attempted my own solution, which hopefully did somehow work; see below.
Note that I am using Thread local-storage here, so this code should be thread-safe (untested yet).
# lib/my_tracker.rb
class MyTracker
Thread.current[:my_tracker_current_tracked_records] = {}
attr_accessor :tracked_records
class << self
def add_to_tracked_records(record, attribute_name)
Thread.current[:my_tracker_current_tracked_records][{model: record.class.name.to_sym, record_id: record.id}] ||= []
Thread.current[:my_tracker_current_tracked_records][{model: record.class.name.to_sym, record_id: record.id}] << attribute_name
end
end
def initialize(block)
#block = block
end
def call_block_while_tracking_records
start_tracking
#block_evaluated_value = #block.call
#tracked_records = Thread.current[:my_tracker_current_tracked_records]
stop_tracking
end
def to_s
#block_evaluated_value
end
# because I am tracking record-attributes, and you might want to track a different object / method, then you'll need to write your own `prepend` extension (look for how to use `prepend` in ruby)
module ActiveRecordExtensions
def _read_attribute(attribute_name)
if Thread.current[:my_tracker_current_tracked_records] && !Thread.current[:my_tracker_is_tracking_locked] && self.class < ApplicationRecord
# I added this "lock" to prevent infinite loop inside `add_to_tracked_records` as I am calling the record.id there, which is then calling this _read_attribute, and then loops.
Thread.current[:my_tracker_is_tracking_locked] = true
::MyTracker.add_to_tracked_records(self, attribute_name)
Thread.current[:my_tracker_is_tracking_locked] = false
end
super(attribute_name)
end
end
module Helpers
def track_records(&block)
my_tracker = MyTracker.new(block)
my_tracker.call_block_while_tracking_records
my_tracker
end
end
private
def start_tracking
Thread.current[:my_tracker_current_tracked_records] = {}
end
def stop_tracking
Thread.current[:my_tracker_current_tracked_records] = nil
end
end
ActiveSupport.on_load(:active_record) do
prepend MyTracker::ActiveRecordExtensions
end
ActiveSupport.on_load(:action_view) do
include MyTracker::Helpers
end
ActiveSupport.on_load(:action_controller) do
include MyTracker::Helpers
end
Usage Example
some_controller.rb
book = Book.find_by(id: 1)
user = User.find_by(id: 43)
my_tracker = track_records do
book.title
if user.created_at == book.created_at
puts 'same date'
end
'thisisthelastlineofthisblockandthereforewillbereturned'
end
puts my_tracker.class
# => #<MyTracker ... >
puts my_tracker.tracked_records
# => {
# {model: :Book, record_id: 1} => ['title', 'created_at'],
# {model: :User, record_id: 43} => ['created_at']
# }
puts my_tracker
# => 'thisisthelastlineofthisblockandthereforewillbereturned'
# notice that `puts my_tracker` above prints out the block itself
# this is because I defined `.to_s` above.
# I need this `.to_s` so I can immediately print the block as-is in the views.
# see example below
some_view.html.erb
<%= track_records { current_user.email } %>
P.S. Maybe it's better that I wrap this up as a gem. If you're interested, let me know
I am introducing sunspot search into my project. I got a POC by just searching by the name field. When I introduced the description field and reindexed sold I get the following error.
** Invoke sunspot:reindex (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute sunspot:reindex
Skipping progress bar: for progress reporting, add gem 'progress_bar' to your Gemfile
rake aborted!
RSolr::Error::Http: RSolr::Error::Http - 400 Bad Request
Error: {'responseHeader'=>{'status'=>400,'QTime'=>18},'error'=>{'msg'=>'Illegal character ((CTRL-CHAR, code 11))
at [row,col {unknown-source}]: [42,1]','code'=>400}}
Request Data: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><add><doc><field name=\"id\">ItemsDesign 1322</field><field name=\"type\">ItemsDesign</field><field name=\"type\">ActiveRecord::Base</field><field name=\"class_name\">ItemsDesign</field><field name=\"name_text\">River City Clocks Musical Multi-Colored Quartz Cuckoo Clock</field><field name=\"description_text\">This colorful chalet style German quartz cuckoo clock accurately keeps time and plays 12 different melodies. Many colorful flowers are painted on the clock case and figures of a Saint Bernard and Alpine horn player are on each side of the clock dial. Two decorative pine cone weights are suspended beneath the clock case by two chains. The heart shaped pendulum continously swings back and forth.
On every
I assuming that the bad char is
that you can see at the bottom. that
is littered in a lot of the descriptions. I'm not even sure what char that is.
What can I do to get solr to ignore it or clean the data so that sold can handle it.
Thanks
Put the following in an initializer to automatically clean sunspot calls of any UTF8 control characters:
# config/initializers/sunspot.rb
module Sunspot
#
# DataExtractors present an internal API for the indexer to use to extract
# field values from models for indexing. They must implement the #value_for
# method, which takes an object and returns the value extracted from it.
#
module DataExtractor #:nodoc: all
#
# AttributeExtractors extract data by simply calling a method on the block.
#
class AttributeExtractor
def initialize(attribute_name)
#attribute_name = attribute_name
end
def value_for(object)
Filter.new( object.send(#attribute_name) ).value
end
end
#
# BlockExtractors extract data by evaluating a block in the context of the
# object instance, or if the block takes an argument, by passing the object
# as the argument to the block. Either way, the return value of the block is
# the value returned by the extractor.
#
class BlockExtractor
def initialize(&block)
#block = block
end
def value_for(object)
Filter.new( Util.instance_eval_or_call(object, &#block) ).value
end
end
#
# Constant data extractors simply return the same value for every object.
#
class Constant
def initialize(value)
#value = value
end
def value_for(object)
Filter.new(#value).value
end
end
#
# A Filter to allow easy value cleaning
#
class Filter
def initialize(value)
#value = value
end
def value
strip_control_characters #value
end
def strip_control_characters(value)
return value unless value.is_a? String
value.chars.inject("") do |str, char|
unless char.ascii_only? and (char.ord < 32 or char.ord == 127)
str << char
end
str
end
end
end
end
end
Source (Sunspot Github Issues): Sunspot Solr Reindexing failing due to illegal characters
I tried the solution #thekingoftruth proposed, however it did not solve the problem. Found an alternative version of the Filter class in the same github thread that he links to and that solved my problem.
The main difference was the i use nested models through HABTM relationships.
This is my search block in the model:
searchable do
text :name, :description, :excerpt
text :venue_name do
venue.name if venue.present?
end
text :artist_name do
artists.map { |a| a.name if a.present? } if artists.present?
end
end
Here is the initializer that worked for me:
(in: config/initializers/sunspot.rb)
module Sunspot
#
# DataExtractors present an internal API for the indexer to use to extract
# field values from models for indexing. They must implement the #value_for
# method, which takes an object and returns the value extracted from it.
#
module DataExtractor #:nodoc: all
#
# AttributeExtractors extract data by simply calling a method on the block.
#
class AttributeExtractor
def initialize(attribute_name)
#attribute_name = attribute_name
end
def value_for(object)
Filter.new( object.send(#attribute_name) ).value
end
end
#
# BlockExtractors extract data by evaluating a block in the context of the
# object instance, or if the block takes an argument, by passing the object
# as the argument to the block. Either way, the return value of the block is
# the value returned by the extractor.
#
class BlockExtractor
def initialize(&block)
#block = block
end
def value_for(object)
Filter.new( Util.instance_eval_or_call(object, &#block) ).value
end
end
#
# Constant data extractors simply return the same value for every object.
#
class Constant
def initialize(value)
#value = value
end
def value_for(object)
Filter.new(#value).value
end
end
#
# A Filter to allow easy value cleaning
#
class Filter
def initialize(value)
#value = value
end
def value
if #value.is_a? String
strip_control_characters_from_string #value
elsif #value.is_a? Array
#value.map { |v| strip_control_characters_from_string v }
elsif #value.is_a? Hash
#value.inject({}) do |hash, (k, v)|
hash.merge( strip_control_characters_from_string(k) => strip_control_characters_from_string(v) )
end
else
#value
end
end
def strip_control_characters_from_string(value)
return value unless value.is_a? String
value.chars.inject("") do |str, char|
unless char.ascii_only? && (char.ord < 32 || char.ord == 127)
str << char
end
str
end
end
end
end
end
You need to get rid of control characters from UTF8 while saving your content. Solr will not reindex this properly and throw this error.
http://en.wikipedia.org/wiki/UTF-8#Codepage_layout
You can use something like this:
name.gsub!(/\p{Cc}/, "")
edit:
If you want to override it globally I think it could be possible by overriding value_for_methods in AttributeExtractor and if needed BlockExtractor.
https://github.com/sunspot/sunspot/blob/master/sunspot/lib/sunspot/data_extractor.rb
I wasn't checking this.
If you manage to add some global patch, please let me know.
I had lately same issue.
Can you execute a block of Ruby code as a different OS user?
What I, ideally, want is something like this:
user("christoffer") do
# do something
end
Possible?
This code can do what you want. Error handling is up to you. ;-)
require 'etc'
def as_user(user, &block)
u = Etc.getpwnam(user)
Process.fork do
Process.uid = u.uid
block.call(user)
end
end
puts("caller PID = #{Process.pid}")
puts("caller UID = #{Process.uid}")
as_user "bmc" do |user|
puts("In block as #{user} (uid=#{Process.uid}), pid is #{Process.pid}")
end
Note, however, that it will require that you run Ruby as root, or as setuid-to-root, which has some severe security implications.
The accepted answer does change UID, but doing this alone can have surprising results when you create files or child processes. Try:
as_user 'bmc' do |user|
File.open('/tmp/out.txt', 'w')
end
You'll find that file was created as root, which isn't what one might expect.
The behavior is less predictable when running a command using the backtics. The results of the following probably aren't what one would expect:
as_user 'puppet' do
puts `whoami`
puts `id`
puts `whoami; id`
end
Testing on a Linux system, the first puts printed root. id printed the following:
uid=1052(chet) gid=0(root) euid=0(root) groups=0(root)
The final puts disagreed:
puppet
uid=1052(chet) gid=0(root) groups=0(root)
To get consistent behavior, be sure to set effective UID as well:
def as_user(user, &block)
u = Etc.getpwnam(user)
Process.fork do
Process.uid = Process.euid = u.uid
block.call(user)
end
end
It can be useful to get a value back from the child process. Adding a little IPC fun gives:
require 'etc'
def as_user(user, &block)
u = (user.is_a? Integer) ? Etc.getpwuid(user) : Etc.getpwnam(user)
reader, writer = IO.pipe
Process.fork do
# the child process won't need to read from the pipe
reader.close
# use primary group ID of target user
# This needs to be done first as we won't have
# permission to change our group after changing EUID
Process.gid = Process.egid = u.gid
# set real and effective UIDs to target user
Process.uid = Process.euid = u.uid
# get the result and write it to the IPC pipe
result = block.call(user)
Marshal.dump(result, writer)
writer.close
# prevent shutdown hooks from running
Process.exit!(true)
end
# back to reality... we won't be writing anything
writer.close
# block until there's data to read
result = Marshal.load(reader)
# done with that!
reader.close
# return block result
result
end
val = as_user 'chet' do
`whoami` + `id` + `whoami; id`
end
puts "back from wonderland: #{val}"