Once the script is daemonized then the logger can't write to the file anymore. So how and when should I initialise the log?
require 'rubygems'
require 'daemons'
require 'logging'
def create_new_logger
logger = Logging.logger['trend-analyzer']
logger.add_appenders(
Logging.appenders.rolling_file('./logs/trend-analyzer.log'),
Logging.appenders.stdout
)
logger.level = :debug
return logger
end
logger = create_new_logger
#this log message gets written to the log file
logger.debug Time.new
Daemons.run_proc('ForestPress', :log_dir => '.logs', :backtrace => true) do
running_as_daemon = true
#this log message does NOT get written to the log file
logger.debug Time.new
loop do
#this log message does NOT get written to the log file
logger.info Time.new
sleep 5
end
end
EDIT
I notice the current path changes from where I executed the script to /. Could this be why I can't log messages?
EDIT 2
I now save the original path before becoming a daemon and then use Dir.chdir to set the path to the original path. I can then open the file directly and write to it. However the logging gem can't write to it still.
Here is how it started working
require 'rubygems'
require 'daemons'
require 'logging'
def create_new_logger
logger = Logging.logger['trend-analyzer']
logger.add_appenders(
Logging.appenders.rolling_file('./logs/trend-analyzer.log'),
Logging.appenders.stdout
)
logger.level = :debug
return logger
end
current_dir = Dir.pwd
Daemons.run_proc('ForestPress', :log_dir => '.logs', :backtrace => true) do
Dir.chdir(current_dir)
logger = create_new_logger
loop do
puts Dir.pwd
logger.debug Time.new
sleep 5
end
end
Related
I'm using Rails 5. I have this file, config/environment_variables.yml
development:
COINBASE_KEY: devkey
COINBASE_SECRET: devsecret
test:
COINBASE_KEY: testkey
COINBASE_SECRET: testsecret
production:
COINBASE_KEY: prodkey
COINBASE_SECRET: prodsecret
I load it with the file, config/initializers/environment_variables.rb
module EnvironmentVariables
class Application < Rails::Application
config.before_configuration do
env_file = Rails.root.join("config", 'environment_variables.yml').to_s
if File.exists?(env_file)
YAML.load_file(env_file)[Rails.env].each do |key, value|
ENV[key.to_s] = value
end # end YAML.load_file
end # end if File.exists?
end # end config.before_configuration
end # end class
end # end module
but when I run my test using
rails test test/services/crypto_currency_service_test.rb
The test variables aren't loading -- rather those from the dev environment are loading. Below is my test file
require 'coinbase/wallet'
require 'minitest/mock'
class CryptoCurrencyServiceTest < ActiveSupport::TestCase
test 'sell' do
last_transaction = MyTransaction.new({
:transaction_type => "buy",
:amount_in_usd => "100",
:btc_price_in_usd => "3000"
})
puts "env: #{ENV['COINBASE_KEY']}"
#client = Coinbase::Wallet::Client.new(api_key: ENV['COINBASE_KEY'], api_secret: ENV['COINBASE_SECRET'])
How do I get the test variables to load by default when I run tests?
Edit: Here's the config/environments/test.rb file, which I haven't (consciously) changed ...
Rails.application.configure do
# Settings specified here will take precedence over those in config/application.rb.
# The test environment is used exclusively to run your application's
# test suite. You never need to work with it otherwise. Remember that
# your test database is "scratch space" for the test suite and is wiped
# and recreated between test runs. Don't rely on the data there!
config.cache_classes = true
# Do not eager load code on boot. This avoids loading your whole application
# just for the purpose of running a single test. If you are using a tool that
# preloads Rails for running tests, you may have to set it to true.
config.eager_load = false
# Configure public file server for tests with Cache-Control for performance.
config.public_file_server.enabled = true
config.public_file_server.headers = {
'Cache-Control' => 'public, max-age=3600'
}
# Show full error reports and disable caching.
config.consider_all_requests_local = true
config.action_controller.perform_caching = false
# Raise exceptions instead of rendering exception templates.
config.action_dispatch.show_exceptions = false
# Disable request forgery protection in test environment.
config.action_controller.allow_forgery_protection = false
config.action_mailer.perform_caching = false
# Tell Action Mailer not to deliver emails to the real world.
# The :test delivery method accumulates sent emails in the
# ActionMailer::Base.deliveries array.
config.action_mailer.delivery_method = :test
# Print deprecation notices to the stderr.
config.active_support.deprecation = :stderr
# Raises error for missing translations
# config.action_view.raise_on_missing_translations = true
end
I would not recommend to write custom code for this. There are existing solutions for setting up environment variables. See dotenv-rails for example. It allows you to put common variables into .env file. Just add gem 'dotenv-rails' to your Gemfile and put common variables into .env file:
# .env
COINBASE_KEY: devkey
COINBASE_SECRET: devsecret
If you need environment-specific variables it allows you to have separate files for this: .env.development, .env.test and .env.production:
#.env.test
COINBASE_KEY: testkey
COINBASE_SECRET: testsecret
#.env.production
COINBASE_KEY: prodkey
COINBASE_SECRET: prodsecret
From the comments in your original post, I suggest checking if config.before_configuration block is not at fault. Might be the case that the rails environment is loaded after that block has run and so you get Rails.env == 'test' when you print that out inside a test, but in the configuration it takes keys from default (development) environment.
Might I suggest moving this part
env_file = Rails.root.join("config", 'environment_variables.yml').to_s
if File.exists?(env_file)
YAML.load_file(env_file)[Rails.env].each do |key, value|
ENV[key.to_s] = value
end # end YAML.load_file
end # end if File.exists?
out in an intializer and then check the environment variables. Might fix the problem. (because initializers surely should respect the environment)
UPDATE: From the documentation it seems that before_configuration block is the VERY first block config part to run, so Rails.env is probably not yet set at that point.
I am running a cucumber script by executing Cucumber::Cli::Main.new(args).execute!
But Ruby moves to the next line and starts reading from the file. But the file is empty as cucumber takes time to process the Cucumber::Cli::Main.new(args).execute!
How to make execution stop until Cucumber has finished executing the script and finished populating the file with HTML. Thank you
Here is the Link for Source Code: https://github.com/cucumber/cucumber-ruby/blob/master/lib/cucumber/cli/main.rb
require 'cucumber'
require 'tempfile'
require 'securerandom'
filename = "#{SecureRandom.urlsafe_base64}"
file = Tempfile.new(filename)
filepath = "#{file.path}"
features = "features/login.feature"
args = features.split.concat %w(--format html --out)
args << "#{filepath}.html"
begin
Cucumber::Cli::Main.new(args).execute!
#value = file.read
ensure
file.close
file.unlink
end
EDIT:
When Cucumber::Cli::Main.new(args).execute! finishes executing it throws a SystemExit Exception with a status 0.
Exit code is 0 when execution went fine
Cucumber always throws an SystemExit exception when it finishes.
Here is the Link for Source Code for Cucumber: https://github.com/cucumber/cucumber-ruby/blob/master/lib/cucumber/cli/main.rb
How to handle the SystemExit Exception in Rails, so that it doesn't skip the next lines of execution.
def run
filename = "#{SecureRandom.urlsafe_base64}"
file = Tempfile.new(filename)
filepath = "#{file.path}"
features = "features/login.feature"
args = features.split.concat %w(-f html -o)
args << "#{filepath}.html"
Cucumber::Cli::Main.new(args).execute! # throws SystemExit Exception Status 0
#output = file.read
file.close
file.unlink
# More Code Below
# # # # # # # # #
end
Two choices I see here, one is to catch the error and then read the file, such as:
begin
Cucumber::Cli::Main.new(args).execute!
rescue SystemExit => e
if e.status == 0
#value = file.read
else
raise e
end
ensure
file.close
file.unlink
end
The other option is to make a runner class that inherits from the CLI and override exit_ok.
class Runner < Cucumber::Cli::Main
def exit_ok
#NOOP
end
end
begin
Runner.new(args).execute!
#value = file.read
ensure
file.close
file.unlink
end
I've got a Rails 4 app on a Puma server with Resque/Resque-Scheduler running background jobs. What I'd like to know is how I merge the log output of my two Resque workers into my server log, or, of that is not possible, how I can view the log output of my Resque workers. Currently I have not been able to figure out how to view the log output for the workers, so I have no idea what's happening under the hood. I found this blogpost, which suggests adding the following likes to my resque.rake file:
task "resque:setup" => :environment do
Resque.before_fork = Proc.new {
ActiveRecord::Base.establish_connection
# Open the new separate log file
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
# Activate file synchronization
logfile.sync = true
# Create a new buffered logger
Resque.logger = ActiveSupport::BufferedLogger.new(logfile)
Resque.logger.level = Logger::INFO
Resque.logger.info "Resque Logger Initialized!"
}
end
That didn't work. I also tried the suggestion in the comments, which was to replace Resque.logger = ActiveSupport::BufferedLogger.new(logfile) with Resque.logger = ActiveSupport::Logger.new(logfile), however that didn't work either. With the second option, I still get a NoMethodError: undefined method 'logger=' for Resque:Module error when I try to boot up a worker.
Here is my current resque.rake file:
require 'resque/tasks'
require 'resque_scheduler/tasks'
namespace :resque do
puts "Loading Rails environment for Resque"
task :setup => :environment do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
require 'postman'
end
end
I've looked at the Resque docs on logging, but am not sure how to use what's there as I admittedly don't know very much about logging in Rails. I haven't had any luck finding other useful resources on the subject.
How I fix it, it is not perfect but just works.
my environment: rails 5.0.1, resque: 1.26.0
at the first time, I set the Resque.logger and Resque.logger.level in config/initializers/resque.rb as most docs suggest:
# config/initializers/resque.rb
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
then in the job, I output log by Resque.logger.info:
# update_xxx_job.rb
class UpdateXxxJob
def self.perform
Resque.logger.info 'Job starts'
...
end
end
it doesn't work, I can see nothing in log/resque.log.
then someone said should set the logfile sync always, no buffer, so I update the config/initializers/resque.rb according a question from stackoverflow:
# config/initializers/resque.rb
logfile = File.open(File.join(Rails.root, 'log', 'resque.log'), 'a')
logfile.sync = true
Resque.logger = ActiveSupport::Logger.new(logfile)
Resque.logger.level = Logger::INFO
still doesn't work.
I also tried config resque logger in lib/tasks/resque.rake:
# lib/tasks/resque.rake
require 'resque'
require 'resque/tasks'
require 'resque/scheduler/tasks'
require 'resque-scheduler'
require 'resque/scheduler/server'
namespace :resque do
task setup: :environment do
Resque.schedule = YAML.load_file(Rails.root + "config/resque_scheduler_#{Rails.env}.yml")
Resque.redis.namespace = "xxx_#{Rails.env}"
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::INFO
end
end
doesn't work.
finally, I decide to move the logger configuration from initializer to the job, so the job now looks like:
# update_xxx_job.rb
class UpdateXxxJob
def self.perform
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
Resque.logger.info 'Job starts'
...
end
end
then I can get what I want in the log/resque.log.
you can try it.
I've had the same problem while setting up mine. Here's what I did:
Resque.before_fork do
# Your code here
end
It seems before_fork accepts a block as an argument rather than assigning a block to it.
I faced the same issue so that I check the source of resque and finally I needed to do the followings at initialization process:
define log formatter.
then define logger with log-file path.
set any log level.
Here the example is at my config/initializers/resque.rb in rails case:
...
Resque.logger = Logger.new("#{Rails.root}/log/resque.log")
Resque.logger.level = Logger::DEBUG
Resque.logger.formatter = ::Logger::Formatter.new # This is important
Resque default logger formatter is set here and its definitions is here. That apparently just ignores the output...
I was annoyed with having to cut way through ANSI sequences in a log on production server (log/production.log), so I added config.colorize_logging = false to config/environments/production.rb. But now when I run a console (bin/rails c), the output is not colorized as well. Why is it so? Is there a way to make logger use ANSI sequences when outputting to screen, and not use them when logging to a file?
UPD What I was able to figure out. When rails app starts, it creates logger to log into a file:
Rails.logger ||= config.logger || begin
path = config.paths["log"].first
unless File.exist? File.dirname path
FileUtils.mkdir_p File.dirname path
end
f = File.open path, 'a'
f.binmode
f.sync = config.autoflush_log # if true make sure every write flushes
logger = ActiveSupport::Logger.new f
logger.formatter = config.log_formatter
logger = ActiveSupport::TaggedLogging.new(logger)
logger
rescue StandardError
logger = ActiveSupport::TaggedLogging.new(ActiveSupport::Logger.new(STDERR))
logger.level = ActiveSupport::Logger::WARN
logger.warn(
"Rails Error: Unable to access log file. Please ensure that #{path} exists and is writable " +
"(ie, make it writable for user and group: chmod 0664 #{path}). " +
"The log level has been raised to WARN and the output directed to STDERR until the problem is fixed."
)
logger
end
And then attaches to it another logger to output messages to STDOUT:
def log_to_stdout
wrapped_app # touch the app so the logger is set up
console = ActiveSupport::Logger.new($stdout)
console.formatter = Rails.logger.formatter
console.level = Rails.logger.level
Rails.logger.extend(ActiveSupport::Logger.broadcast(console))
end
For some reason, breaking with byebug keyword at ActiveSupport::Logger#initialize never succeed when I ran ./bin/rails c.
UPD Okay, the culprit was spring, console (or should I say activerecord) creates its logger here:
console do |app|
require "active_record/railties/console_sandbox" if app.sandbox?
require "active_record/base"
console = ActiveSupport::Logger.new(STDERR)
Rails.logger.extend ActiveSupport::Logger.broadcast console
end
One way to switch colorized_logging on in console would be to set it explicitly to true as follows:
$ bin/rails c
:001 > Rails.application.config.colorize_logging
=> false
:002 > Rails.application.config.colorize_logging = true
=> true
:003 > Rails.application.config.colorize_logging
=> true
There might be a way to set this automatically every time the console is loaded by customizing the console with a .irbrc file
I'm trying to log the progress of my sideqik worker using tail -f log/development.log in development and heroku logs in production.
However, everything inside the worker and everything called by the worker does not get logged. In the code below, only TEST 1 gets logged.
How can I log everything inside the worker and the classes the worker calls?
# app/controllers/TasksController.rb
def import_data
Rails.logger.info "TEST 1" # shows up in development.log
DataImportWorker.perform_async
render "done"
end
# app/workers/DataImportWorker.rb
class DataImportWorker
include Sidekiq::Worker
def perform
Rails.logger.info "TEST 2" # does not show up in development.log
importer = Importer.new
importer.import_data
end
end
# app/controllers/services/Importer.rb
class Importer
def import_data
Rails.logger.info "TEST 3" # does not show up in development.log
end
end
Update
I still don't understand why Rails.logger.info or Sidekiq.logger.info don't log into the log stream. Got it working by replacing Rails.logger.info with puts.
There is a Sidekiq.logger and simply logger reference that you can use within your workers. The default should be to STDOUT and you should just direct your output in production to the log file path of your choice.
It works in rails 6:
# config/initializers/sidekiq.rb
Rails.logger = Sidekiq.logger
ActiveRecord::Base.logger = Sidekiq.logger
#migu, have you tried the below command in the config/initializer.rb ?
Rails.logger = Sidekiq::Logging.logger
I've found this solution here, it seems to work well.
Sidekiq uses the Ruby Logger class with default Log Level as INFO, and its settings are independent from Rails.
You may set the Sidekiq Log Level for the Logger used by Sidekiq in config/initializers/sidekiq.rb:
Sidekiq.configure_server do |config|
config.logger.level = Rails.logger.level
end