I'm using Ruby 2.2.1 and Rails 4.2.0
I have a method on my Rails app to warehouse data from a web service. After retrieving and formatting the data, I'm using assign_attributes to update the model before doing some other logic and saving. The problem is that assigning the variables is crazy slow! Assigning 3 properties (a string and two booleans) is taking between 1 and 3 seconds. My full application needs to assign 30, and it's taking upwards of a minute for each object to be updated.
Sample code:
...
# #trips_hash is a hash of { trip_numbers => trip_details<Hash> }
# #bi_trips is an array of <Trip> (ActiveRecord::Base) objects from the database
#trips_hash.each do |trip_number, trip_details|
trip = #bi_trips.select { |t| t.number == trip_number }.first
...
time_started = Time.now # For performance profiling
trip.assign_attributes( stage: 'foo', all_intl: true, active: false )
p "Done in #{(Time.now - time_started).round(2)} s."
end
...
Here are results for the above code:
"Done in 0.0 s."
"Done in 1.71 s."
"Done in 2.09 s."
"Done in 3.36 s."
"Done in 1.45 s."
"Done in 1.99 s."
"Done in 1.63 s."
"Done in 0.59 s."
"Done in 1.61 s."
"Done in 1.56 s."
"Done in 2.25 s."
"Done in 1.42 s."
"Done in 1.53 s."
"Done in 1.61 s."
Am I going crazy? It seems like it shouldn't take 1-3 seconds to assign 3 properties of an object. I get similar results breaking it into
trip.stage = 'foo'
trip.all_intl = false
trip.active = true
Related
I have a simple Ruby method meant to throttle some execution.
MAX_REQUESTS = 60
# per
TIME_WINDOW = 1.minute
def throttle
cache_key = "#{request.ip}_count"
count = Rails.cache.fetch(cache_key, expires_in: TIME_WINDOW.to_i) { 0 }
if count.to_i >= MAX_REQUESTS
render json: { message: 'Too many requests.' }, status: 429
return
end
Rails.cache.increment(cache_key)
true
end
After some testing I've found that cache_key never invalidates.
I investigated with binding.pry and found the issue:
[35] pry(#<Refinery::ApiReferences::Admin::ApiHostsController>)> Rails.cache.write(cache_key, count += 1, expires_in: 60, raw: true)
=> true
[36] pry(#<Refinery::ApiReferences::Admin::ApiHostsController>)> Rails.cache.send(:read_entry, cache_key, {})
=> #<ActiveSupport::Cache::Entry:0x007fff1e34c978 #created_at=1495736935.0091069, #expires_in=60.0, #value=11>
[37] pry(#<Refinery::ApiReferences::Admin::ApiHostsController>)> Rails.cache.increment(cache_key)
=> 12
[38] pry(#<Refinery::ApiReferences::Admin::ApiHostsController>)> Rails.cache.send(:read_entry, cache_key, {})
=> #<ActiveSupport::Cache::Entry:0x007fff1ee105a8 #created_at=1495736965.540865, #expires_in=nil, #value=12>
So, increment is wiping out the expires_in value and changing the created_at, regular writes will do the same thing.
How do I prevent this? I just want to update the value for a given cache key.
UPDATE
Per suggestion I tried:
MAX_REQUESTS = 60
# per
TIME_WINDOW = 1.minute
def throttle
cache_key = "#{request.ip}_count"
count = Rails.cache.fetch(cache_key, expires_in: TIME_WINDOW.to_i, raw: true) { 0 }
if count.to_i >= MAX_REQUESTS
render json: { message: 'Too many requests.' }, status: 429
return
end
Rails.cache.increment(cache_key)
true
end
No effect. Cache does not expire.
Here's a "solution," I won't mark it correct because surely this isn't necessary?
MAX_REQUESTS = 60
# per
TIME_WINDOW = 1.minute
def throttle
count_cache_key = "#{request.ip}_count"
window_cache_key = "#{request.ip}_window"
window = Rails.cache.fetch(window_cache_key) { (Time.zone.now + TIME_WINDOW).to_i }
if Time.zone.now.to_i >= window
Rails.cache.write(window_cache_key, (Time.zone.now + TIME_WINDOW).to_i)
Rails.cache.write(count_cache_key, 1)
end
count = Rails.cache.read(count_cache_key) || 0
if count.to_i >= MAX_REQUESTS
render json: { message: 'Too many requests.' }, status: 429
return
end
Rails.cache.write(count_cache_key, count + 1)
true
end
Incrementing a raw value (with raw: true option) in Rails cache works exactly the way you desire, i.e. it updates only the value, not the expiration time. However, when debugging this, you cannot rely on the output of read_entry very much as this does not correspond fully with the raw value stored in cache, because the cache store does not give back the expiry time when storing just the raw value.
That is why, normally (without the raw) option, Rails does not store just the raw value, but it creates a cache Entry object which, besides the value, holds additional data, such as the expiry time. Then it serializes this object and saves it to the cache store. Upon reading the value back, it de-serializes the object and still has access to all info, including the expiry time.
However, as you cannot increment a serialized object, you need to store a raw value instead, i.e. use the raw: true option. This makes Rails store directly the value and pass the expiry time as param to the cache store write method (without the possibility to read it back from the store).
So, to sum up, you must use raw: true when caching a value for incrementing and the expiry time will be normally preserved in the cache store. See the following test (done on the mem_cache_store store):
# cache_test.rb
cache_key = "key"
puts "setting..."
Rails.cache.fetch(cache_key, expires_in: 3.seconds, raw: true) { 1 }
puts "#{Time.now} cached value: #{Rails.cache.read(cache_key)}"
sleep(2)
puts "#{Time.now} still cached: #{Rails.cache.read(cache_key)}"
puts "#{Time.now} incrementing..."
Rails.cache.increment(cache_key)
puts "#{Time.now} incremented value: #{Rails.cache.read(cache_key)}"
sleep(1)
puts "#{Time.now} gone!: #{Rails.cache.read(cache_key).inspect}"
When running this, you'll get:
$ rails runner cache_test.rb
Running via Spring preloader in process 31666
setting...
2017-05-25 22:15:26 +0200 cached value: 1
2017-05-25 22:15:28 +0200 still cached: 1
2017-05-25 22:15:28 +0200 incrementing...
2017-05-25 22:15:28 +0200 incremented value: 2
2017-05-25 22:15:29 +0200 gone!: nil
As you can see, the value has been incremented without resetting the expiry time.
Update: I set up a minimal test for you code, though not run through a real controller but only as a script. I made only 4 small changes to the throttle code in your OP:
lowered the time window
changed render to a simple puts
used only a single key as if requests came from a single IP address
print the incremented value
The script:
# chache_test2.rb
MAX_REQUESTS = 60
# per
#TIME_WINDOW = 1.minute
TIME_WINDOW = 3.seconds
def throttle
#cache_key = "#{request.ip}_count"
cache_key = "127.0.0.1_count"
count = Rails.cache.fetch(cache_key, expires_in: TIME_WINDOW.to_i, raw: true) { 0 }
if count.to_i >= MAX_REQUESTS
#render json: { message: 'Too many requests.' }, status: 429
puts "too many requests"
return
end
puts Rails.cache.increment(cache_key)
true
end
62.times do |i|
throttle
end
sleep(3)
throttle
The run prints the following:
$ rails runner cache_test2.rb
Running via Spring preloader in process 32589
2017-05-26 06:11:26 +0200 1
2017-05-26 06:11:26 +0200 2
2017-05-26 06:11:26 +0200 3
2017-05-26 06:11:26 +0200 4
...
2017-05-26 06:11:26 +0200 58
2017-05-26 06:11:26 +0200 59
2017-05-26 06:11:26 +0200 60
2017-05-26 06:11:26 +0200 too many requests
2017-05-26 06:11:26 +0200 too many requests
2017-05-26 06:11:29 +0200 1
Perhaps you don't have caching configured in development at all? I recommend testing this in the memcached store, which is the most preferred cache store in production environment. In development, you need to explicitly switch it on:
# config/environemnts/development.rb
config.cache_store = :mem_cache_store
Also, if you are running a recent Rails 5.x version, you may need to run the rails dev:cache command which creates the tmp/caching-dev.txt file that is used in the development config to actually enable caching in development env.
Is there a way in RSpec to show every single test duration and not just the total suite duration?
Now we have
Finished in 7 minutes 31 seconds (files took 4.71 seconds to load)
but I'd like to have something like
User accesses home and
he can sign up (finished in 1.30 seconds)
he can visit profile (finished in 3 seconds)
.
.
.
Finished in 7 minutes 31 seconds (files took 4.71 seconds to load)
You can use rspec --profile N, which would show you the top N slowest examples.
For a quick solution see #maximf's answer. For an alternative solution, you could write your own rspec formatter, which would give you greater control over what you are measuring.
For example, extnding rspec's base text formatter:
RSpec::Support.require_rpec_core "formatters/base_text_formatter"
module RSpec::Core::Formatters
class TimeFormatter < BaseTextFormatter
Formatters.register self, :example_started, :example_passed
attr_accessor :example_start, :longest_example, :longest_time
def initialize(output)
#longest_time = 0
super(output)
end
def example_started(example)
#example_start = Time.now
super(example)
end
def example_passed(example)
time_taken = Time.now - #example_start
output.puts "Finished #{example.example.full_description} and took #{Helpers.format_duration(time_taken)}"
if #time_taken > #longest_time
#longest_example = example
#longest_time = time_taken
end
super(example)
end
def dump_summary(summary)
super(summary)
output.puts
output.puts "The longest example was #{#longest_example.example.full_Description} at #{Helpers.format_duration(#longest_time)}"
end
end
end
Note that this will only log times on passed examples, but you could add an example_failed failed to do similar, it also only works with RSpec 3. This is based on my work on my own formatter: https://github.com/yule/dots-formatter
Instead of doing rspec --profile Neverytime we run specs (as #maximf said), we can add it to our RSpec configuration:
RSpec.configure do |config|
config.profile_examples = 10
end
I am using Rails 4, Ruby 2.1 with PostgreSQL.
I have a database field called duration which is an interval data type.
When pulling out the data in this column it returns in the format of hh:mm:ss, e.g. 01:30:00.
I am trying to figure out a way to display this as 1 hour, 30 minutes.
Other examples:
02:00:00 to 2 hours
02:15:00 to 2 hours, 15 minutes
02:01:00 to 2 hours, 1 minute
Just use duration + inspect
seconds = 86400 + 3600 + 15
ActiveSupport::Duration.build(seconds).inspect
=> "1 day, 1 hour, and 15.0 seconds"
Or a it can be a little be customized
ActiveSupport::Duration.build(seconds).parts.map do |key, value|
[value.to_i, key].join
end.join(' ')
=> "1days 1hours 15seconds"
P.S.
You can get seconds with
1.day.to_i
=> 86400
Time can be parsed only in ISO8601 format
ActiveSupport::Duration.parse("PT2H15M").inspect
=> "2 hours and 15 minutes"
I would start with something like this:
def duration_of_interval_in_words(interval)
hours, minutes, seconds = interval.split(':').map(&:to_i)
[].tap do |parts|
parts << "#{hours} hour".pluralize(hours) unless hours.zero?
parts << "#{minutes} minute".pluralize(minutes) unless minutes.zero?
parts << "#{seconds} hour".pluralize(seconds) unless seconds.zero?
end.join(', ')
end
duration_of_interval_in_words('02:00:00')
# => '2 hours'
duration_of_interval_in_words('02:01:00')
# => '2 hours, 1 minute'
duration_of_interval_in_words('02:15:00')
# => '2 hours, 15 minutes'
See also
ActionView::Helpers::DateHelper distance_of_time_in_words (and related)
e.g.
0 <-> 29 secs # => less than a minute
30 secs <-> 1 min, 29 secs # => 1 minute
1 min, 30 secs <-> 44 mins, 29 secs # => [2..44] minutes
... etc
https://apidock.com/rails/ActionView/Helpers/DateHelper/distance_of_time_in_words
Perhaps not appropriate to include in a model validation error? (which is my use case)
You can try following method to display such as:
minutes_to_human(45) # 45 minutes
minutes_to_human(120) # 2 hours
minutes_to_human(75) # 2.5 hours
minutes_to_human(75) # 1.15 hours
def minutes_to_human(minutes)
result = {}
hours = minutes / 60
result[:hours] = hours if hours.positive?
result[:minutes] = ((minutes * 60) - (hours * 60 * 60)) / 60 if minutes % 60 != 0
result[:minutes] /= 60.0 if result.key?(:hours) && result.key?(:minutes)
return I18n.t('helper.minutes_to_human.hours_minutes', time: (result[:hours] + result[:minutes]).round(2)) if result.key?(:hours) && result.key?(:minutes)
return I18n.t('helper.minutes_to_human.hours', count: result[:hours]) if result.key?(:hours)
return I18n.t('helper.minutes_to_human.minutes', count: result[:minutes].round) if result.key?(:minutes)
''
end
Translations:
en:
helper:
minutes_to_human:
minutes:
zero: '%{count} minute'
one: '%{count} minute'
other: '%{count} minutes'
hours:
one: '%{count} hour'
other: '%{count} hours'
hours_minutes: '%{time} hours'
This worked for me:
irb(main):030:0> def humanized_duration(seconds)
irb(main):031:1> ActiveSupport::Duration.build(seconds).parts.except(:seconds).reduce("") do |output, (key, val)|
irb(main):032:2* output+= "#{val}#{key.to_s.first} "
irb(main):033:2> end.strip
irb(main):034:1> end
=> :humanized_duration
irb(main):035:0> humanized_duration(920)
=> "15m"
irb(main):036:0> humanized_duration(3920)
=> "1h 5m"
irb(main):037:0> humanized_duration(6920)
=> "1h 55m"
irb(main):038:0> humanized_duration(10800)
=> "3h"
You can change the format you want the resulting string to be inside the reduce. I like the 'h' and 'm' for hours and minutes. And I excluded the seconds from the duration parts since that wasn't important for my usage of it.
Here is a locale-aware helper method which builds upon MasonMc's answer.
# app/helpers/date_time_helper.rb
module DateTimeHelper
def humanized_duration(duration)
ActiveSupport::Duration.build(duration).parts.except(:seconds).collect do |key, val|
t(:"datetime.distance_in_words.x_#{key}", count: val)
end.join(', ')
end
end
You can also replace join(', ') with to_sentence if it reads better, or get fancy and allow passing a locale, like distance_of_time_in_words.
Gotcha
Rather counter-intuitively, x_hours is absent from Rails' default locale file because distance_of_time_in_words doesn't use it.
You'll need to add it yourself, even if using the rails-i18n gem.
# config/locales/en.datetime.yml
en:
datetime:
distance_in_words:
x_hours:
one: "one hour"
other: "%{count} hours"
Here's the output:
humanized_duration(100)
# => '1 minute'
humanized_duration(12.34.hours)
# => '12 hours, 20 minutes'
humanized_duration(42.hours)
# => '1 day, 18 hours, 25 minutes'
I find that duration.inspect serve the purpose pretty well.
> 10.years.inspect
=> "10 years"
I am calling puts statements like this:
puts "something"
Within my Rails Resque workers. What is the best way to read this output real time? tail development.log?
Thanks!
You could always try to use the Logger:
Logger.info "Something"
# Or
Logger.debug "Something"
# Or
Logger.error "Something"
These will definitely show up in your logs. :)
I recommend using Log4r:
In config/environments/*.rb
#format of message in logger
format = Log4r::PatternFormatter.new(:pattern => "%d - [%l]:\t%m.")
# log configuration
configlog = {
"filename" => "log/your_name.log",
"max_backups" => 28, # 7days * 4 files of 6 hours
"maxtime" => 21600, # 6 hours in sec
"maxsize" => 10485760, # 10MB in bytes
"trunc" => false
}
rolling = Log4r::RollingFileOutputter.new("rolling",configlog)
rolling.formatter = format
config.logger = Log4r::Logger.new("your_name.log")
config.logger.add(rolling)
Then in your code:
Logger.info "output"
Logger.debug "output"
In your_name.log you will see:
2013-08-07 10:00:47 - [INFO]: output
2013-08-07 10:00:47 - [DEBUG]: output
I have a cron job on Heroku that needs to run for 8 hours a day. (Once an hour, eight times, each day.).
These do work as expected:
(all code in /lib/tasks/cron.rake)
if Time.now.hour == 6
puts "Ok it's 6am and I printed"
puts "6:00 PST put is done."
end
if Time.now.hour == 7
puts "Ok it's 7am and I printed"
puts "7:00 PST put is done."
end
if Time.now.hour == 8
puts "I printed at 8am"
puts "8:00 PST put is done."
end
if Time.now.hour == 9
puts "9:00 PST open put"
puts "9:00 PST put is done."
end
This code, however, doesn't work:
if Time.now.hour (6..14)
puts "I did the rake job"
end
I've also tried the following variants. Didn't work.
if Time.now.hour == (6..14)
if Time.now.hour = (6..14)
if Time.now.hour == 6..14
if Time.now.hour = 6..14
Anyone know how I can put a range in a Heroku cron job? Listing lots of jobs by hour just seems wrong.
You want to see if that range includes the hour:
if (6..14).include?(Time.now.hour)
puts "I did the rake job"
end
What you want is
if (6..14) === Time.now.hour
# Run command
end