I need to fetch transaction data from a third party API, and save the records periodically (once a month). Here is an example:
class BalanceTransaction::Update
include Service
attr_reader :offset, :transactions
def initialize(offset)
#offset = offset
#transactions = fetch_transactions
end
def call
ActiveRecord::Base.transaction do
transactions.auto_paging_each do |txn|
type = txn[:type]
source = txn[:source]
case type
when 'charge', 'adjustment'
invoice = find_invoice(source)
ac_id = invoice.account.id
update_attrs(id: invoice.id, account_id: ac_id, type:'invoice', attrs: txn)
when 'refund'
refund = find_refund(source)
ac_id = refund.invoice.account.id
update_attrs(id: refund.id, account_id: ac_id, type:'refund', attrs: txn)
end
end
end
true
end
private
def find_invoice(source)
Invoice.find_by!(stripe_charge_id: source)
end
def find_refund(source)
Refund.find_by!(stripe_refund_id: source)
end
def update_attrs(id:, account_id:, type:, attrs:)
BalanceTransaction.create(
account_id: account_id,
stripe_transaction_id: attrs[:id],
gross_amount: attrs[:amount],
net_amount: attrs[:net],
fee_amount: attrs[:fee],
currency: attrs[:currency],
transactionable_id: id,
transactionable_type: type)
end
def fetch_transactions
external_card_balance.all(limit: offset)
rescue *EXTERNAL_CARD_ERRORS => e
ExternalCardErrorHandler.new(e).handle
end
def external_card_balance
Stripe::BalanceTransaction
end
end
I wonder how to bulk insert idempotently from the last time. Should I check created_at and delete them if I find data created after the offset? Could you give me some advice?
Does the transaction have unique id's or any field that could be made unique? Maybe you could use validates_uniqueness_of to avoid saving transactions you already fetched.
Related
I'm working on the Meetup Api.
I would like to save some conferences from the API into my database.
The saving conferences depend of the parameters passing into the view to the controller :
<%= link_to 'See conferences', conferences_path(:title => "ParisRb")%> |
Then I call the good method to look for the good conferences (comparing to the params) among all the one received from the api.
I would like the methods to be very generic and to be able to save any conferences not only 'ParisRb'.
So I modify all my methods in this goal but there is one I can not modify, I don't know how.
This is my whole code. The one I'd like to modify is self.conferences_filter(data) wich is supposed to receive the params from the controller instead of 'ParisRb'. But I know that passing parameters from the controller to the model is not a good practice. So any idea is welcome :)
lib/api_meetup.rb
class ApiMeetup
BASE_URI = "https://api.meetup.com"
def events(urlname)
HTTParty.get(BASE_URI + "/#{urlname}/events")
end
end
conferences_controller.rb
def index
#call to the API
response = ApiMeetup.new.events(params[:title])
api_data = JSON.parse(response.body)
filtered_conferences = Conference.conferences_filter(api_data)
conferences = Conference.save_conferences_from_api(filtered_conferences)
#conferences = conferences.current_conferences
end
conference.rb
#Keep only requested conferences
def self.conferences_filter(data)
requested_conferences = []
data.each do |event|
if event["name"].include?('ParisRb') #This should receive params[:title] instead of 'ParisRb'
requested_conferences << event
end
end
requested_conferences
end
#Save requested conferences from the Meetup API
def self.save_conferences_from_api(conferences)
# data = data_from_api
conferences.each do |line|
conference = self.new
conference.title = line['name']
conference.date = format_date(line['time'])
conference.url = line['link']
if conference.valid?
conference.save
end
end
self.all
end
That's was actually quite obvious.
I just needed to pass to argument to my method :
filtered_conferences = Conference.conferences_filter(api_data, params[:title])
#Keep only requested conferences
def self.conferences_filter(data, title)
requested_conferences = []
data.each do |event|
if event["name"].include?(title)
requested_conferences << event
end
end
requested_conferences
end
I understand what Ruby self means, and I was trying to solve certain challenges on Tealeaf: http://www.gotealeaf.com/books/oo_workbook/read/intermediate_quiz_1
Here is the actual problem:
Snippet 1:
class BankAccount
def initialize(starting_balance)
#balance = starting_balance
end
# balance method can be replaced with attr_reader :balance
def balance
#balance
end
def positive_balance?
balance >= 0 #calls the balance getter instance level method (defined above)
end
end
Now for Snippet 1, running this code:
bank_account = BankAccount.new(3000)
puts bank_account.positive_balance?
prints true on the console, whereas for snippet 2:
Snippet 2:
class InvoiceEntry
attr_reader :product_name
def initialize(product_name, number_purchased)
#quantity = number_purchased
#product_name = product_name
end
# these are attr_accessor :quantity methods
# quantity method can be replaced for attr_reader :quantity
def quantity
#quantity
end
# quantity=(q) method can be replaced for attr_writer :quantity
def quantity=(q)
#quantity = q
end
def update_quantity(updated_count)
# quantity=(q) method doesn't get called
quantity = updated_count if updated_count >= 0
end
end
Now for snippet 2, on running this code:
ie = InvoiceEntry.new('Water Bottle', 2)
ie.update_quantity(20)
puts ie.quantity #> returns 2
Why is this not updating the value?
Why is it working for the first case while not for the second?
You are assigning to quantity the local variable.
If you want to assign to the instance variable (via your def quantity= function) you need to do
self.quantity = updated_count if updated_count >= 0
Essentially, you're making a function call (quantity=) on self.
In snippet 1, balance is a pure function call because there is no assignment going on.
I have a List model below, it has a has_and_belongs_to_many association with recipients. The purpose of the method make_recipient_lists is to save a parsed csv of numbers(initial parameter) in this format [[num1],[num2],[num3]...].
add_recipients work by finding existing recipients then adding them to the list or creating new recipients.
This whole process works well for small amount, 20k of numbers in 28minutes. However, the greater the number, the longer it takes exponentially, 70k took 14hours. Probably because it was checking for duplicates to a cached current_lists.
Question is, is there any way to make this faster? I am probably approaching this problem wrong. Thanks!
class List < ActiveRecord::Base
#other methods above
def make_recipient_lists(numbers,options)
rejected_numbers = []
account = self.user.account
#caching recipients
current_recipients = self.recipients
numbers.each do |num|
add_recipient(num[0], current_recipients)
end
end
def add_recipient(num, current_recipients)
account = self.user.account
recipient = current_recipients.where(number:num, account_id: account.id).first
recipient ||= current_recipients.create!(number:num, account_id: account.id)
recipient
end
end
You could do something like this. I have not tested this, but you get the idea.
def make_recipient_lists(numbers, options)
rejected_numbers = []
account = self.user.account
existing_numbers = self.recipients.where(number: numbers, account_id: account.id).map(&:number)
new_records = (numbers - existing_numbers).map {|n| {number: n, account_id: account.id, list_id: self.id} }
Recipient.create new_records
end
I think, you should use rails active_record query interface. you can use method find_or_create method for this: It will make your queries faster. change your method like this, and check the time difference:
def make_recipient_lists(numbers,options)
rejected_numbers = []
account = self.user.account
#caching recipients
current_recipients = self.recipients
numbers.each do |num|
self.recipients.find_or_create_by(number: num, account_id: account.id)
end
end
Hope it will help. Thanks.
I have a module which returns timesheet records. When there is a scope parameter provided the items method scopes the timesheets so it only returns certain timesheets. How can I use the result of that query in another query? It is about using team_users in the super.where(user_id: team_users) query.
module Collections
class TimesheetCollection < Collection
module TeamScope
def items
if params[:scope].present?
team_users = User.from_team(#manager)
super.where(user_id: team_users)
else
super
end
end
end
attr_reader :ability, :params
def initialize(ability, params, manager)
#ability = ability
#params = params
#manager = manager
extend TeamScope
end
def items
Timesheet.unscoped
end
def paginated
extend Pagination
self
end
end
end
for active record you can pass in an array of ids/values. For your query you could try to do something like this:
team_users = User.from_team(#manager)
super.where(user_id: team_users.collect{|u| u.id})
I'm migrating my old blog posts into my new Rails blog, and I want their updated_at attribute to match the corresponding value on my old blog (not the date they were migrated into my new Rails blog).
How can I do this? When I set updated_at manually it gets overridden by the before_save callback.
Note: This question is only valid for Rails < 3.2.11. Newer versions of Rails allow you to manually set timestamps without them being overwritten.
If it's a one time thing you can turn record_timestamps on or off.
ActiveRecord::Base.record_timestamps = false
#set timestamps manually
ActiveRecord::Base.record_timestamps = true
When I ran into this issue with my app, I searched around for a bit and this seemed like it made the most sense to me. It's an initializer that I can call where I need to:
module ActiveRecord
class Base
def update_record_without_timestamping
class << self
def record_timestamps; false; end
end
save!
class << self
def record_timestamps; super ; end
end
end
end
end
As of recent versions of Rails (3.2.11 as per iGELs comment) you can set the updated_at property in code and the change will be honoured when saving.
I assume rails is keeping track of 'dirty' properties that have been manually changed and not overwriting on save.
> note = Note.last
Note Load (1.4ms) SELECT "notes".* FROM "notes" ORDER BY "notes"."id" DESC LIMIT 1
=> #<Note id: 39, content: "A wee note", created_at: "2015-06-09 11:06:01", updated_at: "2015-06-09 11:06:01">
> note.updated_at = 2.years.ago
=> Sun, 07 Jul 2013 21:20:47 UTC +00:00
> note.save
(0.4ms) BEGIN
(0.8ms) UPDATE "notes" SET "updated_at" = '2013-07-07 21:20:47.972990' WHERE "notes"."id" = 39
(0.8ms) COMMIT
=> true
> note
=> #<Note id: 39, content: "A wee note", created_at: "2015-06-09 11:06:01", updated_at: "2013-07-07 21:20:47">
So short answer, workarounds are not needed any longer in recent versions of rails.
I see two ways to accomplish this easily:
touch (Rails >=5)
In Rails 5 you can use the touch method and give a named parameter time like described in the documentation of touch
foo.touch(time: old_timestamp)
update_column (Rails >=4)
If you want it in Rails 4 and lower or want to avoid all callbacks you could use one of the update_column or update_columns methods which bypass all safe or touch callbacks and validations
foo.update_column(updated_at, old_timestamp)
or
foo.update_columns(updated_at: old_timestamp)
I took Andy's answer and modified it to accept blocks:
module ActiveRecord
class Base
def without_timestamping
class << self
def record_timestamps; false; end
end
yield
class << self
remove_method :record_timestamps
end
end
end
end
This is riffing off of Andy Gaskell's answer:
class ActiveRecord::Base
class_inheritable_writer :record_timestamps
def do_without_changing_timestamps
self.class.record_timestamps = false
yield
ensure
self.class.record_timestamps = true
end
end
The solution is to temporarily set ActiveRecord::Base.record_timestamps to false:
ActiveRecord::Base.record_timestamps = false
# Make whatever changes you want to the timestamps here
ActiveRecord::Base.record_timestamps = true
If you want a somewhat more robust solution, you may want to try something like what mrm suggested:
module ActiveRecord
class Base
def self.without_timestamping
timestamping = self.record_timestamps
begin
self.record_timestamps = false
yield
ensure
self.record_timestamps = timestamping
end
end
end
end
Then you can easily make changes to models without their timestamps being automatically updated:
ActiveRecord::Base.without_timestamping do
foo = Foo.first
bar = Bar.first
foo.updated_at = 1.month.ago
bar.updated_at = foo.updated_at + 1.week
foo.save!
bar.save!
end
Or, if you only want to update records from a specific class without timestamping:
module ActiveRecord
class Base
# Don't delete Rail's ActiveRecord::Base#inherited method
self.singleton_class.send(:alias_method, :__inherited__, :inherited)
def self.inherited(subclass)
__inherited__
# Adding class methods to `subclass`
class << subclass
def without_timestamping
# Temporarily override record_timestamps for this class
class << self
def record_timestamps; false; end
end
yield
ensure
class << self
remove_method :record_timestamps
end
end
end
end
end
end
E.g:
Foo.without_timestamping do
foo = Foo.first
bar = Bar.new(foo: foo)
foo.updated_at = 1.month.ago
foo.save! # Timestamps not automatically updated
bar.save! # Timestamps updated normally
end
Or you could use an approach similar to what Venkat D. suggested, which works on a per-instance basis:
module ActiveRecord
class Base
def without_timestamping
class << self
def record_timestamps; false; end
end
yield
ensure
class << self
remove_method :record_timestamps
end
end
end
end
E.g:
foo = Foo.first
foo.without_timestamping do
foo2 = Foo.new(parent: foo)
foo.updated_at = 1.month.ago
foo.save! # Timestamps not automatically updated
foo2.save! # Timestamps updated normally
end