I am building an online e-commerce store, and I am trying to use rails with action cable to update a product from being out of stock to in-stock at a certain date time e.g 12:00:00 2020-02-19.
The idea is as soon as the time is reached, I want to push a Websocket that the product is now available.
I have tried a few solutions such as:
Thread.new do
while true do
if **SOMETIME** == Time.now
ActionCable.server.broadcast "product_channel",content: "product-in-stock"
end
end
end
The main issue with this approach is that it creates another thread and makes rails unresponsive. Furthermore, if this value is set for say 1 week from now I do not want every user who queries the endpoint to create a brand-new thread running like this.
You have two option use sidekiq jobs or use whenever job gem
https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs
Whenever allow you to set specific day and time, check the documentation for more info
https://github.com/javan/whenever
I want to query a rate-limited external API and save the information to the database. I want to get as close to the limit as possible so the information is up to date. My initial idea was to use sidekiq and create a job that was in a loop that checks for how many times the external API had been called in the past 1 second. This does not work well. I cant find any examples online. Most of the results were geared towards making an API in Ruby on Rails.
#config/initializers/data_poller.rb
require "redis"
redis = Redis.new
external_api = ExternalAPI::TestAPI1.new()
APIWorker.perform_async(external_api)
# app/workers/api_worker.rb
class APIWorker
include Sidekiq::Worker
def perform(external_api)
while true
if can_call_external_api
external_api.get_data
end
end
end
end
Since you are using redis, one solution would be to create a cache variable due to expire in one second.
Before the API call, check if the variable exists.
If it does, do nothing.
If it does not, create it again with one second expiry and do the call (or first do the call and then populate the variable)
I have a "Play" button in my app that checks a stock value from an API and creates a Position object that holds that value. This action uses Resque to make a background job using Resque and Redis in the following way:
Controller - stock_controller.rb:
def start_tracking
#stock = Stock.find(params[:id])
Resque.enqueue(StockChecker, #stock.id)
redirect_to :back
end
Worker:
class StockChecker
#queue = :stock_checker_queue
def self.perform(stock_id)
stock = Stock.find_by(id: stock_id)
stock.start_tracking_position
end
end
Model - stock.rb:
def start_tracking_position
// A Position instance that holds the stock value is created
end
I now want this to happen every 15 minutes for every Stock object. I looked at the scheduling section in the Ruby Toolbox website and have a hard time deciding what fits to my needs and how to start implementing it.
My concern is that my app will create tons of Position objects so I need something that is simple, uses Resque and can withstand this type of object creating without overloading the app.
What gem should I use and what is the simplest way to make my Resque Job happen every 15 minutes when the start_tracking action happens on a Stock object?
I've found resque scheduler to be useful: https://github.com/resque/resque-scheduler.
Configure up the schedule.yml for 15 mins
The biggest issue I found was ensuring it's running post releases etc. In the end I set up God to shutdown and restart
In terms of load. Not sure I follow, the schedulers will trigger events but the load is determined by the number of workers you have and how you decide to implement the creation. You can set the priority of the queues, and workers for the queue, but I guess if you don't process them in a timely way you get a backlog, is that acceptable. ? Normally you would run them of a separate server, this minimising impact to front end
I am writing an app which will sit between a vendors proprietary inventory management system and their Shopify shop. The app will periodically update Shopify from new data generated by the inventory management system. It will also provide end-points for Shopify webhooks.
I am currently doing something like this (pseudo-ruby with much stuff omitted):
def update_product_with_proxy(product_proxy)
product_proxy.variant_proxies.dirty.each do |variant_proxy|
update_variant_with_proxy(variant_proxy)
end
if product_proxy.dirty_proxy
shopify_product = ShopifyAPI::Product.find(product_proxy.shopify_id)
shopify_product.update_attributes({some attributes here})
end
end
Elsewhere:
def update_variant_with_proxy(variant_proxy)
shopify_variant = ShopifyAPI::Variant.find(variant_proxy.shopify_id)
shopify_variant.update_attributes({some attributes here})
end
This seems terribly inefficient as I have to fetch each updated ShopifyAPI::Product and ShopifyAPI::Variant before I can update them (I have their id's cached locally). It takes about 25 minutes for an update cycle updating 24 products each with 16 variants. Rails spends less than 2 seconds updating my product/variant proxies. The other 99% of the time is spent talking to Shopify. I must be doing something wrong.
Given that I know the id of the remote object is there a way to updated it directly without having to fetch it first?
cheers,
-tomek
First things first: You can update variants through their parent product. Once you've grabbed the product it'll have the variant info with it so you can edit them, save, and the changes will be persisted in a single API call. That'll save you some time.
Second: You can create an object locally using the gem, give it an id, and then call save to initiate the PUT request without first fetching the object from Shopify. Something like this should do the trick:
product = ShopifyAPI::Product.new(:id => 1, :title => "My new title")
product.save
Putting those two things together should give you what you want: The ability to update a product's variants in a single API call.
Note: For future reference, the shopify_api gem is built on Active Resource, so anything you can do with that library you can do with the gem.
Rails has a nice set of filters (before_validation, before_create, after_save, etc) as well as support for observers, but I'm faced with a situation in which relying on a filter or observer is far too computationally expensive. I need an alternative.
The problem: I'm logging web server hits to a large number of pages. What I need is a trigger that will perform an action (say, send an email) when a given page has been viewed more than X times. Due to the huge number of pages and hits, using a filter or observer will result in a lot of wasted time because, 99% of the time, the condition it tests will be false. The email does not have to be sent out right away (i.e. a 5-10 minute delay is acceptable).
What I am instead considering is implementing some kind of process that sweeps the database every 5 minutes or so and checks to see which pages have been hit more than X times, recording that state in a new DB table, then sending out a corresponding email. It's not exactly elegant, but it will work.
Does anyone else have a better idea?
Rake tasks are nice! But you will end up writing more custom code for each background job you add. Check out the Delayed Job plugin http://blog.leetsoft.com/2008/2/17/delayed-job-dj
DJ is an asynchronous priority queue that relies on one simple database table. According to the DJ website you can create a job using Delayed::Job.enqueue() method shown below.
class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
Delayed::Job.enqueue( NewsletterJob.new("blah blah", Customers.find(:all).collect(&:email)) )
I was once part of a team that wrote a custom ad server, which has the same requirements: monitor the number of hits per document, and do something once they reach a certain threshold. This server was going to be powering an existing very large site with a lot of traffic, and scalability was a real concern. My company hired two Doubleclick consultants to pick their brains.
Their opinion was: The fastest way to persist any information is to write it in a custom Apache log directive. So we built a site where every time someone would hit a document (ad, page, all the same), the server that handled the request would write a SQL statement to the log: "INSERT INTO impressions (timestamp, page, ip, etc) VALUES (x, 'path/to/doc', y, etc);" -- all output dynamically with data from the webserver. Every 5 minutes, we would gather these files from the web servers, and then dump them all in the master database one at a time. Then, at our leisure, we could parse that data to do anything we well pleased with it.
Depending on your exact requirements and deployment setup, you could do something similar. The computational requirement to check if you're past a certain threshold is still probably even smaller (guessing here) than executing the SQL to increment a value or insert a row. You could get rid of both bits of overhead by logging hits (special format or not), and then periodically gather them, parse them, input them to the database, and do whatever you want with them.
When saving your Hit model, update a redundant column in your Page model that stores a running total of hits, this costs you 2 extra queries, so maybe each hit takes twice as long to process, but you can decide if you need to send the email with a simple if.
Your original solution isn't bad either.
I have to write something here so that stackoverflow code-highlights the first line.
class ApplicationController < ActionController::Base
before_filter :increment_fancy_counter
private
def increment_fancy_counter
# somehow increment the counter here
end
end
# lib/tasks/fancy_counter.rake
namespace :fancy_counter do
task :process do
# somehow process the counter here
end
end
Have a cron job run rake fancy_counter:process however often you want it to run.