I'm not sure if its the best idea to create a background job that scans the database all the time for updated product prices, especially when in my application I expect to have 100,000s if not millions of products being observed for lower prices.
I have two models, Product and Price. a Product can have many different Prices. So the way I thought it should be done was:
Subscribe to a Product.
Create background job that scans database every hour for lower prices.
Notify users by email.
I'm not sure about #2 and if that's the best way for it to be done. What would you guys suggest?
If your Rails app is adding new Price objects to the Products in its database, why not have an ActiveRecord callback (after_create in this instance) every time a new Price object is added?
That ActiveRecord callback could start a delayed job to email the user about the new, lower price.
If it were me, I would use database triggers rather than a background job sweeper.
Here's an intro to triggers:
http://www.mysqltutorial.org/mysql-triggers.aspx
Related
I'm trying out InfluxDB to know if my usecase fits.
My app generates a bunch of events like product created, product deleted, product purchased, payment recieved, category created etc. Each event has some other properties such as who created the product or what the payment method was...
I want to know how many products were purchased or howmany payments were done using a specific payment method or howmany payments were done for a day or till now or with in a time specified. Same for all the events like payment, shipping etc. I am yet to understand the concept of TSDB. Every example I see has some value that is varying, ie; temperature 23,30,23,35,24,33 and so on. In my app each event has a value of 1, since each event contibute to one unit of that event.
Is InfluxDB a good choice for this usecase ? If yes, How would I model my data for use cases like this ?
You could use TSDB for your e-commerce analysis but it might be better to try data warehouse instead, especially when your data volume grows rapidly.
TSDB are best used with time series data plus time series analysis. For example, if you care when the shopping cart is filled and emptied but don't care that much about what is in the shopping cart.
Your use case is more like OLAP and you could check out ClickHouse.
App running on: ruby 2.0, rails 4 and postgresql
1.The multiple tables story - How it works now:
A project has many users, as members.
Also project has many posts, when a post is created a notification is created for each project user.
Let's say if Project A has 100 users, we'll have 100 notifications in database, this will load the database with a lot of duplicates.
But a user can delete its own notification, can view it, we can update his notification with user specific data. And we'll use a rake task to remove the notifications that are older then a specific time interval.
2. The multiple db queries - What we want to do:
There is an idea of creating only one notification for an activity and use many to many relation with a table called notifications_users where we'll keep information about a notification if it was read by a current user, if this user removed this notification from his notifications tab, etc..
The downside of this I think it will be multiple db queries as when we'll need to find something about a notification and user we'll have to look up for the notification_users table to get the information needed.
Also, by building the relation this way it will be harder to clean up the database from old notifications as we will not know if this notification was read or not by some user.
Thank you.
The option (1.) seems pretty reasonable and if you can keep the notification model thin — like user_id, activity_id, timestamps, and maybe a couple more flags, then wouldn’t expect unreasonable performance penalties, since it looks like a pretty usual relational model and I’d expect the database to easily handle it.
If you keep you notification expiration tight, this means that the notification should not grow, or if it does, for example when user just abandoned the account, I would look for specific solutions for the issues, as the appear.
At Assembla.com we do notification with email and this is working pretty well for us, I’d say. So, maybe at some point, you may want to consider this option too.
It seems to me that the (2.) just doesn’t fulfil the business needs, and if you’re not considering them than it’s probably not worth considering this option either.
Before analyzing scenarios given in your question I would like to request you to clear one point which is a bit confusing in your question. The statements given below are confusing.
1) Also project has many posts, when a post is created a notification is created for each project user.
2) Let's say if Project A has 100 users, we'll have 100 notifications in database, this will load the database with a lot of duplicates.
Statement no. 1 describes that when a post is created; a notification is sent for each user. So suppose there are 100 users and 100 posts then the table which has notifications data will have 10000 rows (100*100).
Statement no. 2 describes that if there are 100 users then there would be 100 notifications in the database that means the notification table will have 100 rows.
Of the points given above which one should I consider.
I would like to setup a simple notifications system for my web app that is written in Ruby on Rails. My website is a simple auction site, but each auction has 2 stages: a 5 day "no minimum bid" stage where anyone can bid and a 3 day "$1 minimum bid" stage where only the top 10% of the bidders from the previous stage can bid.
I want to notify my users when the auction goes from the first stage to the second. What is the best way to do this?
I've thought of two design options:
I don't know RoR has this, and whether this is an efficient system (because I want to build a system that will have thousands of users participating in hundreds of auctions - whether it gets that big or not is irrelevant, this is sort of a side project for me to learn to write high quality, scalable RoR) but basically, when a user bids on an auction, they become an observer of some sort on the auction. Then, somehow at the 5 day mark, all the listeners are notified. I don't know how that trigger happens though in RoR (is there some way to say "trigger after 5 days from creation" or some such thing?). To summarize, I would like to use the Observer pattern.
The key issue for me is trying to figure out how to trigger after a set period of time. The best way to do it, I think, would be for each auction to have some set state variable called "auction state". After a certain period of time, that would transition, and at that point every observer on the auction would get notified.
There is a rake task that runs everyday at some set time each day. It goes through each auction and if any auction is on day 6, the observers of that auction are notified. This seems like a pretty bad way of going about it - I like the idea of having a state-ful auction.
Thoughts?
Thanks!
Ringo
You can take a look at https://github.com/pluginaweek/state_machine, specifically transitions callbacks. Alternatively you can also use the standard active record callbacks http://api.rubyonrails.org/classes/ActiveRecord/Callbacks.html
Have a look at https://github.com/bvandenbos/resque-scheduler, if you haven't heard yet, resque is a gem for creating background processing, having queues in redis. This gem resque-scheduler, lets you specify when to run a task/job.
For example in your model you would have an after_create to schedule the task for changing the bid state.
after_create :shedule_state_change
private
def record_signup
Resque.enqueue_in(5.days, ChangeState)
end
And then you could notify all the bidders for that bid inside that task or create an observer for it.
I would appreciate some advice on how to structure a database for the following scenario:
I'm using Ruby on Rails, and so I have the following tables:
Products
Salespeople
Stores
Products are manufactured in batches, so each product item has a Batch code, so I think I will also need a table of batches, which refers to a product type.
Batch
In the real world, Salespeople take Product items (from a specific Batch) and in due course issue it to a Store. Importantly, Batches are large, and may be spread across many Salespeople, and subsequently, Stores.
At some future date, I would like to run the following reports:
Show all Batches of a Product issued to a specific Store.
Show all Batches held by a Salesperson (i.e. not yet sold).
Now, I'm assuming I need to build a table of Transactions, something like,
Transaction
salesperson_id
batch_id (through which the product can be determined)
store_id
typeOfTransaction (whether the Salesperson has obtained some stock, or sold some stock)
quantity
By dynamically running through a table of Transaction records, I can could derive the above reports. However, this seems inefficient and, over time, increasingly slow.
My question is: what is the best way to keep track of transactions like this, preferably without requiring dynamic processing of all transactions to derive total items from a batch given to a given store.
I don't believe I can just keep a central record of stock as Product comes in Batches, and Batches are distributed by Salepeople across Stores.
Thank you.
My question is: what is the best way to keep track of transactions like this, preferably without requiring dynamic processing of all transactions to derive total items from a batch given to a given store.
I don't believe I can just keep a central record of stock as Product comes in Batches, and Batches are distributed by Salepeople across Stores.
Believe it. :-)
In my experience, the only correct way to store this kind of stuff, is to break it down to something akin to T-leger accounting, i.e. debit/credit with a chart of accounts. It requires dynamic processing to derive totals as you've found out, but anything short of that will lead to tricky queries when dealing with reports and audit trails.
You can speed things up significantly, by maintaining partial or complete aggregate balances using triggers (e.g. monthly stock movements per store). This will reduce the number of rows you need to sum when running larger queries. Which of these you'll want to maintain will depend on your app and your reporting requirements.
The situation:
The magazine accepts submissions. Once you submit, an editor will schedule your submission for review. Once it has been reviewed, you are no longer allowed to edit it.
So, I have submission in various states. "Draft", "queued", "reviewed", etc. Most of the switches into these various states are triggered by some action, e.g., a submission becomes queued when an editor schedules it. Easy peasey. However, the switch into the "reviewed" state is not triggered by any action, it just happens after a certain datetime has passed.
I have two thoughts on how to accomplish this:
Run a daily/hourly cron job to check up on all the queued submissions and switch them to reviewed if necessary. I dislike this because
I would prefer it to be hourly, so that I can edit my submission up to three hours before a meeting starts
Hourly cron jobs cost money on Heroku, and this application will either never make money or won't make money for months and months to come
Somehow construct a before_load ActiveRecord callback, that will perform some logic on submissions each time they are loaded. "Queued? No? Nevermind. Otherwise, switch it to 'Reviewed' if its meeting is less than three hours away."
I wanted to get people's input on the second idea.
Is that an atrociously smelly way to accomplish this?
If so, can you suggest an awesomer third way?
If 'no' to both of the above, can you give tips on how to perform such logic each time a record is loaded from the database? I would need to always perform some logic before doing a select from the submissions table (which is gearing up to be the most-queried table in the app...)
If there's no good way to accomplish Option Two (or, I hope!, Option Three), I will resort to Option One with a daily cron job. Being able to edit up to a day before a meeting will just have to suffice.
Maybe using after_find, although your performance will sort of suck, same goes if you do something as crazy as before_load, performance would suck, that said money might be more important than performance, if that is so, I would go with the after_find.