Scenario: "I have 500 different types of posts (So emails are being sent various scheduled days exactly at 7 o'clock in the morning). A user signs up to receive 1 or more of those types of posts every day via email (keep in mind, this could mean 5,000+ emails per day)."
Looking around SO, I keep seeing the "Whenever" gem come up as a solution but some are saying that it could cause spikes in the server. I also read some posts on delayed_job but some say that it freezes which I do not want to happen. I googled some more & still could not find anything close to being specific to my question.
What would be the best way to handle this, given my scenario & why?
Thank you in advance.
Use:
http://rubygems.org/gems/resque
in conjunction with:
https://rubygems.org/gems/resque_mailer
The best part is that you won't have to change the code, you just can send them like you did with ActionMailer!
Related
For a few years now I observe a strange behaviour, most likely triggered by enduser's email clients using my webapp (Ruby on Rails system, doesn't matter tho).
I am running a mid sized business and send out thousands of mails to my customers each month who buy leads from me.
The emails include two links, one to buy the lead and the other to give feedback. There is a dynamic part in both URLs which is a UUID, example:
offer/968ec0c1-e105-4c70-95b2-fd0c799b58f3
and
feedback/968ec0c1-e105-4c70-95b2-fd0c799b58f3
Every now and then, my webapp gets accessed at the same time on both links (which makes me confident it is not the user since it is the very same second they get accessed) but with different dynamic parts in the url, so i see in my logs
offer/NGVjZjA0YT
and
feedback/NGVjZjA0YT
It is always a random string with a length of 10 chars.
So this is not a big deal since it happens only 1-2 times per week and as far as I can tell no user is really affected by this, but still I wonder what's behind this. Did any of you experience a similar thing?
Maybe an email client wants to crawl / load a preview, seeing an uuid pattern in the url and changes it because whatever?!
I disabled link click tracking in the email sending provider (sendgrid), just as a side note. So they won't / shouldn't replace the email links. Also experienced this when sending links via AWS SES.
Im just curious. Any ideas or experiences? Thanks in advance & have a great day!
I'm a new junior developer joining to this awesome community. I'm developing my first big personal project, and I'm stuck with this specific part.
I would like to build a feed notification system like Facebook with the following features:
Track different models and relationships, for example: new badges earned, new comments in subscribed models, new posts by followed users, new comments on my posts, new likes on my posts...
Group the activities, for example, instead of have 400 activities for each likes in my post, has just one notification that says "User X and 399 more likes your post"
Be possible to mark notifications as readed to don't see them again, at least you explore past notifications.
Scalability, good perfomance, and possible integration in the future in an APP developed for example with Iconic framework.
Push notifications are optional, it's ok if the user need to refresh the page to see the new notifications.
So for that, I have readed a lot of. I have watched some Railcast Videos, followed tutorials, but still I'm not really sure how to begin.
I have considered the following methods:
Use public_activity gem, adding a new a new field "readed" to the migration. And thinking how to manage grouped activites. But I have seen a lot of complains about perfomance. I'm expecting to have around 50000 users in my website in the first month (I already have the users), with peaks of 500-1000 users online. So maybe this is not the best way to go... as I would have a lot of activities, a lot of "notifications" and a lot of users.
Use a system like https://getstream.io/ because they also have integrations available for RoR and Ruby. The main concern here is about pricing, because checking it, if I'm not wrong, with that number of users, with around 10 notifications per user per day, I would be paying probably more than 200$ month, and always keep growing as the users grow.
Build my own system, maybe using Redis. But maybe this would be too complex and require a lot of time for a good, efficient and working code.
So still, considered these option, I don't know which one is best for me, or if it's another possibilities.
If someone have asked before these questions, please let me know your thoughts and what you think is the correct way to go.
Thank you !! :)
We've got one 'superuser' account that we use to send messages to selected JIDs. Lets say we've selected ones we want to send a message to, and we got ourselves a huge array of user JIDs (20k at this point). We've got a deamon running in the background sending one message at a time to each user, stopping for a minute after sending 2000 of messages (2500/minute limit). We are using xmpp4r as a client that handles sending messages. Every user has same #xmpp.address. <body> is the same in every message.
Our tigase logs (because thats what we're using), show that the messages did actually hit the jabber server and were sent to appropriate users, one at a time.
The issue we're experiencing is that although everything seemed fine, only a part of users actually got the message. (for example, at one point, considering 100 first messages sent - 1..20 and 91..100 got delivered, the middle 70 did not get delivered at all), we improved couple of things in the meantime but this still might be a clue.
We tried creating an array of 10000 duplicated JIDS (jids of couple of users were duplicated thousands of times), and every single message got delivered (and in the right order).
We already spent a couple of days trying different scenarios and are starting to run out of ideas what might be going wrong.
Got any idea's what we might have missed?
I am form Tigase team. First thing, I recommend to use our online forums as this is where we normally respond to questions. We may not see questions posted here.
Anyway...
There are some details not included in your post.
What do you connect and how? Over standard XMPP connection, over Bosh, something else?
What do you mean by "duplicated JIDs"? How did you duplicate JIDs?
Are all the users for which a message is sent online during the test?
If you can see message in Tigase logs, you should also see what happens to it. Was it submitted to network socket for delivery to a client?
What kind of HW did you use? Is there a chance that the server was overloaded and simply dropped some messages? Seems unlikely if you talk about 100 messages and 70 of them not delivered.
How do you actually know that a message was not delivered and are you sure the client/user was connected at the time?
SO sorry if this is a duplicate, I tried searching for this but wasnt sure what search terms to use and didnt really find anything.
I have a Ruby on Rails app that will be used to send text messages out to users that contain a link to a multiple choice question probably using clickatell. The questions have a date and time associated with them. I want to make the ruby on rails app automatically send those SMS messages to the users' phones on those specified dates.
I don't really know how one would go about doing this. Can anyone point me in the general direction of a a way to schedule events like this in ruby on rails. I don't need an exact solution, maybe if someone could just clarify what exactly this is called so I can find some resources on line.
thanks
It seems the sending out your questions is not reoccuring? In this case I would not do this via a cronjob. I would do this via: https://github.com/bvandenbos/resque-scheduler
So whenever a question is getting scheduled you just add it to the delayed queue and resque-scheduler handles moving them on the correct working queue when its time has come.
This way you don't have to worry about some sort of polling cronjob, this will be done by resque-scheduler automatically. You also get asynchronous handling of sending out the SMSes via resque for free. So if you have to send lots and lots of SMS you can run them in parallel.
So it would go like this:
when a question is saved, you queue a message on the delayed queue in the future for sending out the question
when the date comes up, the message is moved onto 'ready to send'-queue, which is in charge of gathering all the users the question needs to be sent to.
for each of those users you create another message on the 'ready to send'-queue
the 'ready to send'-queue will then send out the actual SMSes
You then can run many workers on the 'ready to send'-queue and have the SMSes be sent out in parallel. You also get error handling for free with resque, it gahers all messages that resulted in an exception in a 'failure' queue, which you can then debug or reschedule again
You could use whenever to schedule events.
I have a rails app deployed on Heroku. I want to add a feature that enables users of the app to set a reminder. I need some way for the app to schedule sending an email at the time specified by the user.
I have found numerous posts referring to using delayed_job for this, but none of the write-ups / tutorials / etc. that I have found directly address what I am trying to accomplish (the descriptions I have found seem more geared towards managing long-running jobs that are to be run "whenever").
Am I on the right track looking at delayed_job for this? If so, can somebody point me towards a tutorial that might help me?
If delayed_job is not quite right for the job, does anybody have a suggestion for how I might approach this?
The most typical way of handling this is to use a cron job. You schedule a job to run every 15 minutes or so and deliver any reminders that come up in that time. Unfortunately, heroku only allows cron jobs to run every hour, which usually isn't often enough.
In this case, I'd use delayedjob and trick it into setting up a recurring task that delivers the notifications as often as necessary. For example, you could create a function that begins by rescheduling itself to run in 10 minutes and then goes on to send any reminders that popped up in the previous 10 minutes.
To view delayedjobs send_at syntax to schedule future jobs check here: https://github.com/tobi/delayed_job/wiki
ADDED after comments:
To send the reminder, you would need to create a function that searches for pending reminders and sends them. For example, let's say you have a model called Reminder (rails 3 syntax cause I like it better):
def self.find_and_send_reminders
reminders = Reminder.where("send_at < ? AND sent = ?", Time.now, false).all
reminders.each do |r|
#the following delayed_job syntax is apparently new, and I haven't tried it. came from the collective_idea fork of delayed_job on github
Notifier.delay.deliver_reminder_email(r)
#I'm not checking to make sure that anything actually sent successfully here, just assuming they did. may want to address this better in your real app
r.update_attributes!(:sent => true)
end
#again using the new syntax, untested. heroku may require the old "send_at" and "send_later" syntax
Reminder.delay(:run_at => 15.minutes.from_now).find_and_send_reminders
end
This syntax assumes you decided to use the single reminder entry for every occurence method. If you decide to use a single entry for all recurring reminders, you could create a field like "last_sent" instead of a boolean for "sent" and use that. Keep in mind these are all just ideas, I haven't actually taken the time to implement anything like this yet so I probably haven't considered all the options/problems.
Check out the runt gem, may be useful for you: http://runt.rubyforge.org/
You can use delayed_job's run_at to schedule at a specific time instead of whenever.
If your application allows the users to change the time of the reminders you need to remember the delayed_job to be able to update it or delete it when required.
Here is more details.
It's good to avoid polling if you can. The worker thread will poll at the database level, you don't want to add polling on top of polling.