I have a mail service that uses mail plugin to send out notification. I have this triggered of the save method. What I have noticed is that the save method is now slower because part of the processing time is generating email template off the GSP and sending the message to the smtp host.
What is the best strategy to speed up this proccess? I am thinking some kind of async processing. Are there facilities in Grails or Groovy that I can use (do I fall back to threads...?).
Yes, there are several alternatives for async processing in Grails. I ended using the Executor plugin for the exact same use-case as you and it works like a charm.
Related
I'm in the process of planning the development of a mail-server to hand the sending of email across our multiple websites. Below is a description of what we are planning to implement and I'd like your opinion/suggestions.
We use ASP.NET MVC and have many web-sites hosted on Azure. We currently send mail internally within each of the web applications using SMTPServer.Send(). Obviously this is not the ideal way to send emails when you have a decently busy set of websites because the send mail call is blocking and cannot guarantee mails are sent. With this I'm worried out getting an influx of mail requests when we launch our next website (we think it'll get a decent amount of traffic and lots of emails will be sent).
My plan of action : to build a centralised mail-server that runs in the background (we use azure and this will be simply another web-application). When each one of our web applications wants to send a mail, instead of doing this internally, it'll call a web method on the mail-server called sendMail() this function will accept certain parameters and insert the mail parameters, content etc. into a database. The mail server will then poll the database at fixed time intervals, select a set of unsent emails and attempt to send them using the same SMTPServer.Send() function. If an email fails for some reason we won't flag it as sent and in the next poll interval the email will be selected again and another send attempt will be made. (we will cap the number of send attempts to say 20).
This will allow each of the websites to run smoothly without having loads of blocking send mail calls internally and the mail server will handle all the sending sequentially and in a controlled environment as a separate standalone web-application.
Thanks in advance!
Looks like a good design, Don't know the entire scenario which let to you building something like an email server. The problem has been solved well by using a service that already exist like Office 365.
Your design is good, My suggestions would be the following,
You can use Azure WebJobs to build the polling agent. You can make the web job run as a scheduled web job that does the polling and sending the mail and it can be written very clean as a simple console app.
You can use Azure API App to build the SendMail() call and you can use the Azure AD Auth on the API to authenticate the caller of the API using the Authentication and Authorization feature to easily secure your email server. You can also enable CORS easily as well to make sure you receive requests from other websites and process it.
Some issues I foresee for you,
Volume and Scaling : You can only process so much email between each polling. If you cannot then you will need to create another polling agent which will making things complicated as they need to know they are picking different sets of emails to send. If you volume is going to be low you should be fine.
Challenge : Why can't the websites send the mail them selves ? And then record it on the database for tracking. All you have to do build module or a component that they use on their web page to create and send the mail. Polymer 1.0 works well for this scenario.
Hope this helps to get you started.
Most major services like github provide Webhooks functionality.
So, with github - you can set hooks to notify you on every commit.
In the same time web hooks are not that easy.
Each web hook has to be ran asynchronously to not block web server at the time of communicating with destination. And it can take a good time (10-15 seconds). There should be implemented repeating functionality (in case if destination is not responding).
So, I think that there for sure should be some service or library which will do this for you.
Do you know any of these?
I need to send data to lots of endpoints and to receive a response from them..
You need a gem providing background job functionality. Sidekiq and Delayed Job are ones of most frequently used.
Idea is that after request (in ruby on rails you can use after_action hook or just do it in controller action) you create a job which will be executed asynchronously. Put logic you need in the job class
Both sidekiq and delayed job have repeating functionality, just pick gem that looks simpler to use
There is a gem called ActiveHook but it does not appear to be maintained anymore.
Benedikt Deicke wrote a good article on sending webhooks with Rails, you should check it.
Preamble: I'm trying to put together a proposal for what I assume to be a very common use-case, and I'd like to use Amazon's SWF and SQS to accomplish my goals. There may be other services that will better match what I'm trying to do, so if you have suggestions please feel free to throw them out.
Problem: The need at its most basic is for a client (mobile device, web server, etc.) to post a message that will be processed asynchronously without a response to the client - very basic.
The intended implementation is to for the client to post a message to a pre-determined SQS queue. At that point, the client is done. We would also have a defined SWF workflow responsible for picking up the message off the queue and (after some manipulation) placing it in a Dynamo DB - again, all fairly straightforward.
What I can't seem to figure out though, is how to trigger the workflow to start. From what I've been reading a workflow isn't meant to be an indefinite process. It has a start, a middle, and an end. According to the SWF documentation, a workflow can run for no longer than a year (Setting Timeout Values in SWF).
So, my question is: If I assume that a workflow represents one message-processing flow, how can I start the workflow whenever a message is posted to the SQS?
Caveat: I've looked into using SNS instead of SQS as well. This would allow me to run a server that could subscribe to SNS, and then start the workflow whenever a notification is posted. That is certainly one solution, but I'd like to avoid setting up a server for a single web service which I would then have to manage / scale according to the number of messages being processed. The reason I'm looking into using SQS/SWF in the first place is to have an auto-scaling system that I don't have to worry about.
Thank you in advance.
I would create a worker process that listens to the SQS queue. Upon receiving a message it calls into SWF API to start a workflow execution. The workflow execution id should be generated based on the message content to ensure that duplicated messages do not result in duplicated workflows.
You can use AWS Lambda for this purpose . A lambda function will be invoked by SQS event and therefore you don't have to write a queue poller explicitly . The lambda function could then make a post request to SWF to initiate the workflow
I have JMS plugin installed in Grails 2.2.0. It works famously. However, I now want to extend the functionality to all my domain objects to persist them across a JMS ActiveMQ connection.
The database is on the remote system which has an ActiveMQ broker on it. I can talk to it fine. But, I'm unclear on how to implement the code to save(), search(), update(), etc from my controller using JMS rather than a local DB. I think I have save() figured out but not for sure. Then how do I hook into the dynamic finders and such to supply the needed service so the actual search criteria is sent over a queue and the response is sent back on another queue and populates the correct Grails results for display? My head hurts from reading so much and getting no where fast.
Thanks,
Marshall
What is the best way to receive email via IMAP protocol in Rails 3?
I am not sure about best, but a simple way is to use Net::IMAP. You can write a rake task and periodically poll the mailbox for emails by using a scheduler like cron.
So your setup would look like this:
A library file which servers as a wrapper for IMAP client and other related operations, like processing the emails.
The rake task which when called downloads the emails and processes them using the wrapper mentioned above.
The scheduler (Cron or any other of your choice. I personally always prefer cron) which periodically calls this rake task.
As per ActionMailer's documentation the solution is to have the email forwarded to your rails application and implement UserMailer.receive(STDIN.read) to process the email.
The documentation (Action Mailer Basics) doesn't sound very convincing about this approach. It's not telling where the call should be implemented but I guess that would be at the MTA level.