I'm writing a simple chat room application in Rails 3.1 - for learning purposes.
For starters I have all the needed models (messages, users, rooms, etc.) and things work great.
The clients poll the server every minute (for example) and get new messages if they have any.
I would like to change the simple polling to long polling and can't figure out if this can be done in the same app or do I have to create some other Push server for the long polling.
I read a lot about EventMachine and changed my rails app to user it as I wanted to use EventMachine for the event driven mechanics. I thought that the EventMachine channel would come in handy for this.
A client would connect and wait for a message in the chat room and it will receive a message only when one was sent to the room.
What I can't figure out is how can I share the EventMachine::Channel instance between all my client connections.
Is this approach even possible or am I going at it the wrong way?
If possible I would like a solution that can run as a single rails application hosted on Heroku.
Yeah sure. I just wrote a demo using event machine. My case is that player walking around a map, and other players should be able to see it.
The demo looks like that:
A client establishes a connection, reporting its own coordinate(generated randomly)
There is an array preserving all the coordinates for each client
When a client moves, it sends its new coordinate to the server. Then the server finds out people near him(from the array), and push the new coordinate to those clients.
I tested it with nearly 5000 clients, and each second 20-30 players moves its position. And the server process only takes less that 100M memory & 50%-60% cpu usage(on a single core).
In your case, I think you should probably try faye too. It's based on event machine, and an appropriate solution to things like chat room.
Expanding what I've mentioned on the comment, check this blog post that explains how to create a text based chat app using EM, and uses AMQP to broadcast the messages to the other users.
I think you can probably do the same or use some in memory queues to share messages, and this definitely should work on heroku, as you don't have a dependency to an external service such as RabbitMQ.
Here's a good discussion about different queue frameworks: ActiveMQ or RabbitMQ or ZeroMQ or
Rails will have streaming added in version 4.
For now, you can streaming (long polling) like in this example with Sinatra and Redis's Pub/Sub feature as a backend. You will have to add another action to handle user sent messages, adding them to Redis's with PUBLISH command. You should use an evented server like Thin or Puma.
Related
I'm building a simple chat app on iOS for fun (and to have projects to gain experience from), using socketsIO and a node backend. I am trying to figure out the best design for messages. I was planning to use a mongoDB database where each conversation would have its message data stored. Whenever the client sends a new message to the server, the server adds it to the appropriate conversation in the database.
I was also hoping to create a user Sign Up/Log In system which would add you to the database.
However, I've googled around quite a bit and I am really not sure if creating a database made up of conversations (that get updated whenever a sentMessage event is triggered) and user data is the right way to go.
Additionally, I've seen some people talk about saving the chats on the actual devices themselves, not in a database? What is the common design pattern for a chat app like this?
for the design I would use socket.io for emitting messages as well. It has a great community behind it, I woul also use MongoDb because everything is using JSON format and it's integrated so well with Node due to it using JavaScript.
Now the part you are interested about, is REDIS. Redis is a database that sits in RAM on the web and should be used with mongodb if you're going to be having higher traffic / need quick speed / less hanging and waiting.
REDIS would be your temporary save for the chat with a session because doing disk write/read/querying is a lot on the machine (looking at you MongoDB), If you plan on saving the chat with every message. Doing so MongoDb would just not scale all the well in the long run and is not as fast as REDIS. Mind you REDIS database will only hold the temporary chat log of let's say the last 1 million chat session or some limit (it's all in RAM so the size is limited can't have Terabytes or hundreds of Gigabytes of RAM on 1 server).
so the data flow would look something like
user sends message
server receives messsage via HTTP(S) post/put - Ajax/Observable
Server will use socket.io to emit the message to the designated user while saving the message to REDIS with a specific key/session/message.
designated user get's the update on their screen via io event.
-- inbetween there should be a check on the REDIS db of whether it is getting full. if it's full remove the last 10,000 inactive messages (could be from 1 year ago if the server hasn't gotten full yet) to make some space.
Saving the chat on the phone is an okay idea as it would save the users data/bandwidth and they could potentially look at their message while offline.
a solution is using SQL Lite which is a lightweight library that will sit inside your app acting as a database which you can perform queries on if your familiar with RDBMS you will have no problem implementing it. But now you gotta find a good way to manage saving data to REDIS/SQL-LITE/MongoDb.
I'm building a web app that tracks stats for a game. The API for that game (Steam Web API) only allows me to retrieve data from the most recently played match.
When a user requests their stats to be tracked, I need make a call to the Steam Web API every 10 minutes or so to check if they have played another match, then store it in the database if they have. The problem is, I check every 10 minutes for every user...
Is there a way to schedule this efficiently so that the server doesn't get overloaded? This application could potentially server 10,000+ users.
Please feel free to correct anything in this question if I got something obviously wrong.
There are many gems for recurring events in ruby, like:
ice cube
recurrence
However, if you are going to have lots of users, and you are worried about server load, I would suggest not using your rails app to do this.
Instead, build another service which doesn't run on your rails app to update your database with statistics.
Having an independent service like this decoupled from your main app allows you to easily put it on another server, and/or have it scale independently from your webapp.
I have two Rails apps (one for web, and one for backend) accessing to the same PGSQL database. I would like to notify the other app if one app changes a table in the database.
How should I go about it?
I think this depends on:
How reliable you need it to be.
How fast you need the notifications to be delivered.
The FAYE solution suggested by #techvineet provides a good fast but unreliable option. (N.b. I don't mean it'll fail often, but it likely will occasionally, maybe 1/1000, If that causes you a problem, then avoid)
If you need something 100% reliable, and speed isn't important, you could write audit events to the database, and then poll that table from each app, if these are committed in the same transaction as the actual work is done, you should be safe... But it'll be as slow as your polling cycle.
Lastly, you if you want something fast AND reliable, then you could look at using something like ActiveMQ or RabbitMQ to give you reliable messaging between the applications to notify changes. You'll need a worker process in each app to listen to changes and deal with them appropriately.
My last comment would be that this 'smells' a little. The fact that you're trying to do this makes me think the architecture of your app might need looking at in the longer term. An obvious way of doing it might be to encapsulate all the business logic into an app which exposes an API, and then calling that API from both front and back end applications.
You can try using Faye http://faye.jcoglan.com/, which is a publish and subscribe messaging server. It can be integrated with Rails https://github.com/jamesotron/faye-rails.git. Messages can be transferred from one app to another by subscribing to the messages and publishing.
Hope this will help.
We have an existing API where a client asks our server for information that we have to get from another external server. When the external server takes a long time, say 10 seconds, it holds up a Rails passenger instance for that whole 10 seconds.
Is there some way to pass the rendering of our reply to delayed_job so that I can free up the Rails instance?
NOTE: Ideally, we would just update our API and reply to our API client that we are busy and to try back again in a few seconds to see if we are ready. However, there are already thousands of clients out there and changing them is not practical at this time.
The usual way to handle this is to queue up the job and return immediately, then poll or use some async notification framework like Pusher or Faye to update the remote client. You definitely cannot pass the connection to DJ as you describe. Another avenue you might investigate is using EventMachine to handle it, a lá http://railstips.org/blog/archives/2011/05/04/eventmachine-and-passenger/. A third alternative would be to precache the data from the remote web service, but that is an avenue very dependent on what you're doing (authorization, for example, is not something you could do there.)
The basic bottom-line is that you're dealing with a bit of an architecture issue. If you absolutely have to talk to the remote service AND output the results in the request cycle, there's not a lot you can do about it short of changing to a more evented backend like EventMachine or Node.js.
I'm writing a Rails web service that interacts with various pieces of hardware scattered throughout the country.
When a call is made to the web service, the Rails app then attempts to contact the appropriate piece of hardware, get the needed information, and reply to the web client. The time between the client's call and the reply may be up to 10 seconds, depending upon lots of factors.
I do not want to split the web service call in two (ask for information, answer immediately with a pending reply, then force another api call to get the actual results).
I basically see two options. Either run JRuby and use multithreading or else run several regular Ruby instances and hope that not many people try to use the service at a time. JRuby seems like the much better solution, but it still doesn't seem to be mainstream and have out of the box support at Heroku and EngineYard. The multiple instance solution seems like a total kludge.
1) Am I right about my two options? Is there a better one I'm missing?
2) Is there an easy deployment option for JRuby?
I do not want to split the web service call in two (ask for information, answer immediately with a pending reply, then force another api call to get the actual results).
From an engineering perspective, this seems like it would be the best alternative.
Why don't you want to do it?
There's a third option: If you host your Rails app with Passenger and enable global queueing, you can do this transparently. I have some actions that take several minutes, with no issues (caveat: some browsers may time out, but that may not be a concern for you).
If you're worried about browser timeout, or you cannot control the deployment environment, you may want to process it in the background:
User requests data
You enter request into a queue
Your web service returns a "ticket" identifier to check the progress
A background process processes the jobs in the queue
The user polls back, referencing the "ticket" id
As far as hosting in JRuby, I've deployed a couple of small internal applications using the glassfish gem, but I'm not sure how much I would trust it for customer-facing apps. Just make sure you run config.threadsafe! in production.rb. I've heard good things about Trinidad, too.
You can also run the web service call in a delayed background job so that it's not hogging up a web-server and can even be run on a separate physical box. This is also a much more scaleable approach. If you make the web call using AJAX then you can ping the server every second or two to see if your results are ready, that way your client is not held in limbo while the results are being calculated and the request does not time out.