What is the best way to implement message passing between 2 gen_servers. e.g. example of system.
So per the image I have multiple session servers that will query the database for a list of players, (players records are added to the DB as they join), that best fit what that session needs e.g players location, level, win ratio etc.
The session will poll all players that the DB returns and take the first reply's until its reached its max players, session will then hand off players records it has accumulated to a server and the job is done.
What i'm asking is the best way to handle the message passing with least amount of error, what i am thinking so far is
Session passes messages to all players
players receive messages returns ok
session receives players ok,adds to list of players, returns message so player can stop receiving messages and change state to in session
4 if player gets no reply from session, check other received messages and reply(session is probably full or died so continue on)
Is this the best implementation for this section?
Would cast be the best way as i am not worried about getting a specific result for either player or session?
Or even if my architecture is completely off any criticism or change is welcome as I am new to Erlang and OTP?
Also session is probably a poor choice of name I was thinking of changing to game instance.
Related
I have a dataflow pipeline that collects user data like navigation, purchases, crud actions etc. I have this requirement to be able to identify patterns real time and then dispatch pubsub events that other services can listen to in order to provide the user real time tips, offers or promotions.
I'm thinking to start grouping the events by user id and then if the match a pattern to create a PCollection that contains the events names that need to be triggered via pubsub.
Is this the right approach? Is there a better way?
This could certainly work for some use cases.
If you are using session based windowing in combination with early firings (triggering upon arrival of each element). You can have all the data needed to identify patterns each time a new element arrives.
However, depending on the rate of user data being pushed and the size of the session, this might result in holding a lot of data in the PCollection and repeating this pattern matching a lot (on the same data), since you have to reuse all the data in the session. Furthermore you cannot use elements that arrived before this session.
Sometimes, you might be better off by keeping a state for each user (without redoing the pattern matching on all the data of the user for this session). Using a state would in fact remove the need to work with windowing.
The new process would now look like this:
For each element that arrives:
Fetch the current state
Calculate the new state (based on the old state and the new element)
If needed, emit a message to PubSub.
To hold your state, you could use BigTable or Datastore.
I am writing a ejabberd module where the user controls when the message is delivered to the recipient instead of delivering immediately(like birthday wishes sending in advance). This is done by adding a custom xml element to the message stanza like the following
<message xmlns="jabber:client" from="test2#ubuntu" to="test1#ubuntu/32375806281445450055240436" type="chat">
<schedule xmlns="ank" year="2015" month="10" day="19" hour="22" minute="36" second="13"/>
<body>hi</body>
</message>
Now these scheduled messages has to be stored in the mnesia database and send to the recipient when the time arrives.
Approach 1:
One approach is to create a table for every user, when the message is received, store the message to the users table and set a timer to process
the message and delete when done like the following sample code
timer:apply_after(SecondsDelay, ?MODULE, post_message_delete, [TableName, RecordUniqueKeyHash, From, To, Packet]).
The post_message_delete method will send the message when called after the timer expires using the route method as shown in the following and delete the record from the mnesia database.
ejabberd_router:route(From, To, Packet)
Creating a table for every user is not feasible due to the limitations on the number of tables in mnesia.
Approach 2:
Another approach is to store all the users messages in one single table and set the timer(same as above) for every message as the message arrives and once the message is processed delete it.
The whole idea of using the mnesia database is to process the messages reliably in the case of ejabberd server crash.
To achieve this we use a pid field in the record of every message. There is a pid field for every message record that contains the pid of the process that is processing this message. Initially it is undefined(When the message arrives at the filter_packet hook) but after when the message processing method is spawned it updates the pid in the record in the mnesia database.
So if the server crashes on reboot in the modules start method all the messages are iterated and checked if the pid is alive(is_process_alive), if not alive then spawn the processing method on the message which will update with the new process pid, process the message and delte once done.
Drawbacks
The drawback of this method is that even if a message has to be delivered far in the future(next month or next year) still a process is running for this message and there are as many processes running as there as many messages.
Approach 3:
To over come the drawbacks of Approach 2, scan the database every hour and accumulate the messages that has to be delivered only for the next hour and process it.
The drawback with this approach is that the database is scanned every hour that might impact performance.
Approach 4:
To over come the performance of Approach 3, we can create tables for every year_month and spawn the message processing function only on the current months table.
What other approach is best suited for this use case using mnesia database?
Even though it's an old question, but it may one day become an issue for somebody else.
I think mnesia is the wrong choice for this kind of data store use case.
Redis from version 2.8.0 has a keyspace event notification features when certain operations are performed including key expiration commands set by EXPIRE, EXPIREAT and other variants. This information can reach your code by the PUBSUB feature. See Redis Keyspace Notifications
on how to start.
Generate a unique key(K) probably UUID for every birthday message.
Store the message, the entire XML, to send under the given generated K.
Store this message key as a value under a key called K:timer using the SET command with TTL set to the time difference between now and the birthday timestamp in seconds, OR use the EXPIREAT to set the message expiration time to the Unix timestamp of the birthday itself. When TTL expires, pubsub clients get notified of the event with the information of the key to expire K:timer. Extract the K and fetch the message with it. Send your message and delete it afterwards.
ISSUES TO CONSIDER:
1: Multiple pubsub clients may get notified of the same expiration event. This may cause the same message to be sent more than once. Implement some sort of locking to prevent this.
2: Redis PUBSUB is a fire and forget message passing construct. So if a client goes down and comes up again, it may have missed event notifications during this time window. One way to ensure reliability is to store the key, K, under different key variants of K:timer, K:timer:1, K:timer:2, K:timer:3,... at increasing TTL offsets(1, 2, 3, minutes in between) to target a worst time window during which the unavailable client may become available.
3: Redis is in-memory. Storing lots of large messages will cost you RAM. One way to solve this is to store only the message key K in redis and store the message (XML) with the same key, K, in any disk base key-value store like Riak, Cassandra etc.
Let's say I have some record like {my_table, Id, Value}.
I constantly overwrite the value so that it holds consecutive integers like 1, 2, 3, 4, 5 etc.
In a distributed environment, is it guaranteed that my event listeners will receive all of the values? (I don't care about ordering)
I haven't verified this by reading that part of the source yet, but it appears that sending a message out is part of the update process, so messages should always come out, even on very fast changes. (The alternative would be for Mnesia to either queue messages or queue changes and run them in batches. I'm almost positive this is not what happens -- it would be too hard to predict the variability of advantageous moments to start batching jobs or queueing messages. Sending messages is generally much cheaper than making a change in the db.)
Since Erlang guarantees delivery of messages to a live destination this is as close to a promise that every Mnesia change will eventually be seen as you're likely to get. The order of messages couldn't be guaranteed on the receiving end (as it appears you expect), and of course a network failure could make a set of messages get missed (rendering the destination something other than live from the perspective of the sender).
Lately I have seen many projects which have a web interface and backend in erlang (e.g. flash games on fb or similar).
I was wondering something like that (maybe simpler) could be implemented.
I believe they have an erlang application to provide authentication, and another one to provide routing to the actual game. But then, how do they start a new instance of the game application for each different user? Maybe they create a new node and start the application there? Can you provide me some insight?
Using the actor model you create a process for each connected player which reacts to messages from the Flash client, sends messages either directly to other player processes or to a central process handling global state. This player process reacts to messages from other players as well and send updates back to the client.
You may then build your system architecture upon this idea. Maybe you have multiple central processes which handles different parts of the game. Maybe you let players be split between different "rooms" where each room is its own process. Maybe you have one player process which is active even when the player is disconnected, or maybe you just queue messages and react to them when the player returns. The architecture will of course largely depend upon the type of game you are building.
I'm writing a FPS XNA game. It gonna be multiplayer so I came up with following:
I'm making two different assemblies — one for the game logic and the second for drawing it and the game irrelevant stuff (like rocket trails).
The type of the connection is client-server (not peer-to-peer), so every client at first connects to the server and then the game begins.
I'm completly decided to use XNA.Framework.Game class for the clients to run their game in window (or fullscreen) and the GameComponent/DrawableGameComponent classes to store the game objects and update&draw them on each frame.
Next, I want to get the answer to the question:
What should I do on the server side? I got few options:
Create my own Game class on the server, which will process all the game logic (only, no graphics). The reason why I am not using the standart Game class is when I call Game.Run() the white window appears and I cant figure out how to get rid of it.
Use somehow the original XNA's Game class, which is already has the GameComponent collection and Update event (60 times per second, just what I need).
UPDATE:
I got more questions:
First, what socket mode should I use? TCP or UDP? And how to actually let the client know that this packet is meant to be processed after that one?
Second, if I is going to use exacly GameComponent class for the game objects which is stored and processed on the server, how to make them to be drawn on the client? Inherit them (while they are combined to an assembly)? Something else?
First of all, your game logic should be on the server.. Not only does that prevent cheating, but it also garantees consistency, especially over random operations. The clients should only be sending their inputs to the server
I'd recommend your keep the server's window visible to make it a debug console, you'll need it, to know what your server is doing exactly.
For a FPS, UDP is recommended. You'll be sending a lot of data and you don't really care if your packets are all received or ordered. While the packets are not garanteed to arrive ordered, you don't really have to worry about it. Most of the time, they will arrive in order anyway. Let's say you send 60 frames per second to your clients and one of your packet arrives in the wrong order: Your client will have erroneous information for 1/60th of a second, which is barely visible.
Finally, you should send a serialized representation of your game state multiple times per second to your clients. They should be able to retrieve that information and draw it the same way as your server. You could even serialize your gamecomponent and send it if you think that's appropriate. It's really up to you to decide.
Hope this helps!