Could you please help me? I've a problem related with the gen_tcp send function. I've been trying to send few tuples, about 10-15 items, which were decoded to amf objects, from my erlang server to my flash client.
case get_tcp:send(Socket, Msg) of
ok -> io:format("sent~n");
{erorr, Err} -> io:format("~w~n", [Err])
end
No errors but the flash client doesn't recieve whole data just about 8-11 items. The socket options are [binary, {active, true}, {reuseaddr, true}]. I've checked my network where I use the client - sent packets were fragmented into two fragments, big and small ones. The big one is the first fragment of packet and small one is the next. Amount of the bigs is the same as amount of sent messages, but the smalls are much less and about number of received messages by the flash client.
It only reproduces if I send data fast, if I do it slowly it seems ok. Does anyone know why it happens? It will be very helpful.
My guess is that this is a framing issue.
TCP is a streaming protocol, so when you read in flash, you are not guaranteed to get all the messages right away. Rather you need some kind of framing setup, say {packet, 2} or {packet, 4} on the socket options. This effectively turns TCP from a streaming protocol into a messaging protocol. And I think you want the latter.
I think you can do like this:
1 Before send msg(your tuple), use term_to_binary(Msg) to get binary, then sizeof the binary, then, after your client receive the msg, use binary_to_term to get tuple.
2 Both the client and server you need to set the socket option{packet, 2 or 4}
I not tried amf, but I use erlang+flash and json instead of amf
What packet option of socket? (I use {packet, 2}, for example)
How did you read data on client side? Maybe you not read all data to end and no new events raised?
Related
E.g. suppose I have a module that implements gen_server behavior, and it has
handle_call({foo, Foo}, _From, State) ->
{reply, result(Foo), State}
;
I can reach this handler by doing gen_server:call(Server, {foo, Foo}) from some other process (I guess if a gen_server tries to gen_server:call itself, it will deadlock). But gen_server:call blocks on response (or timeout). What if I don't want to block on the response?
Imaginary use-case: Suppose I have 5 of these gen_servers, and a response from any 2 of them is enough for me. What I want to do is something like this:
OnResponse -> fun(Response) ->
% blah
end,
lists:foreach(
fun(S) ->
gen_server:async_call(S, {foo, Foo}, OnResponse)
end,
Servers),
Result = wait_for_two_responses(Timeout),
lol_i_dunno()
I know that gen_server has cast, but cast has no way to provide any response, so I don't think that that's what I want in this case. Also, seems like it should not be the gen_server's concern whether caller wants to handle response synchronously (using gen_server:call) or async (does not seem to exist?).
Also, the server is allowed to provide response asynchronously by having handle_call return no_reply and later calling gen_server:reply. So why not also support handling response asynchronously on the other side? Or does that exist, but I'm just failing to find it??
gen_server:call is basically a sequence of
send a message to the server (with identifier)
wait for the response of that particular message
wrapped in a single function.
for your example you can decompose the behavior in 2 steps: a loop that uses gen_server:cast(Server,{Message,UniqueID,self()} with all servers, and then a receive loop that wait for a minimum of 2 answers of the form {UniqueID,Answer}. But you must take care to empty your mail box at some point in time. A better solution should be to delegate this to a separate process which will simply die when it has received the required number of answers:
[edit] make some correction in the code now it should work :o)
get_n_answers(Msg,ServerList,N) when N =< length(ServerList) ->
spawn(?MODULE,get_n_answers,[Msg,ServerList,N,[],self()]).
get_n_answers(_Msg,[],0,Rep,Pid) ->
Pid ! {Pid,Rep};
get_n_answers(_Msg,[],N,Rep,Pid) ->
NewRep = receive
Answ -> [Answ|Rep]
end,
get_n_answers(_Msg,[],N-1,NewRep,Pid);
get_n_answers(Msg,[H|T],N,Rep,Pid) ->
%gen_server:cast(H,{Msg,Pid}),
H ! {Msg,self()},
get_n_answers(Msg,T,N,Rep,Pid).
and you cane use it like this:
ID = get_n_answers(Msg,ServerList,2),
% insert some code here
Answer = receive
{ID,A} -> A % tagged with ID to do not catch another message in the mailbox
end
You can easily implement that by sending each call in a separate process and waiting for responses from as many as required (in essence this is what async is about, isn't? :-)
Have a look at this simple implementation of parallel call which is based on the async_call from rpc library in OTP.
This is how it works in plain English.
You need to make 5 calls so (in the parent process) you spawn 5 child Erlang processes.
Each process sends back to the parent process a tuple containing its PID and the result of the call.
The tuple can be only constructed and send back only when the desired call has been completed.
In the parent process you loop through responses in the receive loop.
You can wait for all responses or just 2 or 3 out of the started 5.
The parent process (which spawns the worker processes) will eventually receive all responses (I mean those you want to ignore). You need a way to discard them if you don't want the message queue to grow infinitely. There are two options:
The parent process itself can be a transient process, created only for the call to spawn the other 5 child processes. Once the desired amount of responses is collected it can send the response back to a caller and die. Messages send to the died process will be discarded.
The parent process can continue receiving messages after it has received the desired amount of responses and simply discard them.
gen_server do not have a concept of async calls on client side. It is not trivial how to implement in consistently because gen_server:call is a combination of monitor for server process, send request message and wait for either answer or monitor down or timeout. If you do something like what you mentioned you will need to deal with DOWN messages from server somehow ... so hypothetical async_call should return some key for yeld and also an internal monitor reference for a case you are processing DONW messages from other processes... and do not want to mix it with yeld errors.
Not that good but possible alternative is to use rpc:async_call(gen_server, call, [....])
But this approach have a limitation in calling process will be a short lived rex child, so if your gen server use caller pid somehow other than send it a reply logic will be broken.
gen_sever:call to the process itself would surely block until timeout. To understand the reason, one should be aware of the fact that gen_server framework actually combine your specific code together into one single module, and gen_server:call would be "translated" as "pid ! Msg" form.
So imagine how this block of code takes effect, the process actually stay in a loop keeping receiving messages, and when the control flow run into a handling function, the receiving process is temporarily interrupted, so if you call gen_server:call to the process itself, since it is a synchronous function, it waits for response, which however would never come in until the handing function returns so that the process can continue to receive messages, so the code is in a deadlock.
things are like this:
I used AsyncSocket to connect with server. when the server send some bytes to client at a time, app will invoke (onSocket: didReadData: withTag:) method; Sometimes, because of the network or other reason, some bytes arrived and the rest bytes arrived later( which means some bytes are delayed).(timeOut = -1)
==================================
Question: how much times did the method(onSocket: didReadData: withTag: ) invoke?
1)only one time - it will wait automatically until the bytes are complete？
2)2 or more times - some arrived(invoke), other arrived(invoke)、、、、、like so?
===================================
which thoughts is right? plz geiv me some advice. Thanks very much.
OK， maybe it's just like this.
Transfer data on TCP: when you send 1000 bytes at a time, because of lag or other reasons, you may receive the 900 bytes first, but the TCP socket will wait the rest 100 bytes
automatically. So if I use asyncSocket to transfer datas and set the timeout, during these seconds, I don't need to deal with the lagged data.
=========================
EDIT
maybe the title of ths question is not appropriate, no one cares the question.
I used the method -readDataToLength: instead, first read header.length of data, and parse it, then read the body.length of data;
I have a device that sends data to my server via gprs . The problem is that it sends raw data and i don't know where i can stop the reading
Currently i am using something TIdHttpServer and something like this to read the strings :
var
s : string;
repeat
s:=s+acontext.Connection.Socket.ReadChar;
until acontext.Connection.Socket.InputBufferIsEmpty;
Is there a better solution to my problem ?
TCP is stream oriented. If the protocol is unknown, the server only can try to read into a byte array (if memory is not a problem) or a file stream. If the client disconnects normally, the data is 'complete'. Unfortunately, if the protocol is unknown, the server can not tell wether the client died or disconnected normally.
InputBufferIsEmpty does not help, as it only says if there is data in the (TCP) buffer - and depending on latency this can happen frequently, but it does not mean that there are no more in-flight bytes.
You could try to 'reverse engineer' the protocol, by sending known strings over the client devices. But if the sender is a black box, there can be many special cases - think of encoding or 'escape' characters etc.
You could make up you own protocol.
Some ideas are:
use a special character or characters combo to define the end of the
message.
append at the start of the message some fixed size field with the size of the message
This could be a very basic question but is Erlang capable of calling a method on another prcoess and wait for it to repond back with some object type without sleeping threads?
Well, if you're waiting for an answer, the calling process will have to sleep eventually… But that's no big deal.
While processes are stuck on the receive loop, other processes can work. In fact, it's not uncommon to have thousands of processes just waiting for messages. And since Erlang processes are not true OS threads, they're very lightweight so the performance loss is minimal.
In fact, the way sleep is implemented looks like:
sleep(Milliseconds) ->
receive
% Intentionally left empty
after Milliseconds -> ok
end.
Yes, it is possible to peek into the mailbox if that is what you mean. Say we have sent a message to another process and now we want to see if the other process has sent something back to us. But we don't want to block on the receive:
receive
Pattern -> Body;
Pattern2 -> Body2
after 0 ->
AfterBody
end
will try to match against the Pattern and Pattern2 in the mailbox. If none matches, it will immediately time out and go to AfterBody. This allows you to implement a non-blocking peek into the mailbox.
If the process is a gen_server the same thing can be had by playing with the internal state and the Timeout setting when a callback returns to the gen_server's control. You can set a Timeout of 0 to achieve this.
What am getting from the question is that we are talking of Synchronous Message Passing. YES ! Erlang can do this perfectly well, its the most basic way of handling concurrency in Erlang. Consider this below:
rpc(Request, To)->
MySelf = self(),
To ! {MySelf,Request},
receive
{To,Reply} -> Reply
after timer:seconds(5) -> erlang:exit({error,timedout})
end.
The code above shows that a processes sends a message to another and immediately goes into waiting (for a reply) without having to sleep. If it does not get a reply within 5 seconds, it will exit.
I am sending data to the server twice. First, I send "Hello world" and then I send "Server".
But the server received the data at 1 read. But the server have to read the data in a two-read operation.
Also, I write the data. Then read data from server and then I write the data.
In this case, the server can read the first data. But server can not read the second data.
The server uses read, write, read.
So how to overcome this issue? How do I write data to socket in BlackBerry?
What you describe is how TCP is supposed to work by default. What you are seeing is the Nagle algorithm (RFC 896) at work, reducing the number of outbound packets being sent so they are processed as efficiently as possible. You may be sending 2 packets in your code, but they are being transmitted together as 1 packet. Since TCP is a byte stream, the receiver should not be making any assumptions about how many packets it gets. You have to delimit your packet data in a higher-level protocol, and the receiver has to process data according to that protocol. It has to handle cases where multiple packets arrive in a single read, a single pakcet arriving in multiple reads, and everything in between, only processing packet data when they have been received in full, caching whatever is left over for subsequent reads to process when needed.
Hard to say without a little more detail, but it sounds like you're using 1-directional communication in the first case - i.e. the client writes, then writes again. There are any number of reasons that the server would receive the 2 writes as 1 read. Buffering on the client, somewhere in the wireless stack (or in the BES), buffering on the server side. All of those are legal with TCP/IP.
Without knowing anything more about your solution, have you thought about defining a small protocol - i.e. the client writes a known byte or bytes (like a 0 byte?) before sending the second write? Then the server can read, then recognize the delimiting byte, and say 'aha, this is now a different write from the client'?
As previously said this is an expected TCP behavior to save bandwidth. Note that to deliver your package TCP adds lot of data (e.g. destination port,sequence number, checksums...).
Instead of flushing the data I´ll recommend you to put more work in your protocol. For example you can define a header that contains the number of bytes to read and then the payload (the actual data).
The following code is a protocol encoded in an string with the structure [length];[data]
StringBuffer headerStr = new StringBuffer();
StringBuffer data = new StringBuffer();
//read header
char headerByte = dataInputStream.readChar();
while (headerByte != ';') {
headerStr.append(headerByte);
headerByte = dataInputStream.readChar();
}
//header has the number of character to read
int header= Integer.parseInt(headerStr.toString());
int bytesReaded = 1;
char dataByte = dataInputStream.readChar();
//we should read the number of characters indicated in the header
while (bytesReaded < header) {
data.append(dataByte);
dataByte = dataInputStream.readChar();
bytesReaded++;
}
For the first query, I guess you are using TCP. If you use UDP, then the server will read the packets in the order you want.
Can you be more clear/elaborative on the second query ?
I would try explicitly telling Connector.open to open up the stream as read_write. Then I would ensure that I flush my connections after each time I talked to the server.
SocketConnection connection = (SocketConnection) Connector.open(url, Connector.READ_WRITE);
OutputStream out = connection.openOutputStream();
// ... write to server
out.flush()
I got a solution to overcome to extract both the string
On sender device
Create a header which contains details of that data eg the data
length, datatype etc.
Add this header to the actual data and send it
On recipient device
read the header
retrieve the actual data length from the header
read the next data upto the data length as specified by the header