A very simple question I can't seem to find a definitive answer for.
I have a classic TCP Indy server. I send data in chunks. Each data packet is send in 1 or more chunks. When it arrives the OnExecute is triggered one or more times for each packet. More then one client can send data at any given time. How do I know which client / packet am I receiving data for in the OnExecute? I know its a trivial question probably but I want to have a definitive answer.
If you can design the protocol, it could be done like this:
the client starts with a initial command which includes total size and the chunk size
the server OnExecute creates a temporary output file stream and stores file information in the context
the client sends the chunks
the server OnExecute reads chunks (using Indy TCP server blocking read methods with the known chunk length) and appends them to the output stream
The Indy TCP context class can be extended to add custom information for the client connection.
Related
I'm using mbedtls to run SSL over TCP. I'm using my own custom I/O functions for the interface between mbedtls and TCP, set using the mbedtls_ssl_set_bio call.
I've noticed that Application records are generated one-to-one with calls to mbedtls_ssl_write. For example, if I write data in 52-byte chunks, the client sends 52-byte records. If I write data in 520-byte chunks, then the client sends 520-byte records. The latter case is preferable; in the former case, a significant amount of bandwidth is lost to TLS headers.
My question is: is it possible to configure mbedtls to "coalesce" multiple small writes into a single record? For example, TCP does this (e.g., if you write data in 52-byte chunks to a TCP connection in quick succession, they do eventually get coalesced into MSS-sized segments). I'm wondering if it's possible to do something similar for TLS.
With TCP, when the write function returns, the data has written to the socket and is out of control of the application. The network stack can still coalesce packets if it wants. Either the sender, or a router, can merge successive TCP packets on a connection if the second packet arrives before the previous one can be sent.
With TLS, when the mbedtls_ssl_write function returns (or similar function in any TLS stack), the data has been encrypted-and-authenticated into a TLS record and the record has been sent over the socket (or whatever the underlying I/O function does). The TCP stack can still coalesce the TCP packets, but it can't coalesce TLS records because it doesn't have the necessary cryptographic keys.
If you want to coalesce data on a TLS connection, either you need to buffer it before you pass it to the TLS stack, or the TLS stack needs to buffer it itself. If you want the TLS stack to buffer it, you need an API with at least two functions: one to take some data as input and not try to send it immediately, and another one to say “send the data now!”. The first function would not really do anything except write the data into some temporary buffer, unless the buffer is full in which case it would need to flush it first. And that's something you can do yourself: just memcpy the data to a temporary buffer, and call mbedtls_ssl_write on that temporary buffer when you're ready to send. So there's not really an incentive for a TLS stack to provide this interface.
i have a client server application (TCP) that's designed with indy delphi.
i want to have a queue and flow control in my server side application.
my server should not lose any clients data when server traffic is full.
for example , in my server side application i want determine maximum of bandwidth for server is 10Mbps and then if server bandwidth (this 10Mbps) was full the other clients be on queue until bandwidth get free .
so i want to know how can i design this with delphi ?
thanks
best regard
The client should not send the message directly to the server. Put the message in a local store (f.i. sqlite-db) and in a thread you read the first message from the local store and try to send it to the server.
If the message was delivered to the server (no exception raised) delete the message from the local store and process the next "first" message in the local store.
Within the TIdTCPServer.OnExecute method which receives the client data, it is possible to 'delay' processing of the incoming request with a simple Sleep command. The client data will stay in the TCP socket until the Sleep command finished.
If your server keeps track of the current 'global' bandwidth usage for all clients, it is possible to set the Sleep time dynamically. You could even set different priorities for different clients.
So you would need a simple but thread safe bandwidth usage monitor, an algorithm which calculates sensible Sleep time values, and a way to assign this Sleep time to the individual client connection contexts.
See also:
https://en.wikipedia.org/wiki/Token_bucket
https://github.com/bandwidth-throttle/token-bucket for an example implementation in PHP
http://www.nurkiewicz.com/2011/03/tenfold-increase-in-server-throughput.html
Is it possible corba service response one request multiple times?
In my case, corba service collects a bunch of data which takes a long time for request. In order to reduce delay client receives response, we want service responses soon when the size of collected data reach 1024k. For example, the total data sizes is 10M, service responses client 10 times on one connection.
My understanding is that corba server should cache connection between client and server, and deliver new data on this cached one once a new data is available. The client, on other handle, should do while loop for incoming response. Either client or server should not close connection until server says all data is connected. This procedure is similar to that in chunked response in Http protocol.
I appreciate if you can provide some tips or sample links in this area.
The CORBA server side is only able to send the data to the client when the server application code returns the function call. If you have just one operation in IDL that returns 10M, than the ORB can only transmit that data to the client when the operation has finished. In order to allow the ORB to send the data it has you have to modify your IDL and add a way for the client to start the operation and than poll for chunks of data to be available. Each implementation of the poll than returns with one chunk.
Some examples of how to do this are part of TAO. You can find the examples under ACE_wrappers/TAO/examples/Content_Server. It also has an example where the server pushes the data to the client when a chunk is available.
First of all my intend is to create 2 programs server and client to send big byte array. I just started to test with UDPclient and udpserver components. I am not if it is right way.
I found program example but it was built with an old Delphi version, when I compile I am getting an error.
HostName.Caption := UDPServer.LocalName;
HostAddress.Caption := GStack.LocalAddress;
Port.Caption := IntToStr(UDPServer.DefaultPort);
BufferSize.Caption := IntToStr(UDPServer.BufferSize);
UDPServer.Active := True;
it seems udpserver.localname was in old versions. I need to find what property to use in here . How to get and write Host name in udpserver components. udpclient has Host property and workes fine, I could compile client program
Can anyone help me what to put instead of "localname" property
And if anyone can advice me another way to send big byte arrays. the size of it will be about 120000 . I will send it in every minute
Thank you
It's best to use TCP as it can detect data-corruption in your transmission and request the corrupted packets again. The detection happens by checking packets against a 16-bit checksum, this will only detect errors, but not allow the system to correct it, which is where requesting the data again comes in.
It's probably most convenient to use the Indy socket library that comes with Delphi to create TCP sockets. The TIdTCPServer and TIdTCPClient components should do the job nicely.
You can find some demo's on using the Indy library here:
http://www.indyproject.org/sockets/demos/index.en.aspx
http://www.atozed.com/indy/Demos/10/index.en.aspx
http://www.ciuly.com/delphi/indy/
GStack.HostName is the replacement for the old LocalName property.
UDP is a message-based transport. 120000 is too many bytes to fit in a single UDP message. You will have to break it up into small chunks. To send large amounts of data using UDP, consider using a UDP-based transfer protocol, such as TFTP. Indy has TIdTrivialFTP and TIdTrivialFTP components for that purpose.
Otherwise, switch to TCP, which is a stream-based transport. Then you can send as much data as you want.
It depends on the data you are trying to send over network. If you want to send a file over network, you should use TCP protocol because it guarantees that the data received at server-side will be the same sent on the client-side. If the data you want to send is lossy like Voice (I mean it's not important we have delays or losses in the middle and in data) you can use UDP protocol. It'll be very faster because it has no overhead for checking the data against corruption. In UDP you may loose your packets or datagrams so UDP is called a connection-less protocol because there is no connection in fact. I think all known programming languages support TCP and UDP connections.
I am using TIdCmdTCPClient and TIdCmdTCPServer. Suddenly I find that I might like to have bi-directional communication.
What would be best? Should I possibly use some other components? If so, which? Or should I kludge and have the 'client' poll the 'server' to ask if it wishes to communciate anything?
This is a very small system. Two clients and ten servers, with a burst of one tarnscation every 30 to 60 seconds for a few minutes once a day, so overhead for polling is inconsequential.
I'm just woder if there is a 'correct' way.
Update: this really is an incredibly simple system. Very little traffic and all of it simple. All transmissions are an indication of even type an an optional single parameter.
<event type> [ <parameter>] e.g. "HERE_IS_SOME_DATA 42"
This can be sent in both directions, hover here is no "reply" as such. Just fire off a message (and hope that it got there)? Receive an Ack with no data? Non-catching of an exception indicates that message was successfully sent?)
Would it be possible (would it be overkill) to use two TIdCmdTCPServer?
Both TIdCmdTCPClient and TIdCmdTCPServer continuously poll their socket endpoints for inbound data during the lifetime of the connection. You do not have to do anything extra for that. So, as soon as a TIdCmdTCPClient connects to the TIdCmdTCPServer, both components will initially be in a reading state until one of them sends a command to the other.
Now, there is a problem with doing that - as soon as either component sends that first command, the receiving component will interpret it as a command and send back a reply, which the other component will interpret as a command and send back a reply, which will be interpretted as a command and send back a reply, and so on, causing an endless cycle of replies back and forth. For that reason, it is not wise to use TIdCmdTCPClient and TIdCmdTCPServer together. You should either use TIdTCPClient with TIdCmdTCPServer, or use TIdCmdTCPClient with TIdTCPServer. Depending on what exactly your protocol looks like, you may have to forgo using TIdCmdTCPClient and TIdCmdTCPServer altogether and just use TIdTCPClient with TIdTCPServer so you have more control over reading and writing on both ends. It is hard to answer with actual code without first knowing what the communication protocol should look like.
A single TCP socket connection can be used in two directions. The server can send data asynchronously to the client at any time. It is up to the client however to read the socket, for asynchronous processing this is done in a listener thread which reads from the socket and synchronizes incoming data operations with the main worker thread.
An example use case in the Indy components is the Telnet client component (TIdTelnet) which has a receive thread listening for server messages.
But you also asked about the 'correct' way - and then the answer depends on other factors such as network stability, guaranteed delivery and how to handle temporary server outages. In enterprise environments, one central messaging hub is preferred in many use cases, so that all parties connect only to this central server which is only responsible for reliable message delivery, and keeps messages until the recipient is available.
You can download the INDY 10 TCP server demo sample code here.