How does a server knows what kind of packets his receiving ? (strcut,array...) ?
PS: i know dumb question
It doesn't - its just a bunch of bits/bytes. The application, using application level protocols, decides how to interpret those bits/bytes.
Just like memory is a bunch of bits/bytes - a pointer to a struct can be forced to point anywhere and the struct can used to read the memory, but the data may be nonsensical. Your application logic ensures (hopefully) that the struct pointer is only applied to memory that contains valid struct data. Similarly, your network application must decide how to interpret what the bits in the packets or TCP stream mean.
An application may use a well-known protocol to decide how to interpret those bytes. For example, the HTTP protocol indicates what the client should transmit and the server knows to interpret that data from the client according to how the HTTP spec. Regardless of what the client sends (e.g. if a gaming client accidentally sent a binary stream to the HTTP server), the HTTP server will nevertheless try to interpret the bits as a HTTP client request.
TCP servers receive a stream of bytes. Any interpretation beyond this is up to the application logic.
Related
I'm using mbedtls to run SSL over TCP. I'm using my own custom I/O functions for the interface between mbedtls and TCP, set using the mbedtls_ssl_set_bio call.
I've noticed that Application records are generated one-to-one with calls to mbedtls_ssl_write. For example, if I write data in 52-byte chunks, the client sends 52-byte records. If I write data in 520-byte chunks, then the client sends 520-byte records. The latter case is preferable; in the former case, a significant amount of bandwidth is lost to TLS headers.
My question is: is it possible to configure mbedtls to "coalesce" multiple small writes into a single record? For example, TCP does this (e.g., if you write data in 52-byte chunks to a TCP connection in quick succession, they do eventually get coalesced into MSS-sized segments). I'm wondering if it's possible to do something similar for TLS.
With TCP, when the write function returns, the data has written to the socket and is out of control of the application. The network stack can still coalesce packets if it wants. Either the sender, or a router, can merge successive TCP packets on a connection if the second packet arrives before the previous one can be sent.
With TLS, when the mbedtls_ssl_write function returns (or similar function in any TLS stack), the data has been encrypted-and-authenticated into a TLS record and the record has been sent over the socket (or whatever the underlying I/O function does). The TCP stack can still coalesce the TCP packets, but it can't coalesce TLS records because it doesn't have the necessary cryptographic keys.
If you want to coalesce data on a TLS connection, either you need to buffer it before you pass it to the TLS stack, or the TLS stack needs to buffer it itself. If you want the TLS stack to buffer it, you need an API with at least two functions: one to take some data as input and not try to send it immediately, and another one to say “send the data now!”. The first function would not really do anything except write the data into some temporary buffer, unless the buffer is full in which case it would need to flush it first. And that's something you can do yourself: just memcpy the data to a temporary buffer, and call mbedtls_ssl_write on that temporary buffer when you're ready to send. So there's not really an incentive for a TLS stack to provide this interface.
We know that a protocol is a rule that computers follow so that they can effectively communicate with each other.
But when we talk about rules, the question arises, what are these rules.?
For example, let's take the http protocol. What rules does it have on which computer follows?
I will now list some of the rules, and if it’s wrong, please correct me and tell me whether they are rules in general or not.
For example http protocol ...
this protocol dictates such rules ..
1.determines on which method (GET, POST and then) the request will be sent.
2. on which host the request will be sent.
3. what format will be requested file.
4.which data cookie will be sent to the server
5. on which port the request will be sent
etc....
when we talk about TCP protocol ...
this protocol dictates such rules ..
1.It records on which port the data will be sent.
2.data is divided into packets and a number is assigned to each packet so that the computer that receives the data will correctly arrange the packets.
3. When a packet is lost on the road, it has rules to send the request again to get the lost packet.
etc....
when we talk about the IP protocol ...
this protocol dictates such rules ..
1.This protocol provides source and destination ip addresses and decides on which routers packets should go. routers work at this level.
etc....
Please confirm if I understand correctly, do these rules represent a protocols or not??
Communication protocols need to define everything needed for two separate software developers to write two separate pieces of software that are able to communicate to each other using that protocol.
This includes things like the context in which the protocol is expected to operate, the various protocol states (e.g. the TCP state machine), the expected behavior in each state, how transitions between the states work, how messages are encoded, and what is the precise meaning of each message is.
Your examples are on the right track, but I suggest you skim at lest some of the initial RFCs for the protocol you mentioned like IP and TCP. They're fairly readable and will give you a better idea of what's need to define a protocol.
Is it possible to modify users' HTTP request to
www.example.com/options
instead of
www.example.com/options_and_params
My scenario is that about 30000 users connect to my company's network backbone and I want to add one or more server (with the code I'm current working on) between the backbone switches and Radware LoadProof to achieve this work.
After googling all the night, I have no lead but some more questions:
I don't need to intercept every packet through the network. With some helps like iptables, I can filter out the package I want. I have done it before using iptables. However, packet is not equal to HTTP stream. Do I need to do HTTP re-construct?
If I successfully find a way to modify HTTP request URL content, I still should put it back to network stream. As I know TCP packets have a checksum and after I modify the content it must be wrong. How do I calculate a new checksum and put the packet back to network?
It's my first time to do network programming or packet processing develop. Any suggestion is appreciate.
This depends on whether you are doing HTTP/1.0 or HTTP/1.1 and whether its an initial request you need to modify or all requests in a single HTTP 1.1 session.
If you have the packet and can modify it before it is sent on and you are trying to modify just the request then given the length of a typical packet and the location of the URL in the HTTP request stream (very near the beginning) and the fact that it will be the first thing sent in the TCP stream I think you can fairly safely assume that it will be present in the first N bytes of the first packet sent and therefore won't be split over multiple packets.
However, if this is an HTTP/1.1 stream then multiple requests will be being sent via the same TCP connection in which case in future requests the URL may well be split over two TCP packets.
If you can maybe force HTTP/1.0 or possibly if you modify the initial or all requests to be HTTP/1.0 then you can be pretty sure that the first packet will correspond to the first packet of the TCP stream and that you are very unlikely to see the URL split over multiple packets, meaning no reconstruction and the ability to just do a replace.
However this will come at a cost of new TCP connections which is pretty inefficient.
If you don't and you leave it as HTTP/1.1 then the URL could be at any random point in any future request and therefore split over multiple TCP packets (two realistically given the size of the URL).
If I got your question right, then this could be probably done with some fast reverse-proxy like nginx.
First of all my intend is to create 2 programs server and client to send big byte array. I just started to test with UDPclient and udpserver components. I am not if it is right way.
I found program example but it was built with an old Delphi version, when I compile I am getting an error.
HostName.Caption := UDPServer.LocalName;
HostAddress.Caption := GStack.LocalAddress;
Port.Caption := IntToStr(UDPServer.DefaultPort);
BufferSize.Caption := IntToStr(UDPServer.BufferSize);
UDPServer.Active := True;
it seems udpserver.localname was in old versions. I need to find what property to use in here . How to get and write Host name in udpserver components. udpclient has Host property and workes fine, I could compile client program
Can anyone help me what to put instead of "localname" property
And if anyone can advice me another way to send big byte arrays. the size of it will be about 120000 . I will send it in every minute
Thank you
It's best to use TCP as it can detect data-corruption in your transmission and request the corrupted packets again. The detection happens by checking packets against a 16-bit checksum, this will only detect errors, but not allow the system to correct it, which is where requesting the data again comes in.
It's probably most convenient to use the Indy socket library that comes with Delphi to create TCP sockets. The TIdTCPServer and TIdTCPClient components should do the job nicely.
You can find some demo's on using the Indy library here:
http://www.indyproject.org/sockets/demos/index.en.aspx
http://www.atozed.com/indy/Demos/10/index.en.aspx
http://www.ciuly.com/delphi/indy/
GStack.HostName is the replacement for the old LocalName property.
UDP is a message-based transport. 120000 is too many bytes to fit in a single UDP message. You will have to break it up into small chunks. To send large amounts of data using UDP, consider using a UDP-based transfer protocol, such as TFTP. Indy has TIdTrivialFTP and TIdTrivialFTP components for that purpose.
Otherwise, switch to TCP, which is a stream-based transport. Then you can send as much data as you want.
It depends on the data you are trying to send over network. If you want to send a file over network, you should use TCP protocol because it guarantees that the data received at server-side will be the same sent on the client-side. If the data you want to send is lossy like Voice (I mean it's not important we have delays or losses in the middle and in data) you can use UDP protocol. It'll be very faster because it has no overhead for checking the data against corruption. In UDP you may loose your packets or datagrams so UDP is called a connection-less protocol because there is no connection in fact. I think all known programming languages support TCP and UDP connections.
1)
Now I am writing IM chat System i face some problem how to send vector that has information from the server to Client
2)
is any way to communicate between tow client ??
I Use CBC2010 - Indy10
Basically communicating over TCP is about sending bytes from client to server, and receiving bytes on the client from the server.
You either can give meaning to those bytes, or have something wrap that for you.
There are many possibilities and protocols to choose from.
On the foundation, you have either UDP (which is unreliable, but incurs almost no overhead, but very well suited for broadcasts) and TCP (which is more reliable, therefore has more overhead, but is easier to use).
A transport protocol that is often used on top of TCP is HTTP, especially since it is easy to get it through proxy servers.
On top of that you can do XML+SOAP or JSON+REST, which make translating from/to your underlying objects a lot easier.
All in all there are a truckload of options to choose from.
A simple start is the Delphi chat example at delphi.about.com. That definitely should get you going.