how to specify reuseaddr socket option on thrift client? - network-programming

I am working with thrift-python now. And After I checked source code of that, I found that on server end, thrift setsockopt(reuseaddr, 1) by default. It's helpful for Ctrl + C and restart server to debug.
But now I have too many clients at localhost, they use short connections to communicate with server. And when they close connecitons, those random ports are exactly in TIME_WAIT status! so they could cover ports which I want to run another thrift server on it. So, How could I specify REUSEADDR option on thrift client end?

I have figure out some detail about thrift,
on the server side, the REUSEADDR option is set by default;
on the client side, it seems that we couldn't specify that option on underlay TCP connections;
on the client side, we can use long connections instead of short connections to avoid too many TIME_WAIT. To do this, we need call transport.open() just once;
on the server side, we can use TForkingServer instead of ThreadedPoolServer;

Related

What's the upper bound connections of TServerSocket in Delphi? [duplicate]

I'm building a chat server with .NET. I have tried opening about 2000 client connections and my Linksys WRT54GL router (with tomato firmware) drops dead each time. The same thing happens when I have several connections open on my Azureus bit-torrent client.
I have three questions:
Is there a limit on the number of open sockets I can have in Windows Server 2003?
Is the Linksys router the problem? If so is there better hardware recommended?
Is there a way to possibly share sockets so that I can handle more open client connections with fewer resources?
AS I've mentioned before, Raymond Chen has good advice on this sort of question: If you have to ask about OS limits, you're probably doing something wrong. The IP protocol only allows for a maximum of 65535 ports and many of these are reserved and not available for general use. I would suggest that your messaging protocols need to be thought out in more detail so that OS limits are not an issue. I'm sure there are many good resources describing such systems, and there are certainly people here that would have good ideas about it.
EDIT: I'm going to put some thoughts about implementing a scalable chat server.
First off, designate a single port on the server for clients to communicate through. Whenever a client needs to update the chat state (a new user message for example) do the following:
create message packet
open port to server
send packet
close port
The server then does the following:
connection request received
get packet
close connection
process packet
for each client that requires updating
open connection to clients
send update packet
close connection
When a new chat session is started, the client starting the session sends a 'new session' message to the server with the clients user details and IP address for responses. The server creates a new chat session and responds with the session ID. The client then sends packets containing the messages the user types, the server processes them and forwards the message to other clients in the same session. When a client leaves the chat, it sends a 'end session' message to the server. The server removes the client from the session and destroys the session when there are no more clients in the session.
Hope that gets you thinking.
i have found some answers to this that i feel i should share:
Windows 2003 server has a limit on the number of ports that may be used. but this is configurable via a registry tweak to change the MaxUSerPort setting from 5000 to say, 64k( max).
Exploring further, i realize that the 64k port restriction is actually per IP address, hence a single server can easily attain much more ports, and hence TCP connections by either installing multiple network cards, or binding more than one IP address to a network card. that way, you can scale your system to handle n x 64k ports.
Had for days a problem with the available sockets on my Window 7 machine. After reading some articles about socket leaks in Win 7, I applied a Windows patch - nothing changed.
Below there is an article describing windows connection problems in great detail:
http://technet.microsoft.com/en-us/magazine/2007.12.network.aspx
For me it worked the following:
Open Regedit
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters: Create TcpNumConnections, REG_DWORD, decimal value 500 (this can be set according to your needs); EnableConnectionRateLimiting, REG_DWORD, value 0;
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip: Create MaxUserPort, REG_DWORD, decimal value 65534
Restart Windows

Should I be afraid to use UDP to make a client/server broadcast talk?

I spent the last two days reading each StackOverflow questions and answers (and googling of course) about Indy TCP and UDP protocol in order to decide which one should I use in my communication method between my User Application and my Windows Service.
From what I saw so far, UDP is the easiest and the only one I managed to work to receive broadcast messages from TidUDPClient (I did not testes the response back yet). And I also noticed that TCP is a bit more complicated with it's thread loop.
But since everywhere I am told UDP is not reliable, UDP is not reliable... I begin to wonder if it's not better to use TCP anyway.
My User Application will be running on many machines, and the Service will be running in one of them, sharing one IP with a Client, or in a dedicated machine, depending on my client's funds. So, should I really be worried about UDP data loss possibilities?
I need broadcast capabilities so my server advises all clients at once about Application updates, and of course, if my the Client Application does not know in which IP the Service/Server is, it will send a broadcast call to be told where the server is. Is that applicable to TCP?
The messages I am sending are requests for users access confirmation, users privileges, and application executable file updates, since the main application can't update itself.
Those messages are encrypted like below, and they might bet bigger sometimes.
e86c6234bf117b97d6d4a0c5c317bbc75a3282dfd34b95446fc6e26d46239327f2f1db352b2f796e95dccd9f99403adf5eda7ba8
I decided to use them both!
Simple use case:
In order to communicate with TCP prococol you have to establish a connection which you can have only if you know IP and Port on both ends.
If you do not have that information when you load your Application, then you use the UDP to Broadcast your IP address and your intention to find the/a Server. You may try about 5 times before you raise the user an error telling that you did not find the Server or that the Server is down.
Sending that message in UDP will (one time or other) reach the UDP ear of the Server, which will now know the IP from the lonely Client's IP and will now begin a proper connection via TCP to be read talk about the critical messages of the Application.
What do you think of that approach?

Asyncronously send file over TCP connection

so I'm making an iOS app, but this is more of a general networking question.
So what I have is one phone that acts as the server and then a bunch of phones connect to the phone as the client. Basically it's a game/music sharer.
It's kind of hard to really get into the semantics of it, but that isn't important.
What is important is that the server and client are repeatedly sending each other commands and positions rapidly over a TCP connection, and sometimes the client wants to send the server a music file (4MB usually) to play as the music.
The problem I initially encountered was that when sending the large file, it would hang the sending of commands from the client to the server.
My naive solution was to create another socket to connect to the server to send the file to the server, the server would check the IP of the new socket, and if it has the IP of an existing connection then it would just tie it to that connection, receive the file, and then disconnect the socket.
But the problem with this is that it takes a 1-2 second delay for the socket to connect, and I'm aware that there are man-in-the-middle attacks that can occur.
Is there a more elegant solution to this problem?
I would not call your solution naive, this is largely how FTP works, separating data and control paths is a good design pattern in my view.
I wouldn't worry about the man in the middle thing. If you wanted, you could add a command to the client that it responds to over the data connection with a secret the server supplies, this would let you associate the connections without using the ip addressing.
If the delay is a problem then why not establish both connections at the start, the overhead of a few tcp connections on an operating system is not usually significant.
You could also use the two connections for both commands and data, alternating between them. Since both the server and client know when a connection is busy they can choose to use the idle one. The advantage of this is that it will keep both connections busy to ensure they are both known to be working.
You probably should also use a different thread for each socket but I suspect you are doing this since it won't work too well without it.

Server separate connection per client

The simple way to support multiple clients is to fork a process for each incoming connection and handle the client through that process and the other process then listens on the default port.
However I have heard that not only is the process forked, but the connection itself is redirected from the default port at which the server is listening to a different port on the server side. This improves performance. I am unable to find any reliable reference to this topic. Is this a standard procedure
This is not the case for most protocols on the internet. For example, public web servers will always listen on port 80 for every request. FTP servers on port 21 and so on. Where did you get your information? With more information on the context, someone might be able to help establish the circumstances you're talking about. What's your source?

Delphi transparent proxy for a TCP connection

Anyone know of any examples of a TCP sockets proxy application written in Delphi? I am building a small broker application that needs to listen for socket connections on a given TCP port, read a XML data packet sent over the connection, serve the request via TCP to a server chosen from a pool of available back end servers, and deliver the resulting response back to the originating TCP connection.
I am very familiar with TCP and socket-level programming, but would prefer to find sample code as a starting point. If there is something available it would save considerable time and effort since I would not need to reinvent the wheel. I recall seeing a small TCP proxy written in Delphi back in 2003/2004, but seem to have lost track of the link.
Are you looking for a socks proxy implementation? If not, then just start with a server (http would most likely be a good start) and build into it the ability to process your inbound XML data packet, and make the appropriate calls. Something like the Synapse framework would make that a fairly simple project. If you grab Synapse, get the latest version from SVN. It supports all of the latest versions of Delphi.
There is a httpproxy demo for synapse available, that might help if your wanting to implement something more traditional. A good starting http server example is also available.
IPWorks from nSoftware should be considered as a commercial alternative to Synapse. They also have some Biztalk adapters available.
I've used both Synapse and IPWorks, and they are both quite good. I haven't used the referenced BizTalk adapters.
BizTalk Adapters
IPWorks

Resources