I'm building a chat server with .NET. I have tried opening about 2000 client connections and my Linksys WRT54GL router (with tomato firmware) drops dead each time. The same thing happens when I have several connections open on my Azureus bit-torrent client.
I have three questions:
Is there a limit on the number of open sockets I can have in Windows Server 2003?
Is the Linksys router the problem? If so is there better hardware recommended?
Is there a way to possibly share sockets so that I can handle more open client connections with fewer resources?
AS I've mentioned before, Raymond Chen has good advice on this sort of question: If you have to ask about OS limits, you're probably doing something wrong. The IP protocol only allows for a maximum of 65535 ports and many of these are reserved and not available for general use. I would suggest that your messaging protocols need to be thought out in more detail so that OS limits are not an issue. I'm sure there are many good resources describing such systems, and there are certainly people here that would have good ideas about it.
EDIT: I'm going to put some thoughts about implementing a scalable chat server.
First off, designate a single port on the server for clients to communicate through. Whenever a client needs to update the chat state (a new user message for example) do the following:
create message packet
open port to server
send packet
close port
The server then does the following:
connection request received
get packet
close connection
process packet
for each client that requires updating
open connection to clients
send update packet
close connection
When a new chat session is started, the client starting the session sends a 'new session' message to the server with the clients user details and IP address for responses. The server creates a new chat session and responds with the session ID. The client then sends packets containing the messages the user types, the server processes them and forwards the message to other clients in the same session. When a client leaves the chat, it sends a 'end session' message to the server. The server removes the client from the session and destroys the session when there are no more clients in the session.
Hope that gets you thinking.
i have found some answers to this that i feel i should share:
Windows 2003 server has a limit on the number of ports that may be used. but this is configurable via a registry tweak to change the MaxUSerPort setting from 5000 to say, 64k( max).
Exploring further, i realize that the 64k port restriction is actually per IP address, hence a single server can easily attain much more ports, and hence TCP connections by either installing multiple network cards, or binding more than one IP address to a network card. that way, you can scale your system to handle n x 64k ports.
Had for days a problem with the available sockets on my Window 7 machine. After reading some articles about socket leaks in Win 7, I applied a Windows patch - nothing changed.
Below there is an article describing windows connection problems in great detail:
http://technet.microsoft.com/en-us/magazine/2007.12.network.aspx
For me it worked the following:
Open Regedit
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters: Create TcpNumConnections, REG_DWORD, decimal value 500 (this can be set according to your needs); EnableConnectionRateLimiting, REG_DWORD, value 0;
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip: Create MaxUserPort, REG_DWORD, decimal value 65534
Restart Windows
Related
Short version: How would you recommend going about connecting a client to a server that are on the same local network, without manually entering the ip, when broadcast is disabled?
Further details: I am working on an educational multiplayer game for children. Many schools appear to be blocking broadcasting for security reasons. The children will be rather young, so it could be difficult for them and error-prone to have to enter the IP manually. They will all be in the same room and will all see the server screen. The game is made in Unity (C#).
Potential solutions: Here's what I thought about:
Connecting both the local server and clients to an external server, communicating the local server ip through the clients through the external server, then connecting directly and disconnect from the server. Not ideal because of the extra hosting costs.
Send a regular UDP message periodically to all ips on the subnet? This will probably be picked up by any decent firewall and blocked though, right?
Putting a QR code on the server that kids would take a picture of with the client app and have it connect that way? May be more of a hassle.
Having the server play random tones corresponding to numbers that the client is listening for? (Speakers may not always work though)
Sounds like the first one is the most sane and easy solution. Do you have any other ideas on what someone in this situation could try?
Is UDP multicast possible?
If yes then a common solution is that all participants join the same multicast group and the server listens on a well known port. If a client wants to know the address of the server it sends a packet to the multicast group, which is received by the client and answered with another packet, which then can be used by the client to determine the servers address.
In addition to that servers can also announce their presence in regular intervals by sending a suitable message to the multicast group.
What I can think of is an ad-hoc communication protocol between all the devices. Say you have 1 server and 10 clients. All the devices should run a service(say server-discovery) that binds to a fixed port say 9999. Now any time the client wants to connect to the server and doesn't know the IP, it starts a scan. Loops through different IPs and tries to connect to 9999. If it manages to hit, it asks for the server IP. In case it manages to hit the server it will get the IP since the server knows it's own IP and the client will maintain the server IP in a cache. If the client hits another client. It can ask for the server IP. It the other client knows the IP it will share the info else decline.
I agree there is a lot of overhead, but I think this will be robust unlike sound and would reduce cost of printing QRs everytime.
on the local network the traffic is direct from host to host.
I don't understand which devices is blocking local broadcasts.
if there aren't too many peers on the LAN ( less then 100 ), I think udp broadcasts work fine and you dont pollute the network.
to have an idea of your "pond", I suggest you to sniff your local traffic.
there are many broadcasts : arp, windows, ipp, dropbox...
I spent the last two days reading each StackOverflow questions and answers (and googling of course) about Indy TCP and UDP protocol in order to decide which one should I use in my communication method between my User Application and my Windows Service.
From what I saw so far, UDP is the easiest and the only one I managed to work to receive broadcast messages from TidUDPClient (I did not testes the response back yet). And I also noticed that TCP is a bit more complicated with it's thread loop.
But since everywhere I am told UDP is not reliable, UDP is not reliable... I begin to wonder if it's not better to use TCP anyway.
My User Application will be running on many machines, and the Service will be running in one of them, sharing one IP with a Client, or in a dedicated machine, depending on my client's funds. So, should I really be worried about UDP data loss possibilities?
I need broadcast capabilities so my server advises all clients at once about Application updates, and of course, if my the Client Application does not know in which IP the Service/Server is, it will send a broadcast call to be told where the server is. Is that applicable to TCP?
The messages I am sending are requests for users access confirmation, users privileges, and application executable file updates, since the main application can't update itself.
Those messages are encrypted like below, and they might bet bigger sometimes.
e86c6234bf117b97d6d4a0c5c317bbc75a3282dfd34b95446fc6e26d46239327f2f1db352b2f796e95dccd9f99403adf5eda7ba8
I decided to use them both!
Simple use case:
In order to communicate with TCP prococol you have to establish a connection which you can have only if you know IP and Port on both ends.
If you do not have that information when you load your Application, then you use the UDP to Broadcast your IP address and your intention to find the/a Server. You may try about 5 times before you raise the user an error telling that you did not find the Server or that the Server is down.
Sending that message in UDP will (one time or other) reach the UDP ear of the Server, which will now know the IP from the lonely Client's IP and will now begin a proper connection via TCP to be read talk about the critical messages of the Application.
What do you think of that approach?
so I'm making an iOS app, but this is more of a general networking question.
So what I have is one phone that acts as the server and then a bunch of phones connect to the phone as the client. Basically it's a game/music sharer.
It's kind of hard to really get into the semantics of it, but that isn't important.
What is important is that the server and client are repeatedly sending each other commands and positions rapidly over a TCP connection, and sometimes the client wants to send the server a music file (4MB usually) to play as the music.
The problem I initially encountered was that when sending the large file, it would hang the sending of commands from the client to the server.
My naive solution was to create another socket to connect to the server to send the file to the server, the server would check the IP of the new socket, and if it has the IP of an existing connection then it would just tie it to that connection, receive the file, and then disconnect the socket.
But the problem with this is that it takes a 1-2 second delay for the socket to connect, and I'm aware that there are man-in-the-middle attacks that can occur.
Is there a more elegant solution to this problem?
I would not call your solution naive, this is largely how FTP works, separating data and control paths is a good design pattern in my view.
I wouldn't worry about the man in the middle thing. If you wanted, you could add a command to the client that it responds to over the data connection with a secret the server supplies, this would let you associate the connections without using the ip addressing.
If the delay is a problem then why not establish both connections at the start, the overhead of a few tcp connections on an operating system is not usually significant.
You could also use the two connections for both commands and data, alternating between them. Since both the server and client know when a connection is busy they can choose to use the idle one. The advantage of this is that it will keep both connections busy to ensure they are both known to be working.
You probably should also use a different thread for each socket but I suspect you are doing this since it won't work too well without it.
I want to write a app which will run on different computers and need all of then to communicate with each other like "utorrent" (peer to peer). This app only will send text messages.
How can I do this? I mean sending one message to remote computer on the internet?
I have a website and every app at start can send some information to it and find information of other apps on other computers (with PHP) but I do not know how address one computer through internet and send the data directly to that. I can find the ip address with PHP but it is the ip address of router (ISP).
How a message reaches a computer? I'm wondering about addressing every computer?
My brain really stuck here, I really appreciate any help. Thanks.
In a peer-to-peer network there's no centralized server for transmitting the data from one client to another, in this case the clients must be able to act as both the server and client. This means that either you'll have to be using UPnP like most modern torrent clients, which handles port forwarding in the router, or you'll have to manually forward a port to the computer in the router.
A centralized server (like a torrent tracker) is usually used to make the clients aware of each other's existence and tell them where to connect. This is where your PHP script comes in, though PHP might not offer the most effective way of doing this, assuming you're using it in combination with a webserver to serve the data though the http protocol.
As for actual text communication, you could use the Indy socket library for that. I found this example, basically which shows how to do it: http://www.ciuly.com/delphi/indy/indy-10-client-server-basic-demo/
Hi
let me make my question clear. Two people using my app are connected to the internet. Both have each other's IP and they want to chat (like Y!messanger) with each other.
I think I need to use Indy components; right? Which component should I use?
Thanks in advance
Have you looked at any of the demos on Indy's website yet?
In general, you are looking to create a "Client/Server" type application. A quick Google search for "indy client server example" pulls up lots of results, including this one: http://www.devarticles.com/c/a/Delphi-Kylix/A-Real-World-Client-Server-Application-in-Delphi/
In reality, this gets a lot more complicated when you have firewalls and NATs with private IP addresses. You will have to consider how your application will either get around or through these types of technologies.
Similar to what Scott said, I think that your biggest problem is getting them talking to each other. My computers at home go through a router, which blocks all incoming connection requests (i.e. requests to start a conversation between two computers) from the Internet. My computers can send connection requests OUT, and start a conversation that way, but unless you modify the router (port forwarding) my computers can not receive connection requests.
You need a server somewhere to which both people will connect, that can then relay messages back and forth. To get really tricky, once the connection is made to the server the two computers can then be put into direct contact, but that involves UDP packets and some clever magic.
You don't have to use Indy components, you just need anything that will handle communications over the network. Any HTTP or sockets network stack will do. Indy is the defacto standard for Delphi Win32.
To do network communications, you will need to create a listener object or service on machine A and a sender object on machine B to send a network message from A to B. To send a message from B to A, you will need a reverse path as well - 4 objects total to perform bidirectional comms. Some object wrappers hide this detail internally. I don't recall offhand whether Indy hides this or not.
It would probably be easiest if you use a common TCP/IP protocol for your machine to machine communications, such as HTTP. This will make it easier to get your connections through firewalls and proxies that frequently exist between arbitrary users. To avoid conflicting with any HTTP web services that might be running on either machine, you should use a custom port number with the IP address: 192.168.1.10:12345, not the standard HTTP web server port 80. This is what most of the IM clients do.