I try to use influxDB to save some monitor data. I started influxDB with default configuration and used python to write some test data through HTTP API in my own mac, everything going well in the beginning. But I encountered a ConnectionError every time after I wrote some data. My test code is
for event_id in xrange(0, 100000):
requests.post("http://127.0.0.1:8086/write?db=mydb", data="test_a,event={0} value=100 1538841600000000000".format(event_id))
After about 16300 executions, an error will be reported.
HTTPConnectionPool(host='127.0.0.1', port=8086): Max retries exceeded with url: /write?db=mydb (Caused by NewConnectionError("<urllib3.connection.HTTPConnection object at 0x110a5fb10>: Failed to establish a new connection: [Errno 49] Can't assign requested address",)
And then, I should wait about 2 seconds before I write new data.
My OS is MacOS 10.14, and all data is saved in an HDD hard disk.
influxDB version is vv1.6.4 (git: master c75cdfdfa6f71a08473fefcec71f6cbcbdef1ff4)
Is influxDB doing some tidying up?
OK, I found the reason, because this API is HTTP protocol and requests use TCP connection to send HTTP request, so every time a request finished, the TCP connection not closed, it will goto TIME_WAIT and wait a timeout before close. So the port resource of my OS are exhausted......
Related
I want to download a file from FTP. If the file is small (usually under 1000MB) it works. However, if the file is big I get an EIdReadTimeout. Why? Should I keep the connection alive? From what I know reading data has its own channel so I don't have to keep the connection alive.
What is odd is that the exception appears at the end of the Get (after Get successfully downloads the whole file): FTP.Get(Name, TempGzFile, TRUE, FALSE) !!!!
Documentation:
TIdFTP.ReadTimeout - Number of milliseconds to wait for an FTP protocol response.
TIdFTP.TransferTimeout - Timeout value for read operations on the data channel for the FTP
client.
By default ReadTimeout is set to 60sec and TransferTimeout to 10sec.
I a using Delphi XE7 (which I guess uses Indy 10). The Passive property for my IdFTP is set to false.
The FTP protocol uses multiple TCP/IP connections - one for the main command/response connection, and separate connections for data transfers. While a data transfer is in progress, the main command connection sits idle. Once the transfer is finished, the command connection receives a response.
If you are passing through a router/firewall that is not FTP-aware, the command connection is likely to get killed if it sits idle for too long during a large transfer. The connection is usually not killed "gracefully", so even the OS does not know the connection is gone. When TIdFTP then tries to read a transfer response that never arrives, it times out.
To account for that, use the TIdFTP.NATKeepAlive property to enable TCP/IP level keep-alives on the command connection during transfers. Set NATKeepAlive.UseKeepAlive to True, and set NATKeepAlive.IdleTimeMS (the idle timeout before keepalives start sending) and NATKeepAlive.IntervalMS (the interval between each keepalive) to suitable values.
Note, however, that IdleTimeMS and IntervalMS are only implemented for Windows 2000+, Linux, and BSD at this time. Other platforms use defaults provided by the OS (which tend to be very large). If you need to customize the values on those platforms, you can use the TIdFTP.OnDataChannelCreate and TIdFTP.OnDataChannelDestroy events to call TIdFTP.Socket.Binding.SetSocketOption() directly as needed.
I have an OSB application that communicates with a web service. I'm sending requests with the test console. The result returned to the test console is empty. How can I increase the time while the application is waiting for a response?
There are two parameters for connection. Connection Timeout & Read Timeout. Make sure, you are putting these value optimal.
(1) Connection Timeout - Timeout for making TCP connection to end-point. (usually 2 sec to 5 seconds.)
(2) Read Timeout - Timeout for getting & reading response from server/endpoint after successful TCP connection. (This should be based on SLA for interface contract.)
Thanks, Anil
If you want to increase business service response timeout, in OEPE (Eclipse).
Please select that Business Service -> HTTP Transport -> (Read Timeout or Connection Timeout).
Update it to appropriate value.
By default its value 0 (zero), means no timeout.
Is redis client using the long connection? If it wasn't long connection, why not use long link to reduce the cost of establishing a connection
Redis clients use TCP connection which is persisted until either side terminates it therefore it's up to the client library how the connection will be handled. I assume most of the clients would try to leave the established connection (or multiple connections in case of a pool) open during the lifetime of application where they are used in order to prevent handshaking before each executed command.
I am using TClientSocket and TServerSocket to comunicate with a server the problem is that sometimes connection is lost either by the server issuing me the following exceptions : Error on WsaSend, acess violation etc or by the Client : Asychronious socket error.
Witch is the best method to recover from these errors and keep the connection open no mather what ?
There is no such thing as "keeping the connection open no matter what". What if the cable gets cut? The best you can do is to send a heartbeat on some interval to let intermediate routers know you are still interested in using that connection, and to carefully handle all errors, and, if necessary, re-establish the connection.
Great question... what you're receiving is WSAECONNABORTED (Asynchronous Socket Error 10053).
How did i prevent it from happening in MY code ? well, there's something called Keepalive, if you look carefully into the name, Keep-Alive, it meant to keep the connection alive, just send Null data to the connection (Can be One-way), that's all...
i made a Timer (named it TmrKeepAlive) and set its interval to 5000ms (5 seconds), More info on KeepAlive.
Edit: Also, if you don't want to write your own KeepAlive mechanism, check this out
When I call the function
IdFtp.List(myList, '', false);
afterwards I have logged in and changed the ftp directory, I get a socket-error #10054 exception ("Connection reset by peer.") occesionally.
When I call that function e.g. 20 times consecutively I get that exception 1 time.
That problem I have only encountered on Vista OS.
Does anybody know what the problem is or how I can avoid it?
Not much you can do about this, because the disconnection is done by the FTP server.
You have a few choices:
Increase (or disable) the timeout settings (every FTP server has a different name for it) on your FTP Server connection settings.
Tell server that you are alive by sending NOOP command periodically (switching to Passive mode can also help).
Catch that exception and reconnect silently (This is my preferred solution because we have many FTP servers and I don't trust the sys-admins to change the FTP server time-out settings).
Here is a screen-shot from FileZilla FTP server time-out settings:
Note that with the above settings, the FTP client will be disconnected after 2 min of non-activity.
setting that value to 0, will disable the time-out.
The FTP protocol uses multiple socket connections. Every time you call List(), a new socket connection is established to transfer the requested listing data. It sounds like the FTP server is not always closing the socket correctly at the end of a transfer.
In the component "IdFTP", change the following properties:
"Passive" = "False"
"TransferType" = "ftASCII"