I am working oncan application which is to push records to azure table & blobs. My application ran for around 8 days perfectly fine but then it started giving connection time out error related to blob. Can anyone please guide to to workaround this?
Error logs below :
"java.net.ConnectException: Operation timed out (Connection timed out)\n\tat java.base/java.net.PlainSocketImpl.socketConnect(Native Method)\n\tat java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)\n\tat java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)\n\tat java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)\n\tat java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)\n\tat java.base/java.net.Socket.connect(Socket.java:609)\n\tat java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:299)\n\tat java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)\n\tat java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)\n\tat java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)\n\tat java.base/sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:266)\n\tat java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:373)\n\tat java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:203)\n\tat java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)\n\tat java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)\n\tat java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:189)\n\tat java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1592)\n\tat java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)\n\tat java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527)\n\tat java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334)\n\tat com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:115)\n\tat com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:744)\n\tat com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:731)\n\tat com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:705)\n\tat
To resolve the error "java.net.ConnectException: Operation timed out (Connection timed out)" try below suggestions if helpful:
Try setting a small timeoutInterval and use a large maximumExecutionTime in your blobRequestOptions.
If the above does not work, use fiddler to verify that you are sending and receiving a response as expected.
Try checking the code you are using to upload files to Azure blob.
Check the system configuration and make sure there is no firewall that blocks the request from Java.
Connection timeouts usually occurs if there are too many requests overloading the server.
Otherwise, try uploading blob in chunk as a workaround.
If still the error persists, try using code snippet mentioned in this link.
Check whether the IP address/domain and port are incorrect or down.
For more information in detail, please check below references:
java - Azure StorageException: An unknown failure occurred : Connection timed out: connect - Stack Overflow
java - What could cause socket ConnectException: Connection timed out? - Stack Overflow
Why would a "java.net.ConnectException: Connection timed out" exception occur when URL is up? - Stack Overflow
Related
Last few days new difficulties connecting some sites but biggest problem example is “Google lens”. Messages include connection timed out and/or io.GRPC/14. Can I fix this, or anywhere to do so?
I am having an error when trying to change an iFrame's location.
WebSocket network error: The operation cannot be completed. (kCFErrorDomainCFNetwork Code 2)
I am having this exception on Safari Mobile (iPad). This error does not occur on Safari Desktop.
kCFErrorDomainCFNetwork Code 2 is kCFHostErrorUnknown. But, what does this mean and how can I fix this?
According to the documentation,
kCFHostErrorUnknown
An unknown error occurred (a name server failure, for example). For additional information, you can query the kCFGetAddrInfoFailureKey key to obtain the value returned by getaddrinfo and look up the value in /usr/include/netdb.h.
Where can I query the kCFGetAddrInfoFailureKey? Also, I don't think I can look up netdb.h.
I found the issue. Since I am using IP Address to access the server. The WebSocket server should also be IP Address.
I am developing a service which reads messages from SQS, does some processing and posts the result to a different service.
When I initially developed this, the service was working perfectly fine, however after some days (about 5 days) I got this exception:
com.amazonaws.http.AmazonHttpClient: Unable to execute HTTP request: Connection to https://sqs.us-east-1.amazonaws.com refused { org.apache.http.conn.HttpHostConnectException: Connection to https://sqs.us-east-1.amazonaws.com refused
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:158)
at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:149)
at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:561)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:280)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:165)
at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:869)
at com.amazonaws.services.sqs.AmazonSQSClient.getQueueAttributes(AmazonSQSClient.java:453)
I reactivated the service and it started working again. My question is has someone faced this issue? If so what is the best way to handle such exception?
Other interesting fact is the same exception happened on all the hosts of the service at nearly the same time. Can this be because of an SQS outage or are such transient connection failures expected?
I am trying to build a java application to scrape a website using HTMLUnit. After extracting some data the application encounters following exception -
java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connection to siteURL refused.
If I run application again, it is able to extract some data again before failing with the same exception. Probably the server see lot of requests from same client IP and refuses connection for a request.
Also, when application encounters this problem, I am able to connect to the site using a browser.
How can I overcome this problem? In web scraping applications how are such problems approached and resolved?
This is how I debug such issues :
Download Fiddler
By default, fiddler listens on port 8888, all you have to do, is configure webClient to use fiddler as proxy & then all requests being sent can be seen (analyzed, modified & re-sent too) in fiddler.
client.getOptions().setProxyConfig(new ProxyConfig("127.0.0.1", 8888));
From what I can say from my previous experience is that the target site blocks after some-time, you can try adding a pause or rotating proxies & user-agents. You can also try clearing cookies.
I have facebook apps with flask with nginx and uwsgi. When it receive POST from facebook, it always has error:
readv() failed (104: Connection reset by peer) while reading upstream
But when I access my apps directly (with GET method), it ran smoothly. What I've done:
Limit #app.route with POST method only - doesn't work.
Add limit in wsgi: uwsgi_buffer_size (in case request from facebook is big), and uwsgi_harakiri (in case uwsgi provide timeout before finish it request) - doesn't work.
I have workaround in django but can't figure out yet how to implement in flask. Could anyone help please?
This is the bug of uwsgi. You can get more from [uWSGI] Several bugs .
The simple solution is that you must read the POST body by wsgi.input, even through POST body is null or you don't need POST params.
The issue is that "upstream" (the actual process that nginx is proxing) is closing the connection.
In my case, Django is my web server and I needed to set DATA_UPLOAD_MAX_NUMBER_FIELDS to be larger because there were too many fields in the POST request.