When use an AWS ELB (Elastic Load Balancer) with HTTPS listener that forward the request to the presto cluster (0.193) behind that runs on HTTP, Java client that uses presto-jdbc (0.213) fails to execute select query due to "plain HTTP request was sent to HTTPS port" error.
Exception in thread "main" java.sql.SQLException: Error executing query
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:274)
at com.facebook.presto.jdbc.PrestoStatement.execute(PrestoStatement.java:227)
at com.facebook.presto.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:76)
at com.DpTool.executeJdbcQuerySsl(DpTool.java:332)
at com.DpTool.prestoJdbcSsl(DpTool.java:315)
at com.DpTool.main(DpTool.java:520)
Caused by: java.lang.RuntimeException: Error fetching next at http://cluster-elb.mydomain:8443/v1/statement/20181120_215602_00038_57ryf/1 returned an invalid response: JsonResponse{statusCode=400, statusMessage=Bad Request, headers={connection=[close], content-length=[236], content-type=[text/html], date=[Tue, 20 Nov 2018 21:56:02 GMT], server=[awselb/2.0]}, hasValue=false} [Error: <html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
</body>
</html>
]
at com.facebook.presto.jdbc.internal.client.StatementClientV1.requestFailedException(StatementClientV1.java:436)
at com.facebook.presto.jdbc.internal.client.StatementClientV1.advance(StatementClientV1.java:383)
at com.facebook.presto.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1742)
at com.facebook.presto.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:119)
at com.facebook.presto.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:249)
... 5 more
It appears that StatementClientV1.advance code try to visit "http://cluster-elb.mydomain:8443/..." which is the ELB address except it should be "https" instead of "http" prefix. The same client code works fine when execute query directly to another HTTPS presto cluster (no ELB). Here is the client Java code. url is "cluster-elb.mydomain:8443"
Properties properties = new Properties();
properties.setProperty("SSL", "true");
Connection conn = DriverManager.getConnection(url, properties);
Statement stmt = conn.createStatement();
String query = "select cluster, query_text from mytable where ds='2018-10-20' limit 10";
ResultSet rs = stmt.executeQuery(query);
Why the presto-jdbc client code is confused and how to make the query work?
This should work in newer versions of Presto. Try it with 0.213.
The problem is due to https://github.com/prestodb/presto/issues/8232 where the nextUri field in QueryStatusInfo is incorrectly set as http instead of https by presto server (when client talk to presto through elb or proxy).
Related
I'm using java.net.http.HttpClient.newHttpClient() under Java 19 (Temurin) and perform sendAsync(...) requests from different treads on the same instance. I assume this is ok, as the javadoc states:
Once built, an HttpClient is immutable...
However, some requests fail with:
java.io.IOException: HTTP/1.1 header parser received no bytes
The weird thing is, it depends on the speed of my requests:
Requests every 5 seconds: 30% failure
Requests every 3 seconds: 0% failure
I've written a test for it:
private final HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://..."))
.setHeader("Content-Type", "application/json")
.POST(HttpRequest.BodyPublishers.ofByteArray("[]".getBytes()))
.build();
#ParameterizedTest
#ValueSource(ints = {3, 5})
void httpClientTest(int intervalSeconds) throws Exception {
HttpClient httpClient = HttpClient.newHttpClient();
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
Thread.sleep(Duration.ofSeconds(intervalSeconds));
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
Thread.sleep(Duration.ofSeconds(intervalSeconds));
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
Thread.sleep(Duration.ofSeconds(intervalSeconds));
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
Thread.sleep(Duration.ofSeconds(intervalSeconds));
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofByteArray()).get();
}
I've already tried the following:
Doing the same with curl on the command line. No requests fail whatever interval I try. So it's probably not a problem with the server.
Running the tests multiple times in parallel. Still the 5-second-intervals fail (then multiple times in parallel). So it's probably not a problem with the server.
Creating an HttpClient.newHttpClient() for every request. No requests fail whatever interval. So it's probably not a problem with the server but with an internal state of the HttpClient (although it claims to be immutable?).
Do you have an idea what I could do, without needing to create a new HttpClient for every request?
Here is the answer for the record: the java.net.HttpClient has a long default HTTP/1.1 keepAlive time, which is longer than what usual servers are configured with. This often results in the server closing idle HTTP/1.1 connections before the client does. If the server closes the connection at about the same time than the client tries to reuse it, some IOException might get raised.
If such exceptions are observed too frequently applications should consider adapting the default keepAlive time in the client to some value shorter than what the servers it connects to are using.
A default value for the HttpClient HTTP/1.1 keepAlive time can be specified on the command line with: -Djdk.httpclient.keepalive.timeout=duration-in-seconds
So for instance - if a server is configured with a keepAlive time of 5s, you could consider supplying -Djdk.httpclient.keepalive.timeout=3 or -Djdk.httpclient.keepalive.timeout=4 on the client's java command line.
I am using WSO2 for token generation. By using this console: https://localhost:9443/carbon/admin/login.jsp
I am calling http://localhost:9763**/oauth2/token** service by passing required parameters
<{username=[], password=[], grant_type=[password], client_id=[], client_secret=[], scope=[]},[Content-Type:"application/x-www-form-urlencoded"]> after 20th request its failing and giving below error.
{ErrorMessage: I/O error on POST request for "http://192.168.200.83:9763/oauth2/token": Connection reset; nested exception is java.net.SocketException: Connection reset.}
How can I increase the number of requests to be served from 20 to 50 or more?
I’m using jersey-client v1.18.1
I need to make 2 sequential requests where the 1st request has an InputStream and then must pass that same InputStream along to the 2nd request (eg. sort of like a proxy). The 2nd request will then write the InputStream to disk and send back to the 1st request the fully qualified path to the location on disk where the 2nd request wrote the file.
The following code-snippet outlines what I have tried, but cannot get to work properly. I’m currently receiving the error:
"com.sun.jersey.api.client.ClientHandlerException: A message body writer for Java type, class org.seleniumhq.jetty9.server.HttpInputOverHTTP, and MIME media type, application/octet-stream, was not found”
I believe I have all the proper Maven dependencies in my project for the MIME and message body writers.
1st Request originating on Host 1 going to Host 2
Client client = Client.create();
client.resource(uri_for_request_1)
client.path(“request_1_servlet");
client.queryParam(“uri_for_request_2", uri_for_request_2);
client.queryParam("targetFilename", targetFilename);
ClientResponse response = client.accept(MediaType.APPLICATION_JSON).entity(inputStream).post(ClientResponse.class);
2nd Request originating on Host 2 going to Host 3
Client client = Client.create();
client.resource(request.getParamater(“uri_for_request_2"))
client.path(“request_2_servlet");
client.queryParam("targetFilename", request.getParamater(“targetFilename");
ClientResponse response = client.accept(MediaType.APPLICATION_JSON).entity(request.getInputStream()).post(ClientResponse.class);
Host 3
Writes InputStream to file and sends back to Host 2 fully qualified path.
Host 2
Sends back to Host 1 fully qualified path.
Variations I’ve tried on post calls:
client.accept(MediaType.APPLICATION_JSON).type(MediaType.APPLICATION_OCTET_STREAM).entity(inputStream).post(ClientResponse.class);
client.accept(MediaType.APPLICATION_JSON).type(MediaType.APPLICATION_OCTET_STREAM).post(ClientResponse.class, inputStream);
I can confirm the 1st request is being made on Host 1 and reaches Host 2. It is the 2nd request on Host 2 that fails during the post() call.
I have succesfully used neo4jphp library with graphenedb with this simple steps as per documentation (considering that graphenedb does not require https)
require('vendor/autoload.php'); // or your custom autoloader
// Connecting to a different port or host
$client = new Everyman\Neo4j\Client(url, port);
// Connecting using HTTP and Basic Auth
$client->getTransport()
->setAuth('username', 'password');
// Test connection to server
print_r($client->getServerInfo());
However, when trying to connect to a graphstory instance (of course they both work fine if I call the rest api from browser, the neo4j console works fine etc.) which requires https as follows
require('vendor/autoload.php'); // or your custom autoloader
// Connecting to a different port or host
$client = new Everyman\Neo4j\Client(url, port);
// Connecting using HTTPS and Basic Auth
$client->getTransport()
->useHttps()
->setAuth('username', 'password');
// Test connection to server
print_r($client->getServerInfo());
I get the following error. They should be identical, I can't get why.
Fatal error: Uncaught exception 'Everyman\Neo4j\Exception' with message 'Can't open connection to https://neo-54f500bf2cc7e-364459c455.do-stories.graphstory.com:7473/db/data/' in /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Transport/Curl.php:91
Stack trace:
#0 /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Transport.php(95): Everyman\Neo4j\Transport\Curl->makeRequest('GET', '/', NULL)
#1 /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Command.php(64): Everyman\Neo4j\Transport->get('/', NULL)
#2 /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Client.php(828): Everyman\Neo4j\Command->execute()
#3 /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Client.php(464): Everyman\Neo4j\Client->runCommand(Object(Everyman\Neo4j\Command\GetServerInfo))
#4 /Applications/XAMPP/xamppfiles/htdocs/graphene/story.php(20): Every in /Applications/XAMPP/xamppfiles/htdocs/graphene/vendor/everyman/neo4jphp/lib/Everyman/Neo4j/Transport/Curl.php on line 91
It seems to me that neo4jphp is not configuring the TLS part in the cURL request.
I fixed it by downloading the certificate bundle from http://curl.haxx.se/docs/caextract.html (ca_bundle.crt) and adding the following line to Everyman\Neo4j\Transport\Curl.php, function makeRequest:
$options[CURLOPT_CAINFO] = "your/path/to/ca-bundle.crt";
I've created an issue on GitHub for this: https://github.com/jadell/neo4jphp/issues/171
I'm the CTO/Lead Dev at Graph Story. Sorry to hear you're having troubles. I've actually just taken a look at your instance and things seem OK from the server side.
Without additional info it's hard to say if there's an issue with your sample connection code. Considering that you've used that same library to connect to GrapheneDB in the past, I think the chances an error in the sample code is low.
Based on the current state of your instance and on the exception thrown by Neo4jPHP, my guess is that port 7473 may be blocked on your network. You can confirm that with local tech support or by switching to a network where you know port 7473 is open and trying to connect again.
I am currently trying to write an auxiliary module for Metasploit. The module basically tries multiple default credentials to get access to the router's management page. The authentication is done via web, i.e. HTTP POST.
Currently, the module works as expected for plain HTTP connections, i.e. unsecured connections, however every connection attempt via HTTPS (port 443), returns nil. Below is the function used within the Metasploit class to retrieve the login page:
def get_login_page(ip)
begin
response = send_request_cgi(
'uri' => '/',
'method' => 'GET'
)
# Some models of ZyXEL ZyWALL return a 200 OK response
# and use javascript to redirect to the rpAuth.html page.
if response && response.body =~ /changeURL\('rpAuth.html'\)/
vprint_status "#{ip}- Redirecting to rpAuth.html page..."
response = send_request_cgi(
'uri' => '/rpAuth.html',
'method' => 'GET'
)
end
rescue ::Rex::ConnectionError
vprint_error "#{ip} - Failed to connect to Web management console."
end
return response
end
When trying to connect via HTTPS, the first send_request_cgi call returns nil. No exception are caught or thrown. I have tried with 3 different hosts to make sure the issue was not with a specific endpoint. All my 3 attempts failed to return a response. At every attempt, I set the RPORT option to 443;
RHOSTS 0.0.0.0 yes The target address range or CIDR identifier
RPORT 443 yes The target port
Note that I have replaced the real IP with 0.0.0.0. Using a web browser, I can actually connect to the router via HTTPS with no issue (other than having to add an exception since the certificate is untrusted) and am presented the login page. With Wireshark, I tried to look at the generated traffic. I can clearly see that nothing is sent by the router. I notice the 3-way handshake being completed and the HTTP GET request being made:
GET / HTTP/1.1
Host: 0.0.0.0
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
There are 3-4 ACK after and then a FIN/PUSH sent by the server.
Based on this page on Metasploit's GitHub, I was under the impression that connections to HTTPS websites were handled by the underlying framework. I have not seen any articles/tutorial/source that leads me to believe otherwise. The doc about the send_request_cgi does not specify any specific requirement to establish a HTTPS connection. Other posts did not had the exact same issue I'm having. At this point I suspect either the OS, the framework or me forgetting to enable something. Other modules I have looked at either only targets HTTP websites - which I doubt - or do not have any special handling for HTTPS connections.
Any help determining the cause would be greatly appreciated.
Version of Metasploit:
Framework: 4.9.3-2014060501
Console : 4.9.3-2014060501.15168
Version of OS:
SMP Debian 3.14.5-1kali1 (2014-06-07)
As per this post on SecurityStreet, the solution was to set SSL to true in the DefaultOptions in the initialize function:
def initialize
super(
...
'DefaultOptions' =>
{
...
'SSL' => true
}
)
...
end
Connections to routers using HTTPS worked afterwards.