I upgraded an existing application and in the new version I require OS 5 - one of the reasons was that I wanted to use ConnectionFactory for communicating via HTTP/HTTPS with our server without having to add all the URL parameters for using BES, BIS, Direct TCP, Wifi and so on.
The ConnectionFactory is now configured to choose the best way to connect to our services via preferred types.
My connection code looks like this:
ConnectionFactory connectionFactory = new ConnectionFactory();
BisBOptions bisOptions = new BisBOptions(BIS_SECRET);
connectionFactory.setTransportTypeOptions(TransportInfo.TRANSPORT_BIS_B, bisOptions);
connectionFactory.setConnectionMode(ConnectionFactory.ACCESS_READ_WRITE);
connectionFactory.setEndToEndDesired(true);
connectionFactory.setPreferredTransportTypes(new int[] { TransportInfo.TRANSPORT_BIS_B, TransportInfo.TRANSPORT_MDS,
TransportInfo.TRANSPORT_TCP_WIFI, TransportInfo.TRANSPORT_TCP_CELLULAR });
ConnectionDescriptor connectionDescriptor = connectionFactory.getConnection("https://myserver.com/serviceurl");
try {
HttpConnection con = (HttpConnection) connectionDescriptor.getConnection();
byte[] bytes = parameter.toString().getBytes(UTF_8);
con.setRequestProperty(CONTENT_LENGTH, String.valueOf(bytes.length));
os = con.openOutputStream();
os.write(bytes);
os.flush();
int responseCode = con.getResponseCode();
if (responseCode == 401) {
throw new InvalidCredentialsException("Invalid credentials");
} else if (responseCode != 200 && responseCode != 500) {
EventLogger.logEvent(RTSID, ("Response code " + responseCode + " " + con
.getResponseMessage()).getBytes(), EventLogger.ERROR);
EventLogger.logEvent(RTSID, bytes, EventLogger.ERROR);
throw new IOException("Invalid request");
}
is = con.openInputStream();
if (is != null) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
int c = 0;
try {
c = is.read();
} catch (Exception ex) {
c = -1;
}
while (c >= 0) {
baos.write(c);
try {
c = is.read();
} catch (Exception ex) {
c = -1;
}
}
String response = new String(baos.toByteArray(), UTF_8);
try {
JSONObject jsonObject;
if (response.startsWith("[")) {
jsonObject = new JSONObject();
jsonObject.put(ARRAY, new JSONArray(response));
} else {
jsonObject = new JSONObject(response);
}
if (responseCode == 500) {
throw new Exception(jsonObject.getString("message"));
}
return jsonObject;
} catch (JSONException e) {
EventLogger.logEvent(RTSID, ("Exception occured: " + e.toString()).getBytes(),
EventLogger.ERROR);
}
}
} finally {
if (is != null) {
try {
is.close();
} catch (Exception e) {
}
}
if (os != null) {
try {
os.close();
} catch (Exception e) {
}
}
if (con != null) {
try {
con.close();
} catch (Exception e) {
}
}
}
My problem is that this works not as well as when I added the connection parameters to my URL manually. I get errors in the server logs looking like the clients close the connection after some kind of timeout.
Here are some log examples:
93.186.30.120 - - [28/Jun/2012:15:50:08 +0200] "POST /service/methodX HTTP/1.1" 400 145 "-" "myapp VendorID/301" 10012567
93.186.22.118 - - [28/Jun/2012:16:30:56 +0200] "POST /service/methodY HTTP/1.1" 400 145 "-" "myapp VendorID/137" 10012435
74.82.68.35 - - [28/Jun/2012:16:53:23 +0200] "POST /service/methodZ HTTP/1.1" 400 145 "-" "myapp BlackBerry9650/6.0.0.524 VendorID/105" 10012644
The IP Adresses are from RIM Networks - so these are Connections comming from BIS
Those connections got status code 400 (Bad Request) from the server
The large numbers at the end of the line (e.g. 10012644) show the time the request was processed on the server in microseconds: 10012644 = about 10 seconds
Do the RIM servers add a connection timeout of 10 seconds? That seems fairly short!
The problem is difficult to reproduce - has anybody experienced something like that before?
I found the reason. Problem was caused by default configuration of Apache module mod_reqtimeout:
<IfModule reqtimeout_module>
# mod_reqtimeout limits the time waiting on the client to prevent an
# attacker from causing a denial of service by opening many connections
# but not sending requests. This file tries to give a sensible default
# configuration, but it may be necessary to tune the timeout values to
# the actual situation. Note that it is also possible to configure
# mod_reqtimeout per virtual host.
# Wait max 20 seconds for the first byte of the request line+headers
# From then, require a minimum data rate of 500 bytes/s, but don't
# wait longer than 40 seconds in total.
# Note: Lower timeouts may make sense on non-ssl virtual hosts but can
# cause problem with ssl enabled virtual hosts: This timeout includes
# the time a browser may need to fetch the CRL for the certificate. If
# the CRL server is not reachable, it may take more than 10 seconds
# until the browser gives up.
RequestReadTimeout header=20-40,minrate=500
# Wait max 10 seconds for the first byte of the request body (if any)
# From then, require a minimum data rate of 500 bytes/s
RequestReadTimeout body=10,minrate=500
</IfModule>
I guess BlackBerry clients wer hit harder because sending request body via RIM BIS infrastructure takes longer.
Set the value to 100 seconds and monitoring if clients are still affected.
Related
I'm having trouble trying to consume the Response of an HTTP Endpoint which Streams real-time events continously. It's actually one of Docker's endpoints: https://docs.docker.com/engine/api/v1.40/#operation/SystemEvents
I am using Apache HTTP Client 4.5.5 and it just halts indefinitely when I try to consume the content InputStream:
HttpEntity entity = resp.getEntity();
EntityUtils.consume(entity);//it just hangs here.
//Even if I don't call this method, Apache calls it automatically
//after running all my ResponseHandlers
Apparently, it can be done by using JDK's raw URL: Stream a HTTP response in Java
But I cannot do that since local Docker communicates over a Unix Socket which I only managed to configure in Apache's HTTP Client with a 3rd party library for Unix Sockets in Java.
If there is a smarter HTTP Client library which I could switch to, that would also be an option.
Any ideas would be greatly appreciated. Thank you!
I managed to solve this issue by generating an infinite java.util.stream.Stream of JsonObject from the response InputStream (I know the json reading part is not the most elegant solution but there is no better way with that API and also, Docker doesn't send any separator between the jsons).
final InputStream content = response.getEntity().getContent();
final Stream<JsonObject> stream = Stream.generate(
() -> {
JsonObject read = null;
try {
final byte[] tmp = new byte[4096];
while (content.read(tmp) != -1) {
try {
final JsonReader reader = Json.createReader(
new ByteArrayInputStream(tmp)
);
read = reader.readObject();
break;
} catch (final Exception exception) {
//Couldn't parse byte[] to Json,
//try to read more bytes.
}
}
} catch (final IOException ex) {
throw new IllegalStateException(
"IOException when reading streamed JsonObjects!"
);
}
return read;
}
).onClose(
() -> {
try {
((CloseableHttpResponse) response).close();
} catch (final IOException ex) {
//There is a bug in Apache HTTPClient, when closing
//an infinite InputStream: IOException is thrown
//because the client still tries to read the remainder
// of the closed Stream. We should ignore this case.
}
}
);
return stream;
I am trying to connect a to Neo4j Aura instance from a .NET core 2.2 web api. I understand I need the Neo4j .Net Driver v4.0.0-alpha01, but I do not seem to be able to connect. There aren't very many examples out there as this driver is new and so is Aura.
I keep getting:
Failed after retried for 6 times in 30000 ms. Make sure that your database is online and retry again.
I configure the driver as such
public void ConfigureServices(IServiceCollection services)
{
string uri = "neo4j://1234567.databases.neo4j.io:7687";//not actual subdomain
string username = "neo4j";
string password = "seeeeeeecret";//not actual password
services.AddCors();
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
services.AddSingleton(GraphDatabase.Driver(uri, AuthTokens.Basic(username, password)));
}
and in my test controller i run this
private async Task<string> Neo4JTestAsync()
{
string db = "MyDb";
string message = "TESTMESSAGE";
IAsyncSession session = _driver.AsyncSession(o => o.WithDatabase(db));
try
{
var greeting = session.WriteTransactionAsync(async tx =>
{
var result = tx.RunAsync("CREATE (a:Greeting) " +
"SET a.message = $message " +
"RETURN a.message + ', from node ' + id(a)",
new { message });
var res = await result;
return "return something eventually";
});
return await greeting;
}
catch (Exception e)
{
return e.Message; // throws "Failed after retried for 6 times in 30000 ms. Make sure that your database is online and retry again"
}
finally
{
await session.CloseAsync();
}
}
I can't get the exact error message you do - but I'm pretty sure this is due to encryption - one of the big differences between the 1.x and 4.x drivers is the default position on Encryption - which is now off by default.
So you'll want to change your initialisation to:
services.AddSingleton(GraphDatabase.Driver(uri, AuthTokens.Basic(username, password), config => config.WithEncryptionLevel(EncryptionLevel.Encrypted)));
That should get you going. Also - make sure you stick with the neo4j:// protocol, as that'll route you properly.
Have you tried bolt:// in the connection string?
string uri = "bolt://1234567.databases.neo4j.io:7687";//not actual subdomain
When writing a message to the Azure Service Bus (using Microsoft.Azure.ServiceBus standard library, not the .Net Framework version) it works fine. However, when switching networks to a network that blocks that traffic and running it again I would expect an error being raised by SendAsync yet no error is thrown, therefor the function considers the send successful even though it is not.
Am I missing some logic to make sure that errors do get raised and trapped, it seems to be inline with all the examples I have seen.
I have tried this possible solution ..
Trouble catching exception on Azure Service Bus SendAsync method
.ContinueWith(t =>
{
Console.WriteLine(t.Status + "," + t.IsFaulted + "," + t.Exception.InnerException);
}, TaskContinuationOptions.OnlyOnFaulted);
.. and at no point does ContinueWith get hit.
[HttpPost]
[Consumes("application/json")]
[Produces("application/json")]
public ActionResult<Boolean> Post(Contract<T> contract)
{
Task.Run(() => SendMessage(contract));
// Other stuff
}
private async Task<ActionResult<Boolean>> SendMessage(Contract<T> contract)
{
JObject json = JObject.FromObject(contract);
Message message = new Message();
message.MessageId = Guid.NewGuid().ToString();
message.ContentType = ObjectType;
message.PartitionKey = ObjectType;
message.Body = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(contract));
foreach (KeyValuePair<String, String> route in DataRouting)
{
JToken jToken = json.SelectToken(route.Value);
if (jToken != null)
{
message.UserProperties[route.Key] = jToken.Value<String>();
}
else
{
String routeError = $"Could not find routing information in request for: {route.Key} in {route.Value}";
Logger.LogError(routeError);
return new UnprocessableEntityObjectResult(routeError);
}
}
// Send the message
try
{
await topicClient.SendAsync(message);
}
catch(Exception ex)
{
return new UnprocessableEntityObjectResult($"'Could not transmit message to service bus - {ex.Message}'");
}
return new OkObjectResult(true);
}
I expect that the error trap would be hit if the SendAsync fails to send the message. However it essentially fire and forgets, the message send is blocked by the firewall but is never reported to the caller by throwing an error.
Ok, found the answer, but I will leave this out there in case anyone else does this to themselves. It was down to my general muppetry when putting the MVC Controller together. Set async on the Post action and configure the await on the send. Obvious really but I missed it.
public virtual async Task<ActionResult<Boolean>> Post(Contract<T> contract){}
...
// Send the message
try
{
await topicClient.SendAsync(message).ConfigureAwait(false);
return new OkObjectResult(true); // Success if we got here
}
catch(Exception ex)
{
return new UnprocessableEntityObjectResult($"'Could not transmit message to service bus - {ex.Message}'");
}
Due to Heartbleed, our Gateway Server was updated and this problem presented itself.
Due to POODLE, SSLv3 is no longer supported.
Note, the problem is only present on Win7+ boxes; WinXP boxes work without issue (same code, different OS = problem); granted WinXP is no longer a valid OS, just wanted to make note of functionality.
Client application (.NET 2.0) sits on a Windows 7 (or 8) box. Server runs within a DMZ behind a Gateway Server. Just to note, I found that this problem is no longer present on .NET 4.0+ - however due to legacy code, I do not have the luxury of updating.
Gateway Server is a pass through box on which Apache HTTP Server with SSL run. Its location is outside the DMZ, and it is used to access the Server which is inside the DMZ. Versions of software running on the Gateway server are Apache/2.2.25 (Win32), mod_jk/1.2.39, mod_ssl/2.2.25, OpenSSL/1.0.1g
Here is the code used on the Client application (with an exorbitant amount of logging added) ... note, 'serverName' typically contains a value such as "https://some.url.com"
private bool ConnectAndAuthenicate(string serverName, out TcpClient client, out SslStream sslStream)
{
client = null;
sslStream = null;
try
{
client = new TcpClient(serverName, 443); // Create a TCP/IP client; ctor attempts connection
Log("ConnectAndAuthenicate: Client CONNECTED"));
sslStream = new SslStream(client.GetStream(), false, ValidateServerCertificate, null);
Log("ConnectAndAuthenicate: SSL Stream CREATED"));
}
catch (Exception x)
{
Log("ConnectAndAuthenicate: EXCEPTION >> CONNECTING to server: {0}", x.ToString()));
if (x is SocketException)
{
SocketException s = x as SocketException;
Log("ConnectAndAuthenicate: EXCEPTION >> CONNECTING to server: Socket.ErrorCode: {0}", s.ErrorCode));
}
if (client != null) { client.Close(); client = null; }
if (sslStream != null) { sslStream.Close(); sslStream = null; }
}
if (sslStream == null) return false;
try
{
sslStream.ReadTimeout = 10000; // wait 10 seconds for a response ...
Log("ConnectAndAuthenicate: AuthenticateAsClient CALLED ({0})", serverName));
sslStream.AuthenticateAsClient(serverName);
Log("ConnectAndAuthenicate: AuthenticateAsClient COMPLETED SUCCESSFULLY"));
return true;
}
catch (Exception x)
{
Log("ConnectAndAuthenicate: EXCEPTION >> AuthenticateAsClient: {0}", x.ToString()));
client.Close(); client = null;
sslStream.Close(); sslStream = null;
}
return false;
}
Note - answers posted pertaining to ServicePointManager have absolutely no effect on the outcome of this application.
Every time that AuthenicateAsClient() is called when application is run on Win 7+ box, the exception occurs - if application is run on WinXP box, code works properly without exceptions.
Any ideas for solutions are very welcome.
Following the trail of setting the ServicePointManager.SecurityProtocol static ctor with a SecurityProtocolType, I found mention of another enum called SslPolicy -- further research found that AuthenicateAsClient has an overload that takes SslPolicy as an argument.
Changing this line in the above code fixed this problem:
sslStream.AuthenticateAsClient(serverName, null, SslPolicy.Tls, false);
I want to store position coords (latitude, longitude) in a table in my MySQL DB querying a url in a way similar to this one: http://locationstore.com/postlocation.php?latitude=var1&longitude=var2 every ten seconds. PHP script works like a charm. Getting the coords in the device ain't no problem either. But making the request to the server is being a hard one. My code goes like this:
public class LocationHTTPSender extends Thread {
for (;;) {
try {
//fetch latest coordinates
coords = this.coords();
//reset url
this.url="http://locationstore.com/postlocation.php";
// create uri
uri = URI.create(this.url);
FireAndForgetDestination ffd = null;
ffd = (FireAndForgetDestination) DestinationFactory.getSenderDestination
("MyContext", uri);
if(ffd == null)
{
ffd = DestinationFactory.createFireAndForgetDestination
(new Context("MyContext"), uri);
}
ByteMessage myMsg = ffd.createByteMessage();
myMsg.setStringPayload("doesnt matter");
((HttpMessage) myMsg).setMethod(HttpMessage.POST);
((HttpMessage) myMsg).setQueryParam("latitude", coords[0]);
((HttpMessage) myMsg).setQueryParam("longitude", coords[1]);
((HttpMessage) myMsg).setQueryParam("user", "1");
int i = ffd.sendNoResponse(myMsg);
ffd.destroy();
System.out.println("Lets sleep for a while..");
Thread.sleep(10000);
System.out.println("woke up");
} catch (Exception e) {
// TODO Auto-generated catch block
System.out.println("Exception message: " + e.toString());
e.printStackTrace();
}
}
I haven't run this code to test it, but I would be suspicious of this call:
ffd.destroy();
According to the API docs:
Closes the destination. This method cancels all outstanding messages,
discards all responses to those messages (if any), suspends delivery
of all incoming messages, and blocks any future receipt of messages
for this Destination. This method also destroys any persistable
outbound and inbound queues. If Destination uses the Push API, this
method will unregister associated push subscriptions. This method
should be called only during the removal of an application.
So, if you're seeing the first request succeed (at least sometimes), and subsequent requests fail, I would try removing that call to destroy().
See the BlackBerry docs example for this here
Ok so I finally got it running cheerfully. The problem was with the transport selection; even though this example delivered WAP2 (among others) as an available transport in my device, running the network diagnostics tool showed only BIS as available. It also gave me the connection parameters that I needed to append at the end of the URL (;deviceside=false;ConnectionUID=GPMDSEU01;ConnectionType=mds-public). The code ended up like this:
for (;;) {
try {
coords.refreshCoordinates();
this.defaultUrl();
this.setUrl(stringFuncs.replaceAll(this.getUrl(), "%latitude%", coords.getLatitude() + ""));
this.setUrl(stringFuncs.replaceAll(this.getUrl(), "%longitude%", coords.getLongitude() + ""));
cd = cf.getConnection(this.getUrl());
if (cd != null) {
try {
HttpConnection hc = (HttpConnection)cd.getConnection();
final int i = hc.getResponseCode();
hc.close();
} catch (Exception e) {
}
}
//dormir
Thread.sleep(15000);
} catch (Exception e) {
} finally {
//cerrar conexiones
//poner objetos a null
}
Thanks for your help #Nate, it's been very much appreciated.