WebRequest.GetResponse() is getting an "underlying connection was closed" - network-programming

I build the WebRequest object and then call GetResponse() on it. The url is "https://services.odata.org/V2/Northwind/Northwind.svc/$metadata".
And it comes up in a browser fine. What's really weird is these are returning a 200 so it should be good. But no data is coming back.
According to Fiddler there are two requests that go out:
Request #1
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: 5E 5B 1A 26 84 02 5A 4B DF 6B BF 22 DF 2C 03 AD 37 62 DA CE 05 79 7E B4 5C 58 AF BE 8C 26 5D CA
"Time": 4/4/1990 2:38:54 PM
SessionID: empty
Extensions:
server_name services.odata.org
supported_groups x25519 [0x1d], secp256r1 [0x17], secp384r1 [0x18]
ec_point_formats uncompressed [0x0]
SessionTicket empty
extended_master_secret empty
renegotiation_info 00
Ciphers:
[C00A] TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C009] TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C014] TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
[C013] TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
[0035] TLS_RSA_WITH_AES_256_CBC_SHA
[002F] TLS_RSA_WITH_AES_128_CBC_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
Compression:
[00] NO_COMPRESSION
Response #1
Request #2
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.1 (TLS/1.0)
Random: 5E 5B 1A 26 0D B0 9F BF 4F D8 FC 5D 27 47 A8 7C CB 83 2C 43 4C 2B C3 8A 0C C0 EB 30 3D 6E AB 51
"Time": 4/4/1990 2:38:54 PM
SessionID: empty
Extensions:
server_name services.odata.org
supported_groups x25519 [0x1d], secp256r1 [0x17], secp384r1 [0x18]
ec_point_formats uncompressed [0x0]
SessionTicket empty
extended_master_secret empty
renegotiation_info 00
Ciphers:
[C00A] TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C009] TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C014] TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
[C013] TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
[0035] TLS_RSA_WITH_AES_256_CBC_SHA
[002F] TLS_RSA_WITH_AES_128_CBC_SHA
[000A] SSL_RSA_WITH_3DES_EDE_SHA
Compression:
[00] NO_COMPRESSION
Response #2

It was the TLS level. Just add:
ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;

Related

How to get to know TCP or UDP protocol from wireshark pcap hexadecimal view

A sample TCP hexadecimal Wireshark capture shown below,
0000 6c 6a 77 8d 48 cf 96 38 a7 7d 02 de 08 00 45 28
0010 00 34 56 92 40 00 28 06 fa e9 a2 7d 13 09 c0 a8
0020 8a f1 01 bb df 84 20 00 27 8e e3 6f a9 7f 80 10
0030 00 7c 25 f3 00 00 01 01 08 0a 54 48 f8 cc 61 38
0040 eb 22
How to get to know TCP or UDP protocol from Wireshark pcap hexadecimal view ?.
Save the hex data, as shown, into a text file and then run text2pcap file.txt file.pcap. You can then load the file.pcap file into Wireshark. Run text2pcap -h for more help or refer to the text2pcap man page for more details about that tool.
0000 6c 6a 77 8d 48 cf 96 38 a7 7d 02 de 08 00 45 28
0010 00 34 56 92 40 00 28 06 fa e9 a2 7d 13 09 c0 a8
8th Byte in the second line we have to check. In here it's 06. So protocol number 06 is TCP. Like wise we can get to know the protocol number from hexadecimal view.

How to validate CRC-32 calculation of Zip file

I want to validate that my ZIP file has a correct CRC-32 checksum.
I read that in a ZIP file the CRC-32 data is in bytes 14 to 17:
Offset Bytes Description[30]
0 4 Local file header signature = 0x04034b50 (read as a little-endian number)
4 2 Version needed to extract (minimum)
6 2 General purpose bit flag
8 2 Compression method
10 2 File last modification time
12 2 File last modification date
14 4 CRC-32 of uncompressed data
18 4 Compressed size
22 4 Uncompressed size
26 2 File name length (n)
28 2 Extra field length (m)
30 n File name
30+n m Extra field
I wanted to validate a CRC-32 checksum of a simple ZIP file I created:
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
-----------------------------------------------
50 4B 03 04 14 00 00 00 00 00 38 81 1C 51 4C 18 | PK........8..QL.
C7 8C 02 00 00 00 02 00 00 00 07 00 00 00 31 32 | nj............12
33 2E 64 61 74 73 73 50 4B 01 02 14 00 14 00 00 | 3.datssPK.......
00 00 00 38 81 1C 51 4C 18 C7 8C 02 00 00 00 02 | ...8..QL.nj.....
00 00 00 07 00 00 00 00 00 00 00 01 00 20 00 00 | ............. ..
00 00 00 00 00 31 32 33 2E 64 61 74 50 4B 05 06 | .....123.datPK..
00 00 00 00 01 00 01 00 35 00 00 00 27 00 00 00 | ........5...'...
00 00 | ..
The CRC-32 is: 0x4C18C78C
I went to this CRC-32 online calculator and added the following un-compressed row from the file:
50 4B 03 04 14 00 00 00 00 00 38 81 1C 51
This is the result:
Algorithm Result Check Poly Init RefIn RefOut XorOut
CRC-32 0x6A858174 0xCBF43926 0x04C11DB7 0xFFFFFFFF true true 0xFFFFFFFF
CRC-32/BZIP2 0xE3FA1205 0xFC891918 0x04C11DB7 0xFFFFFFFF false false 0xFFFFFFFF
CRC-32C 0xB578110E 0xE3069283 0x1EDC6F41 0xFFFFFFFF true true 0xFFFFFFFF
CRC-32D 0xAFE2EEA4 0x87315576 0xA833982B 0xFFFFFFFF true true 0xFFFFFFFF
CRC-32/MPEG-2 0x1C05EDFA 0x0376E6E7 0x04C11DB7 0xFFFFFFFF false false 0x00000000
CRC-32/POSIX 0xFF9B3071 0x765E7680 0x04C11DB7 0x00000000 false false 0xFFFFFFFF
CRC-32Q 0x79334F11 0x3010BF7F 0x814141AB 0x00000000 false false 0x00000000
CRC-32/JAMCRC 0x957A7E8B 0x340BC6D9 0x04C11DB7 0xFFFFFFFF true true 0x00000000
CRC-32/XFER 0xA7F36A3F 0xBD0BE338 0x000000AF 0x00000000 false false 0x00000000
But none of them equal to: 0x4C18C78C.
What am I doing wrong? The CRC-32 of the ZIP is the calculation of all the bytes (0-13) before, no?
The byte sequence you are running against the online CRC calculator are not uncompressed bytes.
50 4B 03 04 14 00 00 00 00 00 38 81 1C 51
Those bytes are the first few bytes of the zip file. The CRC32 value in a zip is calculated by running the CRC32 algorithm against the complete uncompressed payload. In your case the payload is the two byte sequence "ss".
To work that out, I converted your hex dump back into a zip file, tmp.zip. It contains a single member 123.dat
$ unzip -lv tmp.zip
Archive: tmp.zip
Length Method Size Cmpr Date Time CRC-32 Name
-------- ------ ------- ---- ---------- ----- -------- ----
2 Stored 2 0% 2020-08-28 16:09 8cc7184c 123.dat
-------- ------- --- -------
2 2 0% 1 file
When I extract that member to stdout & pipe though hexdump, we find it contains the two bytes string "ss" (hex 73 73)
$ unzip -p tmp.zip | hexdump -C
00000000 73 73 |ss|
Finally, as already mentioned in another comment, you can check that the CRC value is correct by running unzip -t
$ unzip -t tmp.zip
Archive: tmp.zip
testing: 123.dat OK
No errors detected in compressed data of tmp.zip.
I was able to create a zip file that matches the one in the question. The header shows that the compression type == 0, which means no compression, the uncompressed size == 2, the data == {73 73}. CRC32 uses reflected input and output, and the CRC is stored in little endian format, so the CRC == 0x8CC7184C.
I get a match using CRC32 on data of {73 73} using this online CRC calculator:
http://www.sunshine2k.de/coding/javascript/crc/crc_js.html

Why does HAProxy close the connection to the HiveMQ MQTT client at the end of the TLS handshake?

I'm trying to establish a TLS connection to a MQTT broker behind a HAProxy instance. At the end of the handshake (TLS 1.2 application data was sent), the server unexpectedly closes the connection.
I'm using self-signed certificates in form of a Keystore and a Truststore. Keystore and Truststore are tested against the server, for example with MQTTFx and with a MQTT3 Paho client.
As I have to use shared subscriptions, I decided to switch to the HiveMQ MQTT5 async client.
The following code illustrates building the client and connecting to the server.
The variable connectionProperties contains information about the broker and the keystores used.
public class MqttConnector {
Logger logger = LoggerFactory.getLogger(MqttConnector.class);
private ConnectionListener connectionListener;
private Properties connectionProperties;
private Mqtt5AsyncClient client;
public MqttConnector() throws SSLException {
this.connectionListener = new ConnectionListener();
this.connectionProperties = ConnectionProperties.get();
this.client = this.buildMqtt5AsyncClient(this.connectionProperties, this.connectionListener);
}
public MqttConnector(ConnectionListener connectionListener, Properties connectionProperties) throws SSLException {
this.connectionListener = connectionListener;
this.connectionProperties = connectionProperties;
this.client = this.buildMqtt5AsyncClient(this.connectionProperties, this.connectionListener);
}
private Mqtt5AsyncClient buildMqtt5AsyncClient(
Properties connectionProperties,
ConnectionListener connectionListener) throws SSLException {
return MqttClient.builder()
.identifier(connectionProperties.getProperty("clientId"))
.serverHost(connectionProperties.getProperty("host"))
.serverPort(Integer.valueOf(connectionProperties.getProperty("port")))
.addConnectedListener(connectionListener)
.addDisconnectedListener(connectionListener)
.sslConfig()
.keyManagerFactory(KeystoreUtil.keyManagerFromKeystore(
new File(connectionProperties.getProperty("keystore_file")),
connectionProperties.getProperty("keystore_pass"),
connectionProperties.getProperty("keystore_pass")))
.trustManagerFactory(KeystoreUtil.trustManagerFromKeystore(
new File(connectionProperties.getProperty("truststore_file")),
connectionProperties.getProperty("truststore_pass")))
.applySslConfig()
.automaticReconnectWithDefaultConfig()
.useMqttVersion5()
.buildAsync();
}
public void connect() {
this.client.connect().whenComplete((connack, throwable) -> {
if (throwable != null) {
logger.error("connection exception", throwable);
} else {
logger.info(connack.toString());
}
});
}
public void disconnect() {
if (this.client != null && this.client.getState().isConnected()) {
this.client.disconnect();
}
}
This is part of the debug log (debugged with additional -Djavax.net.debug=all to see the SSL output). I removed most of the raw data and the certificate information.
At the end of the handshake the connection is closed by the server.I have no idea why this is happening with tried and tested certificates.
javax.net.ssl|DEBUG|01|main|2019-10-15 15:40:50.356 CEST|X509TrustManagerImpl.java:79|adding as trusted certificates (
"certificate" : {
"version" : "v3",
"serial number" : "02 41 2E 94 57 70 85 FC DF CB 6E F1 24 79 E0 C3 53 2C 41 7A",
"signature algorithm": "SHA256withRSA",
"issuer" : "CN=ca.dev-broker.your-server.de",
"not before" : "2019-07-31 16:05:54.000 CEST",
"not after" : "2029-07-28 16:05:54.000 CEST",
"subject" : "CN=dev-broker.your-server.de",
"subject public key" : "RSA",
"extensions" : [
{
ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:false
PathLen: undefined
]
},
{
ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
DigitalSignature
Non_repudiation
Key_Encipherment
]
},
{
ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
DNSName: dev-broker.your-server.de
DNSName: *.dev-broker.your-server.de
]
}
]}
)
15:44:31.015 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
15:44:32.601 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
15:44:32.603 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 11
15:44:32.651 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
15:44:32.669 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
15:44:32.687 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
15:44:32.708 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
15:44:33.245 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
15:44:33.272 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
5:44:50.873 [epollEventLoopGroup-2-1] DEBUG io.netty.handler.ssl.JdkSslContext - Default protocols (JDK): [TLSv1.2, TLSv1.1, TLSv1]
15:44:50.876 [epollEventLoopGroup-2-1] DEBUG io.netty.handler.ssl.JdkSslContext - Default cipher suites (JDK): [TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384]
javax.net.ssl|ALL|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.877 CEST|SSLContextImpl.java:115|trigger seeding of SecureRandom
javax.net.ssl|ALL|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.877 CEST|SSLContextImpl.java:119|done seeding of SecureRandom
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.962 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 for TLS11
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.963 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for TLS11
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.964 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS11
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.964 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 for TLS10
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.967 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for TLS10
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:50.968 CEST|HandshakeContext.java:290|Ignore unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 for TLS10
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.002 CEST|ServerNameExtension.java:255|Unable to indicate server name
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.003 CEST|SSLExtensions.java:256|Ignore, context unavailable extension: server_name
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.009 CEST|SupportedGroupsExtension.java:841|Ignore inactive or disabled named group: ffdhe2048
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.012 CEST|SupportedGroupsExtension.java:841|Ignore inactive or disabled named group: ffdhe3072
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.013 CEST|SupportedGroupsExtension.java:841|Ignore inactive or disabled named group: ffdhe4096
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.014 CEST|SupportedGroupsExtension.java:841|Ignore inactive or disabled named group: ffdhe6144
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.014 CEST|SupportedGroupsExtension.java:841|Ignore inactive or disabled named group: ffdhe8192
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.026 CEST|SignatureScheme.java:282|Signature algorithm, ed25519, is not supported by the underlying providers
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.027 CEST|SignatureScheme.java:282|Signature algorithm, ed448, is not supported by the underlying providers
javax.net.ssl|ALL|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.043 CEST|SignatureScheme.java:358|Ignore disabled signature sheme: rsa_md5
javax.net.ssl|INFO|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.046 CEST|AlpnExtension.java:161|No available application protocols
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.047 CEST|SSLExtensions.java:256|Ignore, context unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.053 CEST|ClientHello.java:651|Produced ClientHello handshake message (
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "4D 37 59 83 17 36 AD CD 4E D5 4F DB EF 60 88 66 23 B2 ED FA DC 13 81 EF 5D 19 9E DC 9E FA 41 57",
"session id" : "",
"cipher suites" : "[TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035)]",
"compression methods" : "00",
"extensions" : [
]
}
)
15:44:51.076 [epollEventLoopGroup-2-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacityPerThread: 4096
15:44:51.077 [epollEventLoopGroup-2-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxSharedCapacityFactor: 2
15:44:51.078 [epollEventLoopGroup-2-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.linkCapacity: 16
15:44:51.078 [epollEventLoopGroup-2-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.ratio: 8
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.135 CEST|SSLEngineOutputRecord.java:507|WRITE: TLS12 handshake, length = 223
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.136 CEST|SSLEngineOutputRecord.java:525|Raw write (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.176 CEST|SSLEngineInputRecord.java:177|Raw read (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.177 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 93
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.184 CEST|ServerHello.java:866|Consuming ServerHello handshake message (
"ServerHello": {
"server version" : "TLSv1.2",
"random" : "EC 44 2C 42 7F C7 DD 1F C5 F8 C2 E1 13 4B C1 94 71 41 AF EF 96 E1 8D F3 E8 B4 7B 11 D7 74 A1 3F",
"session id" : "37 23 36 B1 49 8B A1 97 C5 2C F0 3A BC 25 D4 BF 13 3D 25 35 A9 31 C0 3E AB DA CE 81 E2 39 4D FF",
"cipher suite" : "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F)",
"compression methods" : "00",
"extensions" : [
"renegotiation_info (65,281)": {
"renegotiated connection": [<no renegotiated connection>]
},
"ec_point_formats (11)": {
"formats": [uncompressed, ansiX962_compressed_prime, ansiX962_compressed_char2]
},
"extended_master_secret (23)": {
<empty>
}
]
}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.185 CEST|SSLExtensions.java:169|Ignore unavailable extension: supported_versions
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.186 CEST|ServerHello.java:962|Negotiated protocol version: TLSv1.2
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.187 CEST|SSLExtensions.java:188|Consumed extension: renegotiation_info
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.188 CEST|SSLExtensions.java:169|Ignore unavailable extension: server_name
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.189 CEST|SSLExtensions.java:169|Ignore unavailable extension: max_fragment_length
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.189 CEST|SSLExtensions.java:169|Ignore unavailable extension: status_request
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.191 CEST|SSLExtensions.java:188|Consumed extension: ec_point_formats
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.194 CEST|SSLExtensions.java:169|Ignore unavailable extension: status_request_v2
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.194 CEST|SSLExtensions.java:188|Consumed extension: extended_master_secret
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.195 CEST|SSLExtensions.java:188|Consumed extension: renegotiation_info
javax.net.ssl|ALL|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.195 CEST|SSLSessionImpl.java:209|Session initialized: Session(1571147091195|TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.196 CEST|SSLExtensions.java:203|Ignore unavailable extension: server_name
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.196 CEST|SSLExtensions.java:203|Ignore unavailable extension: max_fragment_length
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.196 CEST|SSLExtensions.java:203|Ignore unavailable extension: status_request
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.197 CEST|SSLExtensions.java:211|Ignore impact of unsupported extension: ec_point_formats
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.197 CEST|SSLExtensions.java:203|Ignore unavailable extension: application_layer_protocol_negotiation
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.197 CEST|SSLExtensions.java:203|Ignore unavailable extension: status_request_v2
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.197 CEST|SSLExtensions.java:211|Ignore impact of unsupported extension: extended_master_secret
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.198 CEST|SSLExtensions.java:211|Ignore impact of unsupported extension: renegotiation_info
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.209 CEST|SSLEngineInputRecord.java:177|Raw read (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.215 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 2517
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.240 CEST|CertificateMessage.java:358|Consuming server Certificate handshake message (
"Certificates": [...]
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.760 CEST|X509TrustManagerImpl.java:300|Found trusted certificate (
"certificate" : {...}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.766 CEST|SSLEngineInputRecord.java:177|Raw read (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.767 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 333
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.778 CEST|ECDHServerKeyExchange.java:538|Consuming ECDH ServerKeyExchange handshake message (
"ECDH ServerKeyExchange": {
"parameters": {
"named group": "secp256r1"
"ecdh public": {...},
},
"digital signature": {
"signature algorithm": "rsa_pss_rsae_sha256"
"signature": {...},
}
}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.780 CEST|SSLEngineInputRecord.java:177|Raw read (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.782 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 131
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.788 CEST|CertificateRequest.java:651|Consuming CertificateRequest handshake message (
"CertificateRequest": {
"certificate types": [rsa_sign, dss_sign, ecdsa_sign]
"supported signature algorithms": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp512r1_sha512, ed25519, ed448, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, ecdsa_sha224, ecdsa_sha1, rsa_sha224, rsa_pkcs1_sha1, dsa_sha224, dsa_sha1, dsa_sha256, dsa_sha384, dsa_sha512]
"certificate authorities": [CN=ca.your-server.de, CN=ca.dev-broker.your-server.de]
}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:51.812 CEST|X509KeyManagerImpl.java:389|KeyMgr: choosing key: 1 (verified: OK)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.131 CEST|SSLEngineInputRecord.java:177|Raw read (
0000: 16 03 03 00 04 0E 00 00 00 .........
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.131 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 4
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.132 CEST|ServerHelloDone.java:142|Consuming ServerHelloDone handshake message (
<empty>
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.141 CEST|CertificateMessage.java:322|Produced client Certificate handshake message (
"Certificates": [
"certificate" : {...}
]
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.150 CEST|ECDHClientKeyExchange.java:401|Produced ECDHE ClientKeyExchange handshake message (
"ECDH ClientKeyExchange": {
"ecdh public": {...},
}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.387 CEST|CertificateVerify.java:743|Produced CertificateVerify handshake message (
"CertificateVerify": {
"signature algorithm": rsa_pss_rsae_sha256
"signature": {...}
}
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.418 CEST|ChangeCipherSpec.java:109|Produced ChangeCipherSpec message
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.419 CEST|Finished.java:395|Produced client Finished handshake message (
"Finished": {
"verify data": {
...
}'}
)
avax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.422 CEST|SSLEngineOutputRecord.java:507|WRITE: TLS12 handshake, length = 2855
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.427 CEST|SSLEngineOutputRecord.java:525|Raw write (...)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.430 CEST|SSLEngineOutputRecord.java:507|WRITE: TLS12 change_cipher_spec, length = 1
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.431 CEST|SSLEngineOutputRecord.java:525|Raw write (
0000: 14 03 03 00 01 01 ......
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.432 CEST|SSLEngineOutputRecord.java:507|WRITE: TLS12 handshake, length = 16
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.483 CEST|SSLCipher.java:1727|Plaintext before ENCRYPTION (
0000: 14 00 00 0C 91 F1 79 2C A0 3C A8 61 58 45 99 AF ......y,.<.aXE..
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.487 CEST|SSLEngineOutputRecord.java:525|Raw write (
0000: 16 03 03 00 28 00 00 00 00 00 00 00 00 09 AE 57 ....(..........W
0010: EC 20 A9 DD 8F 44 DE 34 3D BE F8 44 F8 D9 7D F0 . ...D.4=..D....
0020: 3F C1 62 92 97 40 2C BD 44 38 E0 8A 7E ?.b..#,.D8...
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.504 CEST|SSLEngineInputRecord.java:177|Raw read (
0000: 14 03 03 00 01 01 ......
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.505 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 change_cipher_spec, length = 1
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.505 CEST|ChangeCipherSpec.java:143|Consuming ChangeCipherSpec message
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.509 CEST|SSLEngineInputRecord.java:177|Raw read (
0000: 16 03 03 00 28 FF 2B 4B EC 5F F7 5C BA 8C 2F 34 ....(.+K._.\../4
0010: 2A C1 FE 1C 6F 92 22 BC B2 B4 E8 93 3C F2 20 48 *...o.".....<. H
0020: 8E FE E2 60 93 7A 3D EA B3 FA 66 05 F2 ...`.z=...f..
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.510 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 handshake, length = 40
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.514 CEST|SSLCipher.java:1629|Plaintext after DECRYPTION (
0000: 14 00 00 0C AB A5 7E 92 19 0D F5 96 FA E2 D5 A4 ................
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.547 CEST|Finished.java:532|Consuming server Finished handshake message (
"Finished": {
"verify data": {
0000: AB A5 7E 92 19 0D F5 96 FA E2 D5 A4
}'}
)
15:44:52.586 [epollEventLoopGroup-2-1] DEBUG io.netty.handler.ssl.SslHandler - [id: 0xf55e3747, L:/10.156.123.175:40054 - R:dev-broker.your-server.de/185.201.147.157:8883] HANDSHAKEN: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.591 CEST|SSLEngineOutputRecord.java:267|WRITE: TLS12 application_data, length = 30
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.592 CEST|SSLCipher.java:1727|Plaintext before ENCRYPTION (
0000: 10 1C 00 04 4D 51 54 54 05 02 00 3C 00 00 0F 63 ....MQTT...<...c
0010: 6F 6E 6E 65 63 74 69 6F 6E 2D 74 65 73 74 onnection-test
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.595 CEST|SSLEngineOutputRecord.java:283|Raw write (
0000: 17 03 03 00 36 00 00 00 00 00 00 00 01 F8 AA 69 ....6..........i
0010: 92 64 CE 34 D2 5D 5A D1 4F 4D D3 34 D8 05 5A F2 .d.4.]Z.OM.4..Z.
0020: C0 8E 63 8B 5A 51 E6 4A 4A A5 B0 E7 61 62 E5 69 ..c.ZQ.JJ...ab.i
0030: 67 84 31 16 09 59 C5 6A 76 4C E1 g.1..Y.jvL.
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.596 CEST|SSLEngineInputRecord.java:177|Raw read (
0000: 15 03 03 00 1A FF 2B 4B EC 5F F7 5C BB 9D 2E 7B ......+K._.\....
0010: FC DB 74 D3 A3 62 BF 86 66 8B 9C 03 04 55 93 ..t..b..f....U.
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.596 CEST|SSLEngineInputRecord.java:214|READ: TLSv1.2 alert, length = 26
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.598 CEST|SSLCipher.java:1629|Plaintext after DECRYPTION (
0000: 01 00 ..
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.600 CEST|Alert.java:232|Received alert message (
"Alert": {
"level" : "warning",
"description": "close_notify"
}
)
javax.net.ssl|WARNING|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.602 CEST|SSLEngineOutputRecord.java:168|outbound has closed, ignore outbound application data
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.603 CEST|SSLEngineOutputRecord.java:507|WRITE: TLS12 alert, length = 2
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.604 CEST|SSLCipher.java:1727|Plaintext before ENCRYPTION (
0000: 01 00 ..
)
javax.net.ssl|DEBUG|19|epollEventLoopGroup-2-1|2019-10-15 15:44:52.609 CEST|SSLEngineOutputRecord.java:525|Raw write (
0000: 15 03 03 00 1A 00 00 00 00 00 00 00 02 30 1D 82 .............0..
0010: 69 FE 1A 9E BE 35 47 37 9E 67 DF 85 5A F4 E5 i....5G7.g..Z..
)
15:44:52.631 [epollEventLoopGroup-2-1] INFO com.you.mqtt.ConnectionListener - Disconnected from MQTT broker.
15:44:52.641 [epollEventLoopGroup-2-1] ERROR com.you.mqtt.ConnectionListener - Server closed connection without DISCONNECT. Will try to reconnect.
15:44:52.666 [RxComputationThreadPool-1] ERROR com.you.mqtt.MqttConnector - connection exception
com.hivemq.client.mqtt.exceptions.ConnectionClosedException: Server closed connection without DISCONNECT.
I'm using Java 11 with the HiveMQ MQTT client version 1.1.2 (latest in maven repo).
The server is a HAProxy, the port is 8883.
I would be really thankful, if someone has a hint or a clue about what is happening here and why the client does not connect to the the MQTT broker behind the HAProxy.
HAProxy Config
The following snippet is the part of the haproxy configuration concerned with TCP connection. The configuration follows recommendations for the VerneMQ broker.
global
daemon
user haproxy
group haproxy
chroot /var/empty
maxconn 20000
#log gi18hd.stackhero-network.com:514 local1 debug
log /dev/log local0 debug
#ssl
tune.ssl.default-dh-param 2048
#ssl-default-bind-options no-sslv3 no-tls-tickets force-tlsv12
#ssl-default-bind-ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
defaults
log global
#option httplog
option tcplog
timeout connect 5s
timeout client 30s
timeout client-fin 30s
timeout server 300s
timeout tunnel 1h
frontend front_tcp
mode tcp
option clitcpka # For TCP keep-alive
log global
option tcplog
bind *:8883 ssl crt-list /etc/haproxy/server-dev-test-crt-list.txt
use_backend sub_dev-broker if { ssl_fc_sni -i dev-broker.your-server.de }
backend sub_dev-broker
mode tcp
option redispatch
balance leastconn
server manager1 10.1.2.51:1883 check fall 1
server manager2 10.1.2.52:1883 check fall 1
server manager3 10.1.2.53:1883 check fall 1
VerneMQ Config
Our broker is a clustered(!) VerneMQ MQTT broker (https://vernemq.com/) in a Docker Swarm. The MQTT 5 protocol is enabled via the env variable DOCKER_VERNEMQ_LISTENER__TCP__ALLOWED_PROTOCOL_VERSIONS=3,4,5.
We use an internal listener on port 18883, which is mapped outside the container to 1883. Again: this setup is working with tools like MQTTFx and other Paho-based MQTT3 clients.
The corresponding docker-compose configuration is:
version: '3.7'
services:
mqttbroker0:
image: erlio/docker-vernemq
networks:
- your-backend
environment:
- DOCKER_VERNEMQ_SWARM=1
- DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on
- DOCKER_VERNEMQ_LOG__CONSOLE__LEVEL=debug
mqttbroker:
image: erlio/docker-vernemq
depends_on:
- mqttbroker0
networks:
- your-backend
environment:
- DOCKER_VERNEMQ_SWARM=1
- DOCKER_VERNEMQ_DISCOVERY_NODE=mqttbroker0
- DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on
- DOCKER_NET_INTERFACE=eth1
- DOCKER_VERNEMQ_LOG__CONSOLE__LEVEL=debug
- DOCKER_VERNEMQ_LISTENER__TCP__SERVER = 0.0.0.0:18883
- DOCKER_VERNEMQ_LISTENER__TCP__ALLOWED_PROTOCOL_VERSIONS=3,4,5
ports:
- "1883:18883"
deploy:
replicas: 2
networks:
your-backend:
external: true
Update: TLS handshake is ok, haproxy can't find the server
The connection is closed after the handshake between client and server. HAProxy logs
front_tcp~ front_tcp/<NOSRV> -1/-1/965 0 SC 2/1/0/0/0 0/0
We have a condition in our haproxy configuration: use_backend sub_dev-broker if { ssl_fc_sni -i dev-broker.your-server.de }. This enables us to use only one haproxy instance and to switch between different endpoints (DEV and TEST servers in our case).
The request coming from the client after the handshake can not be routed to a backend server. HAProxy reacts with NOSRV. It seems that the request does not contain the hostname of the server, so the if-condition fails.
Update: HiveMQ Client 1.1.2 does not use SNI
The HiveMQ Client 1.1.2 does not use the Server Name Indication TLS extension, Milestone 1.1.3 does. We use the SNI information for the switch in our HAProxy configuration. As I see no workaround at the moment and this functionality is not critical in the MQTT subscriber, I will wait for the next release of the HiveMQ client.
Thank you to all participants for thoughts and hints!
As I understand the provided log, there seems to be nothing wrong with the TLS handshake.
After the handshake is finished the client sends a MQTT CONNECT packet. As you are using the HiveMQ client with MQTT 5 as protocol version, this will contain a flag that MQTT 5 is expected.
Now if the used broker can't handle that, it will close the connection (protocol error).
This would also explain, why you were successful in connecting
with MQTTFx and with a MQTT3 Paho.
Alternatively you could change the MQTT client creation to:
return MqttClient.builder()
...
.useMqttVersion3()
.buildAsync();
But as your original goal was to use the shared subscriptions a broker change/upgrade would still be necessary.
Could you perhaps provide more information about the used broker?

Connect to Windows SQL Server 2008 R2 from Rails app

My Rails 4.2.1 app has to connect to a Microsoft SQL 2008 R2 database. I am using the tiny_tds gem version 1.0.4. FreeTDS v1.00.15 is installed on the production server running Ubuntu 14.04.
I run queries inside an each loop and I can't get the loop to complete, the process crashes before completion.
I tried playing with tiny_tds options without success.
Here's the code I am using to get the tiny_tds client (check tds_version and timeout options):
client = TinyTds::Client.new(username: db_conf['username'], password: db_conf['password'], host: db_conf['host'], port: db_conf['port'], database: db_conf['database'], tds_version: '7.3', timeout: 15000, appname: 'ERP')
Here's the FreeTDS log after such an error happens.
packet.c:741:Sending packet 0000 12 01 00 ce 00 00 00 00-16 03 01 00
86 10 00 00 |........ ........| 0010 82 00 80 6e d9 e2 dc 97-9d 77 59
9a 5b da e3 e2 |...n.... .wY.[...| 0020 8b aa 66 ed ec 5e e2 02-e5 6c
fd db e1 ef 47 1a |..f..^.. .l....G.| 0030 9d 63 03 ed 6d 3e 28 3b-b9
64 fd 92 71 34 ff ba |.c..m>(; .d..q4..| 0040 7d 3c 8d ee 7b 34 75
e9-d5 b7 c6 83 a9 7d e6 7f |}<..{4u. .....}..| 0050 71 7e 25 11 82 b8
76 b1-c6 ba 86 b4 c3 0a 47 f0 |q~%...v. ......G.| 0060 51 96 c7 e2 5f
ca 07 b2-95 53 b9 9e bb 2c e7 cb |Q..._... .S...,..| 0070 be 0a b5 eb
b0 f3 41 1d-cd 86 fc a6 53 08 5e 56 |......A. ....S.^V| 0080 29 85 79
14 dc 2b 74 7b-b2 43 2c e8 0e 87 60 e4 |).y..+t{ .C,....| 0090 10 ef
f8 14 03 01 00 01-01 16 03 01 00 30 c7 f0 |........ .....0..| 00a0 35
f5 2c 6e 79 8d 85 b9-bd 60 b7 09 8c 7e 29 18 |5.,ny... ....~).| 00b0
4a 56 ea c3 4e 13 bf e3-c5 8d f6 68 31 31 54 ee |JV..N... ...h11T.|
00c0 bf 2f 75 8d e9 9e c0 a9-d0 d2 9e 5b c9 92 |./u..... ...[..|
tls.c:105:in tds_pull_func_login query.c:3796:tds_disconnect()
util.c:165:Changed query state from IDLE to DEAD
util.c:322:tdserror(0x80b75e0, 0xa04ca80, 20017, 0)
dblib.c:7947:dbperror(0xae62780, 20017, 0) dblib.c:8015:dbperror:
Calling dblib_err_handler with msgno = 20017; msg->msgtext =
"Unexpected EOF from the server (192.168.32.105:1433)"
dblib.c:5777:dbgetuserdata(0xae62780) dblib.c:8037:dbperror:
dblib_err_handler for msgno = 20017; msg->msgtext = "Unexpected EOF
from the server (192.168.32.105:1433)" -- returns 2 (INT_CANCEL)
util.c:352:tdserror: client library returned TDS_INT_CANCEL(2)
util.c:375:tdserror: returning TDS_INT_CANCEL(2) util.c:375:tdserror:
returning TDS_INT_CANCEL(2) tls.c:942:handshake failed
login.c:530:login packet rejected util.c:322:tdserror(0x80b75e0,
0xa04ca80, 20002, 0) dblib.c:7947:dbperror(0xae62780, 20002, 0)
dblib.c:8015:dbperror: Calling dblib_err_handler with msgno = 20002;
msg->msgtext = "Adaptive Server connection failed"
And here's the output of tsql -C:
~$ tsql -C
Compile-time settings (established with the "configure" script)
Version: freetds v1.00.15
freetds.conf directory: /usr/local/etc
MS db-lib source compatibility: no
Sybase binary compatibility: no
Thread safety: yes
iconv library: yes
TDS version: auto
iODBC: no
unixodbc: no
SSPI "trusted" logins: no
Kerberos: no
OpenSSL: yes
GnuTLS: no
MARS: no
Any idea what I should do to fix those Unexpected EOF from the server errors?
In your FreeTDS configuration (often in /etc/freetds/freetds.conf as in your configuration), change the value of text size:
text size = 4294967295
That's the maximum value, IIRC. I believe with FreeTDS 0.91 that your default is probably 64512.
Looking at the SQL Profiler, I found out the Rails application was opening way too many connections on the MSSQL server. Upon reaching its max number of open connection, the MSSQL server refused opening any new connection, resulting in the Unexpected EOF from the server error.
To solve the issue, I had to reuse my open connection when sending queries instead of opening a new connection for each query. I guess this is the correct way to use the tiny_tds connector anyway.
Translated to code:
def self.get_pmi_client
if ##pmi_client.nil? or !##pmi_client.active?
db_conf = Rails.configuration.database_configuration["pmi_#{Rails.env}"]
##pmi_client = TinyTds::Client.new(username: db_conf['username'], password: db_conf['password'], host: db_conf['host'], port: db_conf['port'], database: db_conf['database'])
raise MSSQLConnectionError, t('erp.errors.pmi_connection_error') unless ##pmi_client.active?
end
return ##pmi_client
end

Apple push notification not working for distribution/ad hoc testing

I have been using sandbox push notification for a while, and it is working perfectly. After switching to distribution, push notification stopped working.
Things that I have checked:
Two separate push notification certificates are created, one for development (Sandbox) and one for distribution(production).
Device token is different for sandbox and production.
We use NotNoop APNS on server side. The certificate.p12 is created by exporting the private key under push notification certificate in Apple Keychain Access. Certificate itself is not in the p12 file.
Checked archived package for production with command
codesign -d --entitlements :- "Payload/YourApp.app"
And the code signing entitlement is distribution.
Followed Apple TN 2265 (https://developer.apple.com/library/ios/technotes/tn2265/_index.html). While using openssl to connect to APNS, the connection is closed right after handshake (for production). For sandbox, there is a pause after the handshake, and then more reads & writes.
My command:
> openssl pkcs12 -in Cert_prod.p12 -out Cert_prod.pem
> openssl s_client -connect gateway.push.apple.com:2195 -cert Cert_prod.pem -debug -showcerts -CAfile "Entrust.pem"
The output (for production):
...
Server certificate
subject=/C=US/ST=California/L=Cupertino/O=Apple Inc./CN=gateway.push.apple.com
issuer=/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
---
Acceptable client certificate CA names
/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Root CA
/C=US/O=Apple Inc./OU=Apple Worldwide Developer Relations/CN=Apple Worldwide Developer Relations Certification Authority
/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Application Integration Certification Authority
---
SSL handshake has read 3144 bytes and written 2161 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES256-SHA
Session-ID:
Session-ID-ctx:
Master-Key: CCCE22D7487589D257E547F2693E8AEDF86B693E6EFE09DCAA7EE74C28812E45506A21B0DC0ED6CAAF4395EB2D2899ED
Key-Arg : None
Start Time: 1454828210
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
read from 0x7fa6d3a00060 [0x7fa6d4806600] (5 bytes => 5 (0x5))
0000 - 15 03 01 ...
0005 - <SPACES/NULS>
read from 0x7fa6d3a00060 [0x7fa6d4806605] (32 bytes => 32 (0x20))
0000 - 8f c6 74 25 a6 40 2b 58-29 bc a4 e0 a7 c9 43 a8 ..t%.#+X).....C.
0010 - a0 93 62 78 6a 73 ed aa-4f 61 0a a4 9e fe a7 9e ..bxjs..Oa......
closed
write to 0x7fa6d3a00060 [0x7fa6d480b000] (37 bytes => 37 (0x25))
0000 - 15 03 01 00 20 2c 70 db-af 76 7a 2c 01 70 da 0f .... ,p..vz,.p..
0010 - a9 03 da e2 fb 69 f8 09-63 5d 18 85 81 8c 31 63 .....i..c]....1c
0020 - 63 fb 73 cb 48 c.s.H
The output (for sandbox):
...
Server certificate
subject=/C=US/ST=California/L=Cupertino/O=Apple Inc./CN=gateway.sandbox.push.apple.com
issuer=/C=US/O=Entrust, Inc./OU=www.entrust.net/rpa is incorporated by reference/OU=(c) 2009 Entrust, Inc./CN=Entrust Certification Authority - L1C
---
Acceptable client certificate CA names
/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Root CA
/C=US/O=Apple Inc./OU=Apple Worldwide Developer Relations/CN=Apple Worldwide Developer Relations Certification Authority
/C=US/O=Apple Inc./OU=Apple Certification Authority/CN=Apple Application Integration Certification Authority
---
SSL handshake has read 3160 bytes and written 2161 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1
Cipher : AES256-SHA
Session-ID:
Session-ID-ctx:
Master-Key: 011B90857BC02D4A9DCB7300F8C9F368905F90EA4BDDC0677CD2EA9AAB3550CF750F0B57C5AF0C72F5BCD991658AC4AC
Key-Arg : None
Start Time: 1454828396
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 d7 a1 a1-b2 9e be ce 49 18 26 0e .... .......I.&.
0010 - 46 73 f9 dc 7f b2 75 71-f6 bf 51 65 44 0e f1 60 Fs....uq..QeD..`
0020 - fe 80 2e e2 a6 17 03 01-00 20 f0 be dc 70 85 b0 ......... ...p..
0030 - 82 d0 44 1e 9a b8 fe 59-39 b1 14 be 0a 9d 7c 0e ..D....Y9.....|.
0040 - ce 09 9b c3 f2 1c dd 28-c7 24 .......(.$
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 4c 12 4f-f9 ba 0b e4 80 b3 66 bb .... L.O......f.
0010 - 93 a7 c9 47 9d 84 36 88-33 c0 80 79 a2 97 c6 95 ...G..6.3..y....
0020 - 2a 4b 42 c1 bc 17 03 01-00 20 a0 4d 06 fd 84 78 *KB...... .M...x
0030 - 3d b9 12 ad 65 75 b6 1a-01 91 28 86 40 c4 e9 c2 =...eu....(.#...
0040 - 38 91 47 42 c3 1d e3 c7-f9 41 8.GB.....A
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 35 77 28-ea b0 80 5a ad 9e 99 87 .... 5w(...Z....
0010 - 7d 07 13 14 ff ea a8 48-60 7c 46 f0 59 07 c8 53 }......H`|F.Y..S
0020 - 3f 96 55 fc ec 17 03 01-00 20 60 49 00 86 bf 41 ?.U...... `I...A
0030 - 70 70 5c d5 d7 f5 83 4c-8a b3 10 22 bb ad 4c f2 pp\....L..."..L.
0040 - 3e e0 54 db 5d e7 2e bc-cc 9c >.T.].....
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 25 d3 8a-20 a4 4c 3c b6 f7 fe 95 .... %.. .L<....
0010 - 60 e2 5b 9f fc fd 3d e7-4c 27 cc d0 8e 6c ef 30 `.[...=.L'...l.0
0020 - 56 4a 88 a9 f9 17 03 01-00 20 cc 32 2d d5 e6 ff VJ....... .2-...
0030 - 59 26 14 67 2d 3f 3d 88-d3 3b ea 53 5e 11 72 75 Y&.g-?=..;.S^.ru
0040 - a4 c6 0e d8 95 b4 21 e3-5d dd ......!.].
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (74 bytes => 74 (0x4A))
0000 - 17 03 01 00 20 8a a0 b6-fd 72 7b f4 f6 ef de 65 .... ....r{....e
0010 - 76 40 6b 67 1d b8 83 c1-92 98 50 9e 0f ac d7 da v#kg......P.....
0020 - 66 39 9c 39 9f 17 03 01-00 20 70 a4 1d 63 f1 6e f9.9..... p..c.n
0030 - d6 5a b3 fd f9 7e de de-64 7d 42 0d da 94 59 09 .Z...~..d}B...Y.
0040 - 93 3f 33 e3 d7 d0 3e 2e-10 e7 .?3...>...
read from 0x7f9b19c1aef0 [0x7f9b1a014600] (5 bytes => 5 (0x5))
0000 - 15 03 01 ...
0005 - <SPACES/NULS>
read from 0x7f9b19c1aef0 [0x7f9b1a014605] (32 bytes => 32 (0x20))
0000 - 60 b6 57 1c 8a 8e e9 f0-59 b5 27 73 f5 ba de b1 `.W.....Y.'s....
0010 - 7a ef d7 ab 79 23 0a ea-b2 13 c9 ca 98 e5 c3 36 z...y#.........6
closed
write to 0x7f9b19c1aef0 [0x7f9b1a019000] (37 bytes => 37 (0x25))
0000 - 15 03 01 00 20 f2 6f d0-1b 8d 41 5e 44 df f5 49 .... .o...A^D..I
0010 - 3a 09 6a 11 2b 3b f3 47-1e 70 8d 8d a8 9d 42 45 :.j.+;.G.p....BE
0020 - e9 47 18 26 3e .G.&>
Therefore, it seems that the broken link is between my server and APNS. How can I know why the connection is closed?
I suspected that something is wrong on the certificate, but could not find anything. I took the same steps to create sandbox certificates, and they worked just fine. Did I miss anything for production?
Other related links that I checked:
apple push notification not working in production
http://www.raywenderlich.com/32960/apple-push-notification-services-in-ios-6-tutorial-part-1
Couldn't able to connect to APNS Sandbox server
The certificate.p12 in NotNoop APNS should be exported from the certificate only, without the private key. In Keychain Access, click the small arrow next to the push notification certificate to show the private key, select the certificate only (without the key), and export it to a .p12 file. Updating the file on server solved my problem. Openssl is a good tool for smoke test before archiving the app and running it in ad hoc mode.

Resources