I have a setup as follows to inspect traffic from a device.
Device <---> MitmProxy (Transparent Proxy mode) + Wireshark <---> Internet
Command used to invoke MitmProxy: SSLKEYLOGFILE="$PWD/mitmproxy/sslkeylogfile.txt" mitmweb --mode transparent --showhost
MitmProxy supports logging SSL/TLS master keys which allows Wireshark to decrypt the corresponding TLS traffic.
However, I noticed that the contents of my SSLKEYLOGFILE contained only lines of CLIENT_RANDOM which decrypts TLS 1.2 traffic.
To decrypt TLS 1.3 traffic, I understand that 4 secrets - CLIENT_HANDSHAKE_TRAFFIC_SECRET, SERVER_HANDSHAKE_TRAFFIC_SECRET, CLIENT_TRAFFIC_SECRET and SERVER_TRAFFIC_SECRET are needed, all of which are absent from my SSLKEYLOGFILE.
How may I proceed with decrypting TLS 1.3 in this setup?
This currently is a known bug in mitmproxy, see https://github.com/mitmproxy/mitmproxy/issues/3994. We're waiting for upstream to expose the new OpenSSL hooks.
Related
I am running mosquitto broker on ubuntu on aws ec2, using TLS over port 8883. For example :
mosquitto_sub -h domain.com -t topic --cafile /etc/mosquitto/certs/ca.crt -p 8883
The client is esp8266 using TLS configuration.
Is it possible for someone to still intercept or read the outgoing or incoming mqtt message?
All communication is on port 8883 using a certificate.
(1) The most trivial answer with MQTT context is:
Use another MQTT client and subscribe to '#' to see all messages your broker receives from anywhere.
If you know the topics your esp8266 client is publishing or subscribed to,
subscribe to all these with the other client (e.g. with 'mosquitto_sub').
The broker provides messages on the topic "topic" mentioned above to all subscribed clients, independent from their communication channel security.
This way you get the messages for the esp8266 with any other client.
For example, if the unencrypted channel is not deactivated: mosquitto_sub -h domain.com -t topic -p 1883
TLS ensures encryption for the communication channel only, and not for the actual payload before putting it onto the channel between client and server.
(2) If (1) is not the expected answer and you rather want to wiretap the encrypted communication:
Short answer is: no, not with trivial tooling; that's the purpose of TLS.
A good article for MQTT security fundamentals: https://www.hivemq.com/blog/mqtt-security-fundamentals-tls-ssl/
Long answer is:
[Disclaimer: no assumptions about advanced abilities of 3rd parties]
The TLS communication is properly encrypted, and to decrypt the MQTT communication requires the session secrets client and server use after
their successful TLS handshake.
A solutions requires you as the operator of the broker.
There is a short article about how to do it:
https://adrianalin.gitlab.io/popsblog.me/posts/decrypt-mqtt-tls-traffic/
In essence:
You can use Wireshark to eavesdrop the traffic, if you configure the session secrets there.
To get the secrets you adjust and re-build mosquitto according the article, so that your broker print the session secrets.
(Whether you are able to setup this with root privileges on your EC2 instance in AWS, is another question).
I want to make beat input work with TLS client authentication without it works
So I made custom graylog image with selfsigned certificates
FROM graylog/graylog:4.3
USER root
ADD beat.crt /usr/local/share/ca-certificates/beat.crt
RUN chmod 644 /usr/local/share/ca-certificates/beat.crt && update-ca-certificate
Next I made beat input with tls auth requared
bind_address: 0.0.0.0
no_beats_prefix: true
number_worker_threads: 80
override_source: <empty>
port: 5044
recv_buffer_size: 1048576
tcp_keepalive: false
tls_cert_file: <empty>
tls_client_auth: required
tls_client_auth_cert_file: /usr/local/share/ca-certificates/beat.crt
tls_enable: false
tls_key_file: <empty>
tls_key_password:********
And set filebeat on another machine
folder "tls" added as volume when running filebeat in docker --volume="/home/filebeat/:/tls"
output.logstash:
hosts: ["graylog_ip_here:5044"]
ssl.certificate_authorities: ["/tls/beat.pem"]
ssl.certificate: "/tls/beat.crt"
ssl.key: "/tls/beat.key"
beat crt look inside like so pem is the same file
-----BEGIN CERTIFICATE-----
MIIFVzCCAz+gAwIBAgIJALJI6zP
After all this had been set I'm getting error on graylog server
ERROR: org.graylog2.plugin.inputs.transports.AbstractTcpTransport - Error in Input cause io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Unknown beats protocol version: 3)
As said in documentation here you should apply both steps to make it work. The first step is to set up a TLS exchange. The second is for authenticating specific users.
TLS Beats Input
To enable TLS on the input, a certificate (and private key file) is needed. It can be the same or
a different certificate as the one of your REST/web interface, as long as it matches all hostnames
of your input. Just reference the files TLS cert file and TLS private key file in the Beats Input
configuration and restart the input.
The ingesting client will verify the presented certificate against his know CA certificates,
if that is successful communication will be established using TLS.
Add client authentication to Beats input
Create one directory (/etc/graylog/server/trusted_clients ) that will hold all client certificates
you allow to connect to the beats input. This directory must be available on all Graylog server
that have the input enabled. Write that path in the beats input configuration
TLS Client Auth Trusted Certs and select required for the option TLS client authentication.
After this setting is saved only clients that provide a certificate that is trusted by the CA
and is placed inside the configured directory (/etc/graylog/server/trusted_clients)
can deliver messages to Graylog.
Does Azure IoT Edge works with Zscaler proxy AND SSL inspection
Indeed, we need to check header information from IoT Edge for security reason which require to decrypt .
IoT Edge run in CentOS or Ubuntu ESX VM and Zscaler certificat can be uploaded in the OS file system
regards
You will need the Zscaler root certificate added to list of root certificates that are trusted by OpenSSL in both the edgeAgent and edgeHub modules. Without that the TLS negotiation will fail with not trusting the certificate chain from any server that they attempt to negotiation a TLS connection with. I have done this. It will work.
I am new to MQTT and I have a frustrating problem.
I have been using MQTT.fx to subscribe to a topic; I have set the:
Broker Address
Port
Client ID
Enable SSL/TLS
Topic
This works well, however I would like to use mosquitto_sub.
I am attempting to subscribe to the same topic in the following way:
mosquitto_sub -h host -p 8883 -t topic -i client id
This is not working for me. I am using it on a Ubuntu VM.
My powers of observation tell me that I should enable TLS, however I'm not quite sure how to do that, I have stuffed around with certificates and enabling TLS in may ways but have not got the right combo. I know it is required as if I uncheck the SSL/TLS box in MQTT.fx I am unable to connect.
I would really like to replicate what I have in MQTT.fx with mosquitto.
In the mosquitto_sub command, use the --capath argument to point to /etc/ssl/certs. It needs a pointer to the trusted certificates.
To enable SSL with mosquitto_sub you need to specify a CA certificate.
This can be done in 1 of 2 ways.
--cafile /path/to/a/file where the file contains the required trusted CA certificate (either on it's own or part of a concatenated set)
--capath /path/to/directory where the directory contains a collection of files ending in .crt which contain the CA certificates to be trusted. The ca certs should also be indexed with the c_rehash function.
Both these options are mentioned in the mosquito_sub man page as ways to enabled SSL
e.g.
mosquitto_sub -h host -p 8883 --cafile ca.crt -t topic -i client id
i am aware of a 3rd way (short cut) which is using the flag --tls-use-os-certs
also as a side note, mosquitto_sub/pub also sends SNI within the tls connection request, which is great news if you are using SNI based routing on the broker side.
don't know if the MQTT standard actually prescribes this, but at least mosquitto client's implementation does support it
We are developing an application which needs to consume AWS IoT service based on a MQTT protocol deviation. We are currently facing issues to get connected with MQTT broker provided by AWS IoT cloud server.
Following is the environment:
iOS Version: 8.0 / 9.0
Programming language: Swift
Library for MQTT: Moscapsule
Steps followed:
Set initial config clientid, host, port
Set client certificate with private key, providing .pem file path (e.g. cert.pem, privateKey.pem)
Set server certificate which is root certificate .pem file path (e.g. rootCA.pem)
Set tls opts with tsl_insecure: false, cert_reqs: SSL_VERIFY_PEER, tls version: tlsv1.2, ciphers: nil
Problems faced:
When trying to connect to server/broker gives error “unable to create TLS_Context”.
With setting tls cert_reqs: SSL_VERIFY_NONE, gives connection status success with subcribe and publish sucess, but doesn’t reflect on server or broker.
Any help in this context is highly appreciable.
The AWS SDK for iOS already supports connecting to AWS IoT over MQTT. You can see an example Swift program which transfers data to and from AWS IoT over MQTT using certificate-based authentication here. If you'd like to use a different MQTT client and just need to know how to set it up, you might start with the AWS SDK for iOS, and then have a look at the code involved in setting up the TLS connection.
Thanks for using AWS IoT.