I am new to MQTT and I have a frustrating problem.
I have been using MQTT.fx to subscribe to a topic; I have set the:
Broker Address
Port
Client ID
Enable SSL/TLS
Topic
This works well, however I would like to use mosquitto_sub.
I am attempting to subscribe to the same topic in the following way:
mosquitto_sub -h host -p 8883 -t topic -i client id
This is not working for me. I am using it on a Ubuntu VM.
My powers of observation tell me that I should enable TLS, however I'm not quite sure how to do that, I have stuffed around with certificates and enabling TLS in may ways but have not got the right combo. I know it is required as if I uncheck the SSL/TLS box in MQTT.fx I am unable to connect.
I would really like to replicate what I have in MQTT.fx with mosquitto.
In the mosquitto_sub command, use the --capath argument to point to /etc/ssl/certs. It needs a pointer to the trusted certificates.
To enable SSL with mosquitto_sub you need to specify a CA certificate.
This can be done in 1 of 2 ways.
--cafile /path/to/a/file where the file contains the required trusted CA certificate (either on it's own or part of a concatenated set)
--capath /path/to/directory where the directory contains a collection of files ending in .crt which contain the CA certificates to be trusted. The ca certs should also be indexed with the c_rehash function.
Both these options are mentioned in the mosquito_sub man page as ways to enabled SSL
e.g.
mosquitto_sub -h host -p 8883 --cafile ca.crt -t topic -i client id
i am aware of a 3rd way (short cut) which is using the flag --tls-use-os-certs
also as a side note, mosquitto_sub/pub also sends SNI within the tls connection request, which is great news if you are using SNI based routing on the broker side.
don't know if the MQTT standard actually prescribes this, but at least mosquitto client's implementation does support it
Related
I am running mosquitto broker on ubuntu on aws ec2, using TLS over port 8883. For example :
mosquitto_sub -h domain.com -t topic --cafile /etc/mosquitto/certs/ca.crt -p 8883
The client is esp8266 using TLS configuration.
Is it possible for someone to still intercept or read the outgoing or incoming mqtt message?
All communication is on port 8883 using a certificate.
(1) The most trivial answer with MQTT context is:
Use another MQTT client and subscribe to '#' to see all messages your broker receives from anywhere.
If you know the topics your esp8266 client is publishing or subscribed to,
subscribe to all these with the other client (e.g. with 'mosquitto_sub').
The broker provides messages on the topic "topic" mentioned above to all subscribed clients, independent from their communication channel security.
This way you get the messages for the esp8266 with any other client.
For example, if the unencrypted channel is not deactivated: mosquitto_sub -h domain.com -t topic -p 1883
TLS ensures encryption for the communication channel only, and not for the actual payload before putting it onto the channel between client and server.
(2) If (1) is not the expected answer and you rather want to wiretap the encrypted communication:
Short answer is: no, not with trivial tooling; that's the purpose of TLS.
A good article for MQTT security fundamentals: https://www.hivemq.com/blog/mqtt-security-fundamentals-tls-ssl/
Long answer is:
[Disclaimer: no assumptions about advanced abilities of 3rd parties]
The TLS communication is properly encrypted, and to decrypt the MQTT communication requires the session secrets client and server use after
their successful TLS handshake.
A solutions requires you as the operator of the broker.
There is a short article about how to do it:
https://adrianalin.gitlab.io/popsblog.me/posts/decrypt-mqtt-tls-traffic/
In essence:
You can use Wireshark to eavesdrop the traffic, if you configure the session secrets there.
To get the secrets you adjust and re-build mosquitto according the article, so that your broker print the session secrets.
(Whether you are able to setup this with root privileges on your EC2 instance in AWS, is another question).
I want to make beat input work with TLS client authentication without it works
So I made custom graylog image with selfsigned certificates
FROM graylog/graylog:4.3
USER root
ADD beat.crt /usr/local/share/ca-certificates/beat.crt
RUN chmod 644 /usr/local/share/ca-certificates/beat.crt && update-ca-certificate
Next I made beat input with tls auth requared
bind_address: 0.0.0.0
no_beats_prefix: true
number_worker_threads: 80
override_source: <empty>
port: 5044
recv_buffer_size: 1048576
tcp_keepalive: false
tls_cert_file: <empty>
tls_client_auth: required
tls_client_auth_cert_file: /usr/local/share/ca-certificates/beat.crt
tls_enable: false
tls_key_file: <empty>
tls_key_password:********
And set filebeat on another machine
folder "tls" added as volume when running filebeat in docker --volume="/home/filebeat/:/tls"
output.logstash:
hosts: ["graylog_ip_here:5044"]
ssl.certificate_authorities: ["/tls/beat.pem"]
ssl.certificate: "/tls/beat.crt"
ssl.key: "/tls/beat.key"
beat crt look inside like so pem is the same file
-----BEGIN CERTIFICATE-----
MIIFVzCCAz+gAwIBAgIJALJI6zP
After all this had been set I'm getting error on graylog server
ERROR: org.graylog2.plugin.inputs.transports.AbstractTcpTransport - Error in Input cause io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Unknown beats protocol version: 3)
As said in documentation here you should apply both steps to make it work. The first step is to set up a TLS exchange. The second is for authenticating specific users.
TLS Beats Input
To enable TLS on the input, a certificate (and private key file) is needed. It can be the same or
a different certificate as the one of your REST/web interface, as long as it matches all hostnames
of your input. Just reference the files TLS cert file and TLS private key file in the Beats Input
configuration and restart the input.
The ingesting client will verify the presented certificate against his know CA certificates,
if that is successful communication will be established using TLS.
Add client authentication to Beats input
Create one directory (/etc/graylog/server/trusted_clients ) that will hold all client certificates
you allow to connect to the beats input. This directory must be available on all Graylog server
that have the input enabled. Write that path in the beats input configuration
TLS Client Auth Trusted Certs and select required for the option TLS client authentication.
After this setting is saved only clients that provide a certificate that is trusted by the CA
and is placed inside the configured directory (/etc/graylog/server/trusted_clients)
can deliver messages to Graylog.
I want to find mosquitto_sub/pub command option like to cert_reqs option in
mosquitto_tls_opts_set(mosq, cert_reqs, tls_version, ciphers); // C++ Code
I want to connect to the broker using only the ca.crt file.
When I set cert_reqs=0. I can connect to the broker using only the ca.crt file.
But I couldn't find this option in mosquitto_sub/pub cmd.
From the mosquitto_sub man page:
--insecure
When using certificate based encryption, this option disables verification of the server hostname in the server certificate. This
can be useful when testing initial server configurations but makes it
possible for a malicious third party to impersonate your server
through DNS spoofing, for example. Use this option in testing only. If
you need to resort to using this option in a production environment,
your setup is at fault and there is no point using encryption.
I have a kafka connect cluster (cp_kafka_connect_base) on docker, and I need to include a .pem file in order to connect to a source over TLS. It seems there are already a number of trusted certificates included in connect, so how would I add a new trusted certificate without invalidating the old ones?
Specific problem
I want to use MongoDB Source Connector, alongside a number of other connectors. As per documentation, I have imported my .pem certificate in a .jks store, and added the following envvars to my kafka connect containers:
KAFKA_OPTS="
-Djavax.net.ssl.trustStore=mystore.jks
-Djavax.net.ssl.trustStorePassword=mypass
This lets me connect to my data source, but invalidates other TLS connections, unless I add them all to my .jks. Since all other TLS connections work out of the box, I shouldn't need to manually import every single one of them to a .jks, just to make one connector implementation happy.
I have also tried setting:
CONNECT_SSL_TRUSTSTORE_TYPE: "PEM"
CONNECT_SSL_TRUSTSTORE_LOCATION: "myloc"
but the truststore location config isn't known, and TLS doesn't work:
WARN The configuration 'ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig:384)
I'm trying to enroll an intermediate certificate authority with its root certificate authority, I'm using TLS and the CAs run in separated docker containers using the fabric-ca images. If I send the CSR trough the docker network there are no problems, but if I try to send it trough my local network I have this problem: x509: cannot validate certificate for "HOST_IP" because it doesn't contain any IP SANs. I read that the reason for my problem, is that the TLS certificate(s) don't contain IP SANs (IP subject alternative names), this is a field in the TLS certificate that basically says: "The certificate was issued to a host that its IP is: <1.2.3.4>". So, if that is the problem, how I could add this field to my certificate? and if not, what is the problem and how I could resolve it? Thanks and regards!
You can follow the explanation given here with a similar IP SANS issue. There is also detailed explanation of IP SANS issue in HLF mailing List. Also make sure when you are using Fabric CA Client, to generate the certificates, use the server's hostname to request the same. for e.g. fabric-ca-client enroll -d --enrollment.profile tls -c <client configuration file> -u https://admin:adminpw#<server hostname>:7054 --csr.hosts $PEER_HOST. If IP address is given in URL TLS handshake will fail with bad TLS certificate error on the server side and 'certificate does not contain IP SANS' error on the client side as the ca-cert.pem used to perform the handshake with the server contains hostname and not the IP address.