AWS IoT: Use MQTT on port 443 - mqtt

Question

I am trying to setup AWS IoT in Pi on port 443 using Paho MQTT .
As AWS document (https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html) mentioned that
Clients wishing to connect using MQTT with X.509 Client Certificate authentication on port 443 must implement the Application Layer Protocol Negotiation (ALPN) TLS extension and pass x-amzn-mqtt-ca as the ProtocolName in the ProtocolNameList.
I actually don't know how to achieve it properly in Paho MQTT (https://github.com/eclipse/paho.mqtt.python)
What I tried to do (mqtt_apln.py)
import sys
import ssl
import time
import datetime
import logging, traceback
import paho.mqtt.client as mqtt
MQTT_TOPIC = "topictest"
MQTT_MSG = "hello MQTT"
IoT_protocol_name = "x-amzn-mqtt-ca"
aws_iot_endpoint = "xxxxxxx.iot.eu-west-1.amazonaws.com"
url = "https://{}".format(aws_iot_endpoint)
ca = ".xxxxx/rootCA.pem"
cert = ".xxxxx/xxxxx-certificate.pem.crt"
private = ".xxxxx/xxxxxx-private.pem.key"
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(log_format)
logger.addHandler(handler)
# Define on connect event function
# We shall subscribe to our Topic in this function
def on_connect(mosq, obj, rc):
mqttc.subscribe(MQTT_TOPIC, 0)
# Define on_message event function.
# This function will be invoked every time,
# a new message arrives for the subscribed topic
def on_message(mosq, obj, msg):
print "Topic: " + str(msg.topic)
print "QoS: " + str(msg.qos)
print "Payload: " + str(msg.payload)
def on_subscribe(mosq, obj, mid, granted_qos):
print("Subscribed to Topic: " +
MQTT_MSG + " with QoS: " + str(granted_qos))
def ssl_alpn():
try:
#debug print opnessl version
logger.info("open ssl version:{}".format(ssl.OPENSSL_VERSION))
ssl_context = ssl.create_default_context()
ssl_context.set_alpn_protocols([IoT_protocol_name])
ssl_context.load_verify_locations(cafile=ca)
ssl_context.load_cert_chain(certfile=cert, keyfile=private)
return ssl_context
except Exception as e:
print("exception ssl_alpn()")
raise e
mqttc = mqtt.Client()
# Assign event callbacks
mqttc.on_message = on_message
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
ssl_context= ssl_alpn()
mqttc.tls_set_context(context=ssl_context)
logger.info("start connect")
mqttc.connect(aws_iot_endpoint, port=443)
logger.info("connect success")
mqttc.loop_start()
In Pi, I installed python 2.7.14 and paho-mqtt
But When I run python mqtt_apln.py, it shows error: ImportError: No module named paho.mqtt.client
Any suggestion is appreciated

Answer 1

I think there are two things going on here. First, a pip install paho-mqtt should make the package active for the current referenced python. For instance, under a 3.6.2 virtualenv should return:
$ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your
pip.conf under the [list] section) to disable this warning.
paho-mqtt (1.3.1)
pip (9.0.1)
setuptools (28.8.0)
How did you install the paho-mqtt package, via an apt package or directly with pip? Personally I virtualenv everything or include the package within the application directory via pip install package_name -t . to reference the current working directory.
From there it's working with the ALPN configuration. I reduced your code to just publish to the test topic on my end-point and used the AWS IoT console-->Test to subscribe to the test topic. For both python 2.7.12 and 3.6.2 I successfully received messages.
The main changes were to remove the callbacks, place a mqttc.publish followed by a time.sleep(3) to give the thread time to publish, then closed the connection.
Here is the code paired down for the publish only:
import sys
import ssl
import time
import datetime
import logging, traceback
import paho.mqtt.client as mqtt
MQTT_TOPIC = "topictest"
MQTT_MSG = "hello MQTT"
IoT_protocol_name = "x-amzn-mqtt-ca"
aws_iot_endpoint = "xxxxxx.iot.us-east-1.amazonaws.com"
ca = "ca.pem"
cert = "xxxx-certificate.pem.crt"
private = "xxxx-private.pem.key"
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(log_format)
logger.addHandler(handler)
def ssl_alpn():
try:
#debug print opnessl version
logger.info("open ssl version:{}".format(ssl.OPENSSL_VERSION))
ssl_context = ssl.create_default_context()
ssl_context.set_alpn_protocols([IoT_protocol_name])
ssl_context.load_verify_locations(cafile=ca)
ssl_context.load_cert_chain(certfile=cert, keyfile=private)
return ssl_context
except Exception as e:
print("exception ssl_alpn()")
raise e
mqttc = mqtt.Client()
ssl_context= ssl_alpn()
mqttc.tls_set_context(context=ssl_context)
logger.info("start connect")
mqttc.connect(aws_iot_endpoint, port=443)
logger.info("connect success")
mqttc.loop_start()
# After loop start publish and wait for message to be sent.
# Hard coded delay but would normally tie into event loop
# or on_publish() CB
# JSON payload because, pretty
mqttc.publish('test', '{"foo": "bar"}')
time.sleep(3)
mqttc.loop_stop()
Please let me know if that works for you? It's awesome that AWS now supports MQTT connections on port 443 without having to utilize websockets (and the need for SigV4 credentials).

Answer 2

I ran into the same issue regarding the use of paho-mqtt with aws IoT core.
https://aws.amazon.com/de/blogs/iot/how-to-implement-mqtt-with-tls-client-authentication-on-port-443-from-client-devices-python/
The tutorial uses no clientID. Depending on your security rules you have to deliver a client ID to be able to connect properly. Here an SDK-example ruleset where only the clients "sdk-java", "basicPubSub" and "sdk-nodejs-*" are allowed to connect.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Publish",
"iot:Receive"
],
"Resource": [
"arn:aws:iot:eu-central-1:036954049003:topic/sdk/test/java",
"arn:aws:iot:eu-central-1:036954049003:topic/sdk/test/Python",
"arn:aws:iot:eu-central-1:036954049003:topic/topic_1",
"arn:aws:iot:eu-central-1:036954049003:topic/topic_2"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Subscribe"
],
"Resource": [
"arn:aws:iot:eu-central-1:036954049003:topicfilter/sdk/test/java",
"arn:aws:iot:eu-central-1:036954049003:topicfilter/sdk/test/Python",
"arn:aws:iot:eu-central-1:036954049003:topicfilter/topic_1",
"arn:aws:iot:eu-central-1:036954049003:topicfilter/topic_2"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Connect"
],
"Resource": [
"arn:aws:iot:eu-central-1:036954049003:client/sdk-java",
"arn:aws:iot:eu-central-1:036954049003:client/basicPubSub",
"arn:aws:iot:eu-central-1:036954049003:client/sdk-nodejs-*"
]
}
]
}
To allow connection in case you have permits based on clientid change this line:
mqttc = mqtt.Client(client_id=MYCLIENTID)
MYCLIENTID is one of the three "sdk-java", "basicPubSub" or "sdk-nodejs-*" for this example.