MQTT on ESP8266 with NodeMCU - problems with publishing - lua

I'm building a battery powered IoT device based on ESP8266 with NodeMCU.
I use mqtt to periodically perform measurements and publish results.
I know, that to allow network stack running, I should avoid tight loops and rely on callback functions. Therefore it seemed to me that the right organization of my measurement code should be:
interval=60000000
function sleep_till_next_sample()
node.dsleep(interval)
end
function close_after_sending()
m:close()
sleep_till_next_sample()
end
function publish_meas()
m:publish("/test",result,1,0,close_after_sending)
print("published:"..result)
end
function measurement()
-- The omitted part of the function accesses
-- the hardware and places results
-- in the "result" variable
m = mqtt.Client("clientid", 120, "user", "password")
m:connect("172.19.1.254",1883,0, publish_meas)
end
The init.lua ensures, that the node has connected to the WiFi AP (if not, it retries up to 20 times, and if no connection is established, it puts the node on sleep until the next measurement time).
After WiFi connection is done, it calls the measurement function.
The interesting thing is, that the above code doesn't work. There are no errors displayed in the console, but the mqtt broker does not receive published messages.
To make it working, i had to add additional idle time, by adding timers in the callback functions.
The finally working code looks like below:
interval=60000000
function sleep_till_next_sample()
node.dsleep(interval)
end
function close_after_sending()
m:close()
tmr.alarm(1,500,0,function() sleep_till_next_sample() end)
end
function publish_meas()
m:publish("/test",result,1,0,function() tmr.alarm(1,500,0,close_after_sending) end)
print("published:"..result)
end
function measurement()
-- The omitted part of the function accesses
-- the hardware and places results
-- in the "result" variable
m = mqtt.Client("clientid", 120, "user", "password")
m:connect("172.19.1.254",1883,0, function() tmr.alarm(1,500,0, publish_meas) end)
end
The above works, but I'm not sure if it is optimal. To conserve the battery power I'd like to minimize the time before the node is put on sleep after the measurement is completed and results published.
Is there any better way to chain the necessary calls to m:connect, m:publish, m:close and finally node.dsleep so, that the results are correctly published in the minimal time?

Perhaps this was solved by more recent firmware. I am working through a problem that I thought might be somewhat explained by this issue, so tried to reproduce the problem as described.
My simplified test code is substantially similar; it calls dsleep() from the PUBACK callback of mqtt.Client.publish():
m = mqtt.Client("clientid", 120, "8266test", "password")
m:lwt("/lwt", "offline", 0, 0)
function main(client)
print("connected - at top of main")
m:publish("someval",12345,1,0, function(client)
rtctime.dsleep(SLEEP_USEC)
end)
end
m:on("connect", main)
m:on("offline", function(client) is_connected = false print ("offline") end)
m:connect(MQQT_SVR, 1883, 0, mainloop,
function(client, reason) print("failed reason: "..reason) end)
and when run, does successfully publish to my MQTT broker.
I am using:
NodeMCU custom build by frightanic.com
branch: master
commit: 81ec3665cb5fe68eb8596612485cc206b65659c9
SSL: false
modules: dht,file,gpio,http,mdns,mqtt,net,node,rtctime,sntp,tmr,uart,wifi
build built on: 2017-01-01 20:51
powered by Lua 5.1.4 on SDK 1.5.4.1(39cb9a32)

Related

Success callback function called on failure. Possible bug?

I'm working on a little project, using an ESP8266 with NodeMCU and Lua. I suspect I've found a bug, but since I'm new to Lua (and the other two too!), I'm hoping for some help in confirming if I'm right, or if I've missed something (more likely!).
The NodeMCU firmware contains a built-in SNTP client module which updates the synchronised time to a system clock (rtctime module). The success callback function seems to get called when (or possibly before) the NTP sync fails. This happens for example if the wifi is not connected, or sometimes on the first sync attempt after boot (with wifi connected). According to the doco, rtctime.get() returns zero if current time is not available; this is the result I get, further showing that the NTP sync hasn't been successful. I can't work out why the success function is being called at this point, in advance of, or instead of the failure function (as I would have expected).
The sntp module I'm referring to is here - unfortunately, the C source code was a bit over my head: https://nodemcu.readthedocs.io/en/master/en/modules/sntp/
My (minimal) code:
-- Define callback function for ntp sync success
function ntpSyncSuccess (sec, usec, server, info)
print('SNTP time sync successful!')
print("rtctime.get() returns: ", rtctime.get())
end
-- Configure and start NTP time sync with auto repeat enabled
sntp.sync("0.au.pool.ntp.org",
ntpSyncSuccess(sec, usec, server, info), --success callback
function() -- error callback
print('SNTP time sync failed!')
end,
1 -- enable autorepeat (SNTP sync every 1000 seconds (~17 min))
)
Serial output result when I boot the device and run the code (Note the 2nd and 3rd last lines):
NodeMCU custom build by frightanic.com
branch: master
commit: 11592951b90707cdcb6d751876170bf4da82850d
SSL: false
modules: cron,file,gpio,i2c,net,node,rotary,rtctime,sntp,struct,tmr,uart,wifi
build created on 2019-01-16 03:11
powered by Lua 5.1.4 on SDK 2.2.1(6ab97e9)
lua: cannot open init.lua
> print(uart.setup(0, 115200, 8, 0, 1, 1 ))
115200
> dofile("ntpTest.lua")
SNTP time sync successful!
rtctime.get() returns: 0 0 0
> SNTP time sync failed!
Just so that we can "close" this Q here (once you accept the answer). It has to be a function reference rather than a function invocation.
sntp.sync("0.au.pool.ntp.org",
ntpSyncSuccess, -- no (), no parameters
function() -- error callback

NodeMCU tmr.alarm always returns false

I have a problem with NodeMCU, trying to start a tmr.alarm on a esp8266.
tmr.alarm never starts the timer, and always returns false.
I've tried changing the timer, and even changing the type of the alarm (AUTO, SINGLE...) but i always have the same result.
Here's part of the code, i'm trying to comunicate with a DS18B20 (OneWire temperature sensor) once every 5 seconds or so.
if not tmr.alarm(1, 5000, tmr.ALARM_AUTO, function()
-- Comunication with the sensor
end)
then print("Comunication with DS18B20 couldn't be started.") end
The output is always
Comunication with DS18B20 couldn't be started.
Don't use the old 0.9.x NodeMCU binaries from https://github.com/nodemcu/nodemcu-firmware/releases they're no longer supported and contain lots of bugs. Build a custom firmware from the dev or master branch.
I'm not aware of any timer bugs in recent versions.

UART data error when using uart.alt(1)

I am trying to acquire rs232 data from a device connected to the ESP8266 (data will then be sent our via http/wifi).
I am using max3232 IC to provide the necessary 3.3v TTL to the ESP8266.
I have have connected the max3232 (pin 12) to GPIO pin 13 (rx) on the ESP8266 (I am only receiving data not sending data, so only the rx pin is connected).
The code i am using:
--
--file: test2.lua
--
tst2 = require "tst2"
tst2.start()
--tst2.lua (testing script)
local module = {}
function module.start()
print("in tst2.start")
uart.alt(1) --use alt GPIO pin 13 (Rx)
uart.setup(0, 9600,8, uart.PARITY_NONE, uart.STOPBITS_1,0)
uart.on("data",10,
function(data)
file.open("data.tmp", "w+")
file.writeline("starting")
for i=1,10 do
file.writeline(string.byte(string.sub(data,i,i)) )
end
file.writeline("from uart: ", data)
file.writeline("finished")
file.close()
end, 0)
uart.alt(0) --switch back to standard Rx/Tx pins
end
return module
The rs232 device connected to the ESP8266 is putting out a single alphabetic character every 3 seconds, however the data written to file (data.tmp) is as follows
starting
10
13
10
13
10
13
10
13
10
13
from uart:
finished
file.close()
Problems:
1- The rs232 device is not issuing any newln or cr characters, but these are appearing in the data file.
2- the string "file.close()" is being written to the data file, and looks like it is the actual lua command that follows the final file.writeline command.
3- the alphabetic data is not appearing in the data file.
4- switching back to the standard uart pins via uart.alt(0) does not work (the ESP8266 must be rebooted - this is not a major issue as the standard uart pins are only used during debugging).
I am writing the rs232 data to a file instead of simply printing it out on the screen (I am using ESPlorer v0.2.0) because the uart.alt(1) command redirects the serial port to the alternative ESP8266 gpio pins.
I think I am doing something fundamentally wrong with the uart set up, but i can't tell what it is.
SOLVED:
It appears that you can't connect the ESP8266 to both the serial port for debugging (e.g. the serial port on a pc running ESPlorer) and also have the alternate serial pins (ESP8266 GPIO 13 and 15) connected (to an external serial device) at the same time.
The nodemcu uart.alt() function does not appear to "turn off" the standard serial i/o pins.
Disconnecting the pc from the standard serial i/o pins solved the problem (debugging becomes an issue, but there are work-arounds to resolve this).
(updated) one workaround is to use a simple telnet server to interact with the lua interpreter. you can either connect the ESP8266 to your wifi router or, even better, set it up as an access point (AP) so that all you have to do is to connect your computer to it and then simply telnet in (to the gateway's IP). so, in addition to the telnet code, you'll need set up the AP in your init.lua. full code for the telnet server and the AP setup is below. A nice benefit is that I can program and monitor the ESP8266 from my phone using an off-the-shelf telnet app!
jj = [[
sock = 22 -- just a placeholder, so it stays global. may not be needed.
-- use sock:send("hello") to insert your own custom output to the client.
telnet_srv = net.createServer(net.TCP, 180)
telnet_srv:listen(2323, function(socket)
local fifo = {}
local fifo_drained = true
local function sender(c)
if #fifo > 0 then
c:send(table.remove(fifo, 1))
else
fifo_drained = true
end
end
local function s_output(str)
table.insert(fifo, str)
if socket ~= nil and fifo_drained then
fifo_drained = false
sender(socket)
end
end
sock = socket -- make the socket globally available.
node.output(s_output, 0) -- re-direct output to function s_ouput.
socket:on("receive", function(c, l)
node.input(l) -- works like pcall(loadstring(l)) but support multiple separate line
end)
socket:on("disconnection", function(c)
node.output(nil) -- un-regist the redirect output function, output goes to serial
end)
socket:on("sent", sender)
print("Welcome to NodeMCU world.")
end)
]]
file.open("telnet.lua", "w")
file.write(jj)
file.close()
jj = [[
wifi.setmode(wifi.STATIONAP);
wifi.ap.config({ssid="ESPtest",pwd=""});
print("Server IP Address:",wifi.ap.getip())
dofile("telnet.lua")
]]
file.open("init.lua","w")
file.write(jj)
file.close()
node.restart()
output:
Server IP Address: 192.168.4.1 255.255.255.0 192.168.4.1
>

MQTT / ESP8266 / NodeMCU / Lua code not publishing

I have a problem with the following Lua code on an ESP8266...
function sendData(humidity,temperature)
-- Setup MQTT client and events
print("sendData() entered")
print("Setting up mqtt.Client...")
m = mqtt.Client(mqtt_client_id, 120, username, password)
print("Attempting client connect...")
m:connect(mqtt_broker_ip , mqtt_broker_port, 0, function(conn)
print("Connected to MQTT")
print(" IP: " .. mqtt_broker_ip)
print(" Port: " .. mqtt_broker_port)
print(" Client ID: " .. mqtt_client_id)
print(" Username: " .. mqtt_username)
payload = "Temp: " .. temperature .. " Hmdy: " .. humidity
m:publish("pt/env",payload, 0, 0, function(conn)
print("Going to deep sleep for " .. (DSLEEPTIME/1000) .. " seconds")
node.dsleep(DSLEEPTIME*1000,4)
end)
end)
end
The code is being successfully called with the following...
-- Connect to network
wifi.setmode(wifi.STATION)
wifi.setphymode(wifi_signal_mode)
wifi.sta.config(wifi_SSID, wifi_password)
wifi.sta.connect()
print("Attempting to connect...")
ip = wifi.sta.getip()
if ip ~= nil then
print("Got IP: " .. ip)
print("About to call sendData()...")
sendData(humidity, temperature)
print("Returned from sendData()...")
end
Using ESPlorer I see the following...
Attempting to connect...
Attempting to connect...
Attempting to connect...
Attempting to connect...
Attempting to connect...
Attempting to connect...
Got IP: 192.168.0.39
About to call sendData()...
sendData() entered
Setting up mqtt.Client...
Attempting client connect...
Returned from sendData()...
So it basically enters sendData(...) and I see the output from the line...
print("Attempting client connect...")
...but I never see the logging in the m:connect(...) block such as...
print("Connected to MQTT")
...it seems it simply returns immediately.
The MQTT broker is a Raspberry Pi running Mosquitto and I've tested it with apps on my Android phone and tablet. I get successful publishing / subscription between phone and tablet in both directions.
I'm a Lua novice and only understand the basics of MQTT and I'm at a loss for what's wrong with the m:connect(...) block if anyone can help it would be appreciated.
UPDATE: PROBLEM SOLVED - Apologies for not getting back to this thread sooner. The problem was simply down to the version of Mosquitto I was running on my RPi (which complied to MQTT v3.1). The NodeMCU MQTT library supports MQTT v3.1.1 and is NOT backward compatible. In essence there wasn't very much wrong with my code although I did make some changes - it was simply down to MQTT versions being incompatible.
You didn't tell us what NodeMCU version you use. Warning: do NOT use any of the pre-built 0.9.x binaries available at https://github.com/nodemcu/nodemcu-firmware/releases. Build your own firmware as per http://nodemcu.readthedocs.io/en/dev/en/build/.
I always helps to strip a failing function and to make use of all available callback functions. I can confirm the following works on a nearly 2 months old firmware from the dev branch sending data to cloudmqtt.com:
function sendData(humidity, temperature)
print("Setting up mqtt.Client...")
m = mqtt.Client("SO-36667049", 120, "user", "password")
print("Attempting client connect...")
m:connect("m20.cloudmqtt.com", 12703, 0, 0,
function(conn)
print("Connected to MQTT")
payload = "Temp: " .. temperature .. " Hmdy: " .. humidity
m:publish("topic", payload, 0, 0,
function(client)
print("Message sent")
end)
end,
function(client, reason)
print("Connection failed, reason: " .. reason)
end)
end
Differences:
m:connect defines secure y/n and autoreconnect y/n explicitly. It always confuses me if only a subset of all optional parameters is set. Is your 0 in m:connect interpreted as secure or as autoreconnect? I don't know Lua well enough to tell why I code it explicitly.
Make use of the extra callback for function for failed connection attempts. See http://nodemcu.readthedocs.org/en/dev/en/modules/mqtt/#connection-failure-callback-reason-codes for failure reason codes.
Do NOT use the same name for variables in callback functions as used in "parent" functions. Note how you use m:connect(..., function(conn) and then inside that function you use m:publish(..., function(conn) again. You don't interact with the conn object in your code, so no harm done. However, this may bite you in other projects.
Your code is looking fine. If m:connect fails nothing will happen as you did not provide a callback function for failed connection attempts.
Also you don't check m:connect's return value for success.
Refer to
http://nodemcu.readthedocs.org/en/dev/en/modules/mqtt/#mqttclientconnect
And check if your connection attempt fails.

ESP8266 with NodeMCU firmware: receiving empty messages with secured MQTT subscription

I'm trying to securely connect ESP8266 to MQTT Azure Protocol Gateway broker on cloud or local computer (tried both) like this (not important - connection works correctly):
m = mqtt.Client("{deviceId}", "3600", "{iotHub}/{deviceId}", "{SASToken}")
...
m:on("message", function(conn, top, data)
print(data) -- EMPTY string here!
end)
...
m:connect({IP}, 8883, 1, function(conn)
m:subscribe("devices/{deviceId}/messages/devicebound/#", 1, function(conn)
...
end)
end)
ESP connects to server and handshake is completed successfully. When I publish some data, I can read it on server properly, it is OK. I subscribe to topic without problems. When I send data in cloud-to-device message from server to ESP to subscribed topic, 'on message' event is called but data attribute passed to the function is EMPTY string.
I'm using latest NodeMCU master build based on 1.4.0 SDK (tried both integer and float version). I can't turn on debugging, because i don't have NodeMCU developer yet.
I tried following:
dev version - not help
free memory up to 32kB - not help
captured packets with WireShark: packets contain encrypted data with some lenght, so it is not empty and packet size is less than 2kB buffer size
Can someone please advise me where could be a problem or how to debug it for more info? I would approciate any ideas. Thank you.
EDIT:
I've tried debug mode, and there is nothing interesting on output:
enter mqtt_socket_received.
MQTT_DATA: type: 3, qos: 0, msg_id: 0, pending_id: 0
enter deliver_publish.
string
userdata: 3fff3e88
devices/ESP/messages/devicebound
<- here should be printed data
On
leave deliver_publish.
receive, queue size: 0
leave mqtt_socket_received.
enter mqtt_socket_timer.
timer, queue size: 0
keep_alive_tick: 71
leave mqtt_socket_timer.
enter mqtt_socket_received.
MQTT_DATA: type: 7, qos: 1, msg_id: 8813, pending_id: 0
receive, queue size: 0
leave mqtt_socket_received.
This may be stupid but sometimes lua expects specific names in some functions.
Change this
function(conn, top, data)
for this
function(conn, topic, data)
I have worked previously with mqtt in lua with password protection, and i had to drop it beacuse i wasnt recieving the message content either, however i think mine was due to the amount of messages i was recieving/delivering. I also changed broker from mosquitto to mosca.
I hope you find a fix soon and please share it we might be able to use it to :)

Resources