ESP8266 UPnP Port Forwarding - IoT [closed] - iot

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
Is it possible to use the UPNP protocol for automatic port forwarding on the router using ESP8266?
I need to be able to access my ESP8266 module even when I am away from home.
Currently I have configured port forwarding manually in my router settings.
But in the future, in order for my project to become a commercial product, it needs to be able to do automatic port forwarding as this would be a barrier for the average user.
On the internet I found something talking about UPNP on ESP8266, but it was not about port forwarding.
Thank you very much in advance!

You can have a look at my library that I made just for that:
https://github.com/ofekp/TinyUPnP
I have an example for an IOT device (LED lights) within the package, I cannot attach the link due to low reputation.
You can have a look at the example code. All made for ESP8266.
Very simple to use, just call addPortMapping with the port you want to open, just as showed in the example.
You have to do this every 36000 (LEASE_DURATION) seconds, since UPnP is lease based protocol.
Declare:
unsigned long lastUpdateTime = 0;
TinyUPnP *tinyUPnP = new TinyUPnP(-1); // -1 means blocking, preferably, use a timeout value (ms)
Setup:
if (tinyUPnP->addPortMapping(WiFi.localIP(), LISTEN_PORT, RULE_PROTOCOL_TCP, LEASE_DURATION, FRIENDLY_NAME)) {
lastUpdateTime = millis();
}
Loop:
// update UPnP port mapping rule if needed
if ((millis() - lastUpdateTime) > (long) (0.8D * (double) (LEASE_DURATION * 1000.0))) {
Serial.print("UPnP rule is about to be revoked, renewing lease");
if (tinyUPnP->addPortMapping(WiFi.localIP(), LISTEN_PORT, RULE_PROTOCOL_TCP, LEASE_DURATION, FRIENDLY_NAME)) {
lastUpdateTime = millis();
}
}
I only checked it with my D-Link router.
To anyone interested in how the library works:
It sends an M_SEARCH message to UPnP UDP multicast address.
The gateway router will respond with a message including an HTTP header called Location.
Location is a link to an XML file containing the IGD (Internet Gateway Device) API in order to create the needed calls which will add the new port mapping to your gateway router.
One of the services that is depicted in the XML is <serviceType>urn:schemas-upnp-org:service:WANPPPConnection:1</serviceType> which is what the library is looking for.
That service will include a eventSubURL tag which is a link to your router's IGD API. (The base URL is also depicted in the same file under the tag URLBase)
Using the base URL and the WANPPPConnection link you can issue an HTTP query to the router that will add the UPnP rule.
As a side note, the service depicted in the XML also includes a SCPDURL tag which is a link to another XML that depicts commands available for the service and their parameters. The package skips this stage as I assumed the query will be similar for many routers, this may very well not be the case, though, so it is up to you to check.
From this stage the package will issue the service command using an HTTP query to the router. The actual query can be seen in the code quite clearly but for anyone interested:
Headers:
"POST " + <link to service command from XML> + " HTTP/1.1"
"Content-Type: text/xml; charset=\"utf-8\""
"SOAPAction: \"urn:schemas-upnp-org:service:WANPPPConnection:1#AddPortMapping\""
"Content-Length: " + body.length()
Body:
"<?xml version=\"1.0\"?>\r\n"
"<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\" s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">\r\n"
"<s:Body>\r\n"
"<u:AddPortMapping xmlns:u=\"urn:schemas-upnp-org:service:WANPPPConnection:1\">\r\n"
" <NewRemoteHost></NewRemoteHost>\r\n"
" <NewExternalPort>" + String(rulePort) + "</NewExternalPort>\r\n"
" <NewProtocol>" + ruleProtocol + "</NewProtocol>\r\n"
" <NewInternalPort>" + String(rulePort) + "</NewInternalPort>\r\n"
" <NewInternalClient>" + ipAddressToString(ruleIP) + "</NewInternalClient>\r\n"
" <NewEnabled>1</NewEnabled>\r\n"
" <NewPortMappingDescription>" + ruleFriendlyName + "</NewPortMappingDescription>\r\n"
" <NewLeaseDuration>" + String(ruleLeaseDuration) + "</NewLeaseDuration>\r\n"
"</u:AddPortMapping>\r\n"
"</s:Body>\r\n"
"</s:Envelope>\r\n";
I hope this helps.

I don't see why not. UPnP implements multiple profiles, the one you are interested in is named IGD (Internet Gateway Device), which most home routers implement to allow client applications on the local network (e.g Skype, uTorrent, etc.) to map ports on the router's NAT.
UPnP works over IP multicast to discover and announce devices implementing UPnP services over the address 239.255.255.250. Devices interested in such announcements subscribe to this multicast group and listen on port 1900. In fact, UPnP does not itself provide a discovery mechanism, but relies on a protocol called SSDP (Simple Service Discovery Protocol) to discover hosts on the local network.
All that's needed is an UDP socket bound to the aforementioned address and port to subscribe and publish messages on your home multicast group. You'd need to use an implementation of SSDP to discover your router, once you have discovered your router, you can send commands using UPnP wrapped around SOAP enveloppes.
There are many implementations of the UPnP IGD profile in Posix C, which you may reuse and port to the ESP 8266 (e.g MiniUPnP, gupnp-igd).

Related

Send data from ESP32 to a server via WiFi

I am not new here but this is my first question.
I have searched a lot and quite frankly can't understand how this is supposed to work.
I get data periodically (temperature) to my ESP32 and while having it set as a WiFi client, connect to my router and somehow store this data on my Laptop(or somewhere else, like a local/web site, don't know if that's possible/better).
How is the connection supposed to work? I have installed XAMPP and run the Apache and MySQL servers and I tried to connect to my Laptop with some sketches from Arduino using the ESP32 libraries
// Use WiFiClient class to create TCP connections
WiFiClient client;
const int httpPort = 80;
const char* host = "192.168.1.109"; //The local IP of my Laptop
if (!client.connect(host, httpPort)) {
Serial.println("connection failed");
return;
}
but it doesn't connect.
Can someone please explain to me how this connection is supposed to take form or is this question too vague? I really just wanna know the "how-things-should-work-together" in this situation.
Thank you in advance.
OK, so after a lot of research and trying, I managed to work it out. I can now send an HTTP request (like GET or POST) from my ESP32 to a local server that is running on my laptop using XAMP and get a response. I can also connect to my local IP from my mobile phone (which is also in the same WiFi network).
Just for anyone else who wants to connect to a location in a server hosted on a PC in a local network, the steps are:
Create a local server on your PC, laptop whatever using an application like XAMPP (I have Windows 10 so WAMP would also work), download, install, open and start Apache.
Make sure that the Firewall lets your requests pass through (for me it was open by default, but I had seen elsewhere Firewall being an issue)
Go to your network settings, select the network that your devices(ESP32, phone, etc.)are connected and change its profile to Private, meaning that you trust this network, making your PC discoverable and able to accept requests. (That is really simple but took me hours to find)
Now, in order to connect from your phone to your PC, open a browser and enter the local IP (that is the IP that is given to your PC from the router as a local network name) of your PC to a browser and that's it, you are connected.
If you installed and ran XAMP, when connecting to your local IP(from same PC or other local device), it will forward you to 192.168.x.x/dashboard. If you want to create new workspaces and files, browse the XAMP folder in the installed location and inside the '/htdocs' subfolder do your testing.
For the ESP32 communication in Arduino(basic steps, not full code):
#include <WiFi.h>
#include <HTTPClient.h>
String host = "http://192.168.x.x/testfolder/";
String file_to_access = "test_post.php";
String URL = host + file_to_access;
void setup(){
WiFi.begin(ssid, password); //Connect to WiFi
HTTPClient http;
bool http_begin = http.begin(URL);
String message_name = "message_sent";
String message_value = "This is the value of a message sent by the ESP32 to local server
via HTTP POST request";
String payload_request = message_name + "=" + message_value; //Combine the name and value
http.addHeader("Content-Type", "application/x-www-form-urlencoded");
int httpResponseCode = http.sendRequest("POST", payload_request);
String payload_response = http.getString();
}
In the test_post.php (located in "C:\xampp\htdocs\testfolder\") file I used a simple script to echo a message received using a POST request, so it's only 'readable' from POST requests. Connecting to it from your browser will give you the "Sorry, accepting..." message.
<?php
$message_received = "";
if ($_SERVER["REQUEST_METHOD"] == "POST"){
$message_received = $_POST["message_sent"];
echo "Welcome ESP32, the message you sent me is: " . $message_received;
}
else {
echo "Sorry, accepting only POST requests...";
}
?>
Finally, using Serial prints, the output is:
Response Code: 200
Payload: Welcome ESP32, the message you sent me is: This is the value of a message sent by the ESP32 to local server via HTTP POST request
There it is, hope that this helps someone.

ESP8266, NodeMCU, soft AP - UDP server-like soft AP, independent access point

I am using NodeMCU (with ESP8266-E) with an upgraded firmware. All basic commands work perfectly but there is one problem.
I wanted to create an independent access point, which could have a behaviour like a UDP server. That means without direct connection to any other access points. A simple UDP server like soft AP.
I followed these steps:
I have uploaded a new firmware to NodeMCU.
I have downloaded ESPlorer for better work with NodeMCU.
I have uploaded the source code below.
I have connected to the NodeMCU access point on my desktop.
I have sent some strings to the NodeMCU using a Java UDP client program.
I have looked at the messages on ESPlorer.
NodeMCU has not received any such strings.
--
print("ESP8266 Server")
wifi.setmode(wifi.STATIONAP);
wifi.ap.config({ssid="test",pwd="12345678"});
print("Server IP Address:",wifi.ap.getip())
-- 30s timeout for an inactive client
srv = net.createServer(net.UDP, 30)
-- server listens on 5000, if data received, print data to console
srv:listen(5000, function(sk)
sk:on("receive", function(sck, data)
print("received: " .. data)
end)
sk:on("connection", function(s)
print("connection established")
end)
end)
When I tried to send a message using a Java application, there was no change in ESPlorer. Not even when I tried to send a message using the Hercules program (great program for TCP, UDP communication).
I guess that maybe it will be the wrong IP address. I am using the IP address of the AP and not the IP address of the station.
In other words I am using this address: wifi.ap.getip() and not this address wifi.sta.getip() for connections to the UDP server. But sta.getip() returns a nil object. Really I don't know.
I will be glad for any advice.
Thank you very much.
Ok, let's restart this since you updated the question. I should have switched on my brain before I gave you the first hints, sorry about this.
UDP is connectionless and, therefore, there's of course no s:on("connection"). As a consequence you can't register your callbacks on a socket but on the server itself. It is in the documentation but it's easy to miss.
This should get you going:
wifi.setmode(wifi.STATIONAP)
wifi.ap.config({ ssid = "test", pwd = "12345678" })
print("Server IP Address:", wifi.ap.getip())
srv = net.createServer(net.UDP)
srv:listen(5000)
srv:on("receive", function(s, data)
print("received: " .. data)
s:send("echo: " .. data)
end)
I ran this against a firmware from the dev branch and tested from the command line like so
$ echo "foo" | nc -w1 -u 192.168.4.1 5000
echo: foo
ESPlorer then also correctly printed "received: foo".
This line is invalid Lua code. connected is in the wrong place here. you can't just put a single word after a function call.
print(wifi.ap.getip()) connected
I guess you intended to do something like
print(wifi.ap.getip() .. " connected")
Although I think you should add som error handling here in case wifi.ap.getip() does not return an IP.
Here you do not finish the function definition. Neither did you complete the srv:on call
srv:on("receive", function(srv, pl)
print("Strings received")
srv:listen(port)
I assume you just did not copy/paste the complete code.

Using agent name instead of IP as Address in SNMP4J

I'm trying to make a program that sends SNMP queries to some switches in the network.
Using the Net-snmp tools, I can send get requests to the switch using its name, and it works fine. But SNMP4J requires an IP address in CommunityTarget, so I get an IllegalArgumentException.
This is the relevant part of the code:
TransportMapping transport = new DefaultUdpTransportMapping();
transport.listen();
CommunityTarget comtarget = new CommunityTarget();
comtarget.setCommunity(new OctetString("public"));
comtarget.setVersion(SnmpConstants.version1);
comtarget.setAddress(new UdpAddress("switchName")); // exception happens here
comtarget.setRetries(2);
comtarget.setTimeout(1000);
How can I work around this?
You can get the IP address by using DNS resolution, like this answer says:
InetAddress address = InetAddress.getByName(switchName);
System.out.println(address.getHostAddress());

MBeanServerConnection.invoke hangs forever

We have an app that invokes various remote methods on MBeans using MBeanServerConnection.invoke.
Occasionally one of these methods hangs.
Is there any way to have a timeout on the call? so that it will return with an exception if the call takes too long?
Or do I have to move all those calls into separate threads so they don't lock up the UI and require killing the app?
See http://weblogs.java.net/blog/emcmanus/archive/2007/05/making_a_jmx_co.html
===== Update =====
I was thinking about this stuff when I first responded, but I was on my mobile and I can't type worth a damn on it.....
This is really an RMI problem, and unless you use a different protocol, there's not much you can do, except, as you say, move all those calls into separate threads so they don't lock up the UI.
But.... if you have the option of fiddling with the target server and you can customize the connecting client, you have at least 1 option which is to customize the JMXConnectorServer on your target servers.
The standard JMXConnectorServer implementation is the RMIConnectorServer. Part of it's specification is that when you create a new instance using any of the constructors (like RMIConnectorServer(JMXServiceURL url, Map environment)), the environment map can contain a key/value pair where the key is RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE and the value is a RMIClientSocketFactory. Therefore, you can specify a socket factory method like this:
RMIClientSocketFactory clientSocketFatory = new RMIClientSocketFactory() {
public Socket createSocket(String host, int port) {
Socket s = new Socket(host, port);
s.setSoTimeout(3000);
}
};
This factory creates a Socket and then sets its SO_TIMEOUT using setSoTimeout, so when the client connects using this socket, all operations, including connecting, will timeout after 3000 ms.
You could also checkout the JMXMP connector and server in the jmx-optional package of the OpenDMK. (links are to my github mavenized). No built in solution, mind you, but they're super easy to extend and JMXMP is simple TCP socket based rather than RMI, so this type of customization would be trivial.
Cheers.
# Nicholas : The above code is not working.I mean request is not getting timeout after 3000. ms.
map.put(RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE , new RMIClientSocketFactory() {
#Override
public Socket createSocket(String host, int port) throws IOException {
if(logger.isInfoEnabled() ){
logger.info("JMXManager inside createSocket..." + host + ": port :" + port);
}
Socket s = new Socket(host, port);
s.setSoTimeout(3000);
return s;
}
});
cs = JMXConnectorServerFactory.newJMXConnectorServer(url,map,mbeanServer);
As I answered on: How to set request timeout for JMX Connector the RMI properties can help you. All the properties are on Oracle documentation site:
http://docs.oracle.com/javase/7/docs/technotes/guides/rmi/sunrmiproperties.html.
For example: -Dsun.rmi.transport.tcp.responseTimeout=60000 is a client side tcp response timeout. There are also properties for connect timeout and for server side connections.
I also am not happy how the JMX/RMI/TCP stack hides important settings from lower level protocols, and makes it not available for a single connection.

How to decide on port number between client and server communication on internet

I have a client application which runs as a Java applet from a user's browser and connects to a server via a given port. The server is running on a publicly accessible cloud. Based on my previous experience of writing socket code , I can decide upon a random port number (say 5999) and use it for client server communication. However in this case the client can be any user machine and there can be many users accessing the server.
So the question is how to ensure that I use a port number which is least likely to be used by any other service on the client's computer.
I have also explored webservices based protocols for this purpose but I didnt use it for the reason that my requirement is really simple and it can be fulfilled with a simple socket communication and a custom protocol. I feel webservices tools and stuff like SOAP , CORBA are too heavy weight.
Choose one that is not in the Service Name and Transport Protocol Port Number Registry and hope for the best.
Also, a client can connect to many servers on the same port. When the clients connect, they will use a random port on there end.
Only the server needs to worry about using a free port, and the clients need to know what this port is else they will not be able to connect to your server.
You don't have to choose a portnumber on the users PC. Just the server port has to be one specific number.
When the client connects to the servers IP and port number, the operating system chooses a random free port for the client itself.
There are plenty of them as #thecoshman mentioned, and I compiled some of them for quick reference :)
258, 285, 325-332, 334-343, 703, 708, 717-728, 732-740, 743, 745-746, 755-757, 766, 768, 778-779, 781-785, 787, 788-799, 803-809, 811-827, 834-846, 849-852, 855-859, 863-872, 874-885, 889-899, 904-909, 914-952, 954-988, 1002-1007, 1009, 1491, 2194-2196, 2259, 2369, 2378, 2794, 2873, 3092, 3126, 3301, 3546, 3694, 3994, 4048, 4144, 4194-4196, 4198, 4315, 4317-4319, 4332, 4337-4339, 4363-4365, 4367, 4380-4388, 4397-4399, 4424, 4434-4440, 4459-4483, 4489-99, 4501, 4503-4533, 4539-4544, 4560-4562, 4564-4565, 4571-4572, 4574-4589, 4606-4620, 4622-4657, 4693-4699, 4705-4710, 4712-4724, 4734-4736, 4748, 4757-4773, 4775-4783, 4792-4799, 4805-4826, 4828-4836, 4852-4866, 4872-4875, 4886-4893, 4895-4898, 4903-4911, 4916-4935, 4938-4939, 4943-4948, 4954-4968, 4972-4979, 4981-4983, 4992-4998

Resources