I am trying to communicate with my hosted server using an Arduino Ethernet shield.
Now the problem is this:
Using a web browser, I can see that calling the URL works fine, but calling the IP address does not. I assume that the hosting provider hosts several different URLs using the same IP address.
How can I make a URL dependent call from within the Arduino libraries? The standard Arduino libraries only require the IP address of my host, not the URL.
You need to ensure the http GET/POST request has the correct Host field.
Here's a sample get request from http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
There's a lot of good technical information on this site.
Sample HTTP Get Request:
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.w3.org
This may require some customization of your code, but should do the trick !!
other options:
https://github.com/interactive-matter/HTTPClient or
https://github.com/amcewen/HttpClient
Related
Has anybody succedeed in creating a web server at home using a huawei hg633 router ?
I started by using the no-ip service and didn't get very far, as stated in this document this router is not compatible with no-ip.
So I tried using duckdns, following the instructions in the same document but that didn't work either. I also read in this relatively old post that dynamic dns is broken on those routers. In fact when I save my ddns settings, the password looks truncated.
Any port forwarding I set on my router doesn't seem to work, but that's probably related with the problems above (?)
Anything else I could try here ?
Thanks a lot !
For the record, my web server works, I can access it from within my network.
I know in airports, for example, I've connected to their AP, and it pops up a browser window to log in on my device. Is it possible to do so with NodeMCU in lua, or even with c firmware?
This can accomplished by setting the DNS server for a connecting client [via DHCP] to a sort of DNS proxy. It doesn't need to be a fully featured DNS server, it only needs to be able to either return a static DNS answer for any host name query or forward the request to a real DNS server, to resolve host names as usual.
The static answer effectively hijacks web requests at the DNS level, by forging the DNS answer, causing all host names to resolve to the IP address of a local web server. That local web server ignores any Uri details and serves a login prompt for every request. It must also maintain a list of client MAC addresses that have authenticated.
NodeMCU does have a built-in DHCP server, as part of it's built-in WiFi AP, but running both a web and a DNS proxy in ESP8266's limited memory would be a hell of a trick. I think that two of them working cooperatively, interfaced using the SPI bus might be workable... maybe even three of them, one dedicated to maintaining the list of authenticated MACs, expiring them, etc.
Note that the only part of this I have done on an ESP 8266 is some very simple web server functionality, so it's mostly theory. If you try it I'd be very interested in hearing about it. :-)
You might want to try out CaptiveIntraweb project (https://github.com/reischle/CaptiveIntraweb) which is based on NodeMCU.
There is also thread (http://www.esp8266.com/viewtopic.php?f=32&t=3618) on ESP8266 community forum that discusses the solution details.
I have a python script which triggers a http POST request using standard libraries. I'm trying to view and debug the requests content by capturing in wireshark.
The request is sucesfull because I recieve the correct server response.
In wireshark I listen on all available interfaces but cannot find the request.
I have also tried some filtering options like http filter but the request was still not found.
I have Ubuntu installed and the server host is not localhost
How can I find the request?
You are unable to capture HTTP POST request because you are using Wireshark http filters. Do not filter traffic with HTTP filters, instead of that use website/URL IP address.
I have just tried to filter the traffic with the help of Wireshark and found that I can filter it with the help of IP address.
Refer below link,
https://www.w3schools.com/python/ref_requests_post.asp
Use Filter : "ip.addr==192.229.133.221"
I can filter the packets, but its TCP encrypted traffic and we cannot get additional information without decryption.
Refer below answer, it will help you to find the IP address of your URL.
How to use filters in the wireshark?
If you are not comfortable with it, use “Microsoft Network Monitor” tool. It shows applications name along with IP address and ports.
Once you know the IP address and ports used, you can filter the traffic with the help of Wireshark easily.
Note:- I will suggest instead of "Wireshark" or "Microsoft Network Monitor”, please use fiddler. Refer below link,
How to capture Visual Studio Code traffic through Fiddler?
So here is my issue, I have a website hosted from a virtual machine on my server and am using a dyndns service to point a url to my IP. My ISP recently set up a new modem which unfortunately has its own built in gateway and router. After fighting it to forward port 80 I tested it by trying to navigate to the site via the URL and it didn't work, then I tested it on my phone connected to cell data network and it worked! I am able to visit the site via the URL as long as I am not connected to my network. i find this very weird and cannot figure out why.
I am able to view the site on my network by typing in the local IP of the server.
Any suggestions why this might be occurring?
Yes, this is a pain. Usually your modem won't route traffic from inside that's destined for its public IP address.
When you come from outside, the traffic hits the modem from the external line, and the port forwarding rules get applied, and the traffic reaches your web server. But those port forwarding rules don't get applied to internal traffic. You're trying to browse the web server on the modem, rather than on your server.
I did once find a modem that allowed forwarding of internal traffic, but that was a long time ago, and I haven't see one like it since. What I do these days is to use the internal address when I'm on the internal network, and the external address when I'm not. For things that get scripted, I have a little function that determines whether I'm on my local network or not, and programmatically chooses the right way to address the server.
This is because your router does not support hairpinning (or does not have it set up).
From Cisco Support Community:-
The term hairpinning comes from the fact that the traffic comes from one source into a router or similar devices, makes a U-turn and goes back the same way it came.
Visualize this and you see something that looks like a hairpin.
Hairpin NAT is a useful technique for accessing an internal server using a public IP. Since you are using a public IP to attempt to access a server in your network, the traffic will attempt to go out to the internet. In order to reach the server, the traffic will need to be redirected to the correct location.
The problem is how you are doing your internal routing DNS.
You can do DNS Lookup and trace route to see where the Website name is not resolving and whether if you ping the domain e.g. ping something.com return the public IP.
I resolved ours by doing policy routing on website FQDN to go through a different WAN. It's working fine. This works for those with different WAN terminating at the site.
The other way is redo the DNS configuration in internal network.
I'm using the Play Framework which uses http://localhost:9000 by default. I'm also trying something with Twitter and it needs to use a callback url for authentication, but Twitter won't accept http://localhost:9000 as a callback URL.
How can I configure my localhost to map to something like http://mylocal.loc, (similar as with an Apache vhost), instead of http://localhost:9000?
The problem is that the URL needed to be entered in the following format:
http://127.0.0.1:9000/twitter-callback
The above works perfectly as a Twitter callback address.
Twitter isn't trying to access localhost directly, it simply takes the above address as far as I understand, sticks it into the HTTP response header, prompting whichever browser being used to perform a straight forward 302 redirect.
The following blog post had some invaluable information in regards to this question:
http://www.tonyamoyal.com/2009/08/17/how-to-quickly-set-up-a-test-for-twitter-oauth-authentication-from-your-local-machine/
The reason that twitter can't use localhost as a callback url is because localhost is a redirect to your computers loopback interface. In other words, localhost is always the computer that you're on. In order for other computers (including twitter) to access your host, you need to use an external IP address, or a hostname.
To get your IP address, visit whatsmyip. This will tell you your external IP address (which other computers on the internet can access). If you have a static IP address, you can purchase a domain name, or get a free one from something like no-ip or dyndns to make it easier to remember and type. You'll need to point a DNS record from that domain to your IP. You'll also probably need to do some port forwarding and stuff to get it to go to your computer on port 9000, rather than your router (dependent on your network setup).
Possibly an easier option would be to obtain a free hosting/domain service whilst you're testing.
EDIT: josef's problem was not related to the absence of internet access to his local server, see his own answer for what was going on and a solution. This answer handles the case where a local server needs to be visible from the internet.
localhost, aka 127.0.0.1 is the name that on each computer points to the computer itself. So Twitter looks at itself, obviously doesn't see the service, end of story.
If your computer is connected to a local network, most likely that network is NATed and using private addresses like 192.168.x.x, 10.x.x.x or 172.16x.x.x. These addresses are private (not known outside of the local network because not routed on the internet), so that doesn't help you either.
What remains is your public IP address, ie the address your router gets from your ISP. Via DNS you can map that address to a name, a free service that allows you to map a fixed name also to a variable address is DynDNS.
But wait, there is more! Your router protects your network by not allowing traffic originating OUTSIDE the private network IN, unless you define some forwarding rule in the router, in your case a rule that forwards incoming tcp traffic on port 9000 to your machine's port 9000.
Once all that has been taken care of, your computer will be accessible from the outside, and your callback should work.
Edit your hosts file and add the following line:
127.0.0.1 mylocal.loc
For Windows, it is located in C:\Windows\System32\drivers\etc\. On *nix, you can find it in /etc.