More of a theoretical question. Yes, I know those types of questions are frowned upon here. But suppose APIs like MaxMind GeoIP's didn't exist, and you wanted to build a service that translated IP addresses to the city it's in. What would you do? How would you go about solving that problem?
Use the whois database to find out who the IP is allocated to.
Use DNS to do a reverse lookup on the IP address.
Use traceroute to guess the location based on the routers leading up to the host
As an example, lets look at stackoverflow.com (64.34.119.12)
$ whois 64.34.119.12 | egrep "(OrgName)|(City)|(State)|(Country)"
OrgName: Peer 1 Network Inc.
City: New York
StateProv: NY
Country: US
whois shows us that they're probably in NY, NY.
$ host 64.34.119.12
12.119.34.64.in-addr.arpa domain name pointer stackoverflow.com.
Reverse DNS (PTR record) doesn't help here. However, a lot of ISPs do have region specific reverse DNS entries that can help with this process (especially for dynamically assigned hosts). As an example:
$ host 24.25.195.1
1.195.25.24.in-addr.arpa domain name pointer sndgca-dns-cac-01.san.rr.com.
This host is obviously identified and located by it's PTR record: a Time Warner (Road Runner) caching DNS server in San Diego, CA.
Now, back to the stackoverflow.com locating process:
$ traceroute stackoverflow.com
traceroute to stackoverflow.com (64.34.119.12), 64 hops max, 40 byte packets
ms
(... removed the first few hops ...)
8 ae-6-6.ebr1.Chicago1.Level3.net (4.69.140.189) 30.9 ms 28.855 ms 25.183 ms
9 ae-2-2.ebr2.NewYork2.Level3.net (4.69.132.66) 54.373 ms 54.197 ms 48.171 ms
10 ae-6-6.ebr2.NewYork1.Level3.net (4.69.141.21) 48.250 ms 50.712 ms 51.293 ms
11 ae-62-62.csw1.NewYork1.Level3.net (4.69.148.34) 59.493 ms ae-82-82.csw3.NewYork1.Level3.net (4.69.148.42) 51.694 ms ae-62-62.csw1.NewYork1.Level3.net (4.69.148.34) 58.315 ms
12 ae-3-89.edge1.NewYork1.Level3.net (4.68.16.142) 51.529 ms ae-2-79.edge1.NewYork1.Level3.net (4.68.16.78) 50.77 ms ae-3-89.edge1.NewYork1.Level3.net (4.68.16.142) 50.567 ms
13 10ge-te-1-1.nyc-75bre-dis-2.peer1.net (216.187.115.77) 50.776 ms 51.193 ms 56.166 ms
14 gig2-0.nyc-gsr-b.peer1.net (216.187.123.5) 54.994 ms 58.813 ms 51.794 ms
15 gwny01.stackoverflow.com (64.34.41.58) 54.121 ms 52.65 ms 54.166 ms
16 * * *
(... remaining time outs removed ...)
The traceroute output confirms our suspicion that stackoverflow.com is hosted in NY, NY.
That's one way to tackle geolocation without an API.
Im really guessing here but this may work.
IP ranges are assigned by IANA to regional providers. There are five of them. In turn each regional provider assign IP ranges to end users and internet service provides.
If you can get/buy the information from the regional providers and their ISP clients you are in business.
Related
I trying to use mosquitto broker for an IoT application. I have a embedded hardware, actually not much documented/exampled on the internet. I've succesfully implemented an TCP client on this hardware, and now i can send/listen messages throught any port i want via TCP and i listen via netcat. But when i tried to connect mosquitto, it doesn't accept the literal language. I digged on the internet. The broker take messages like shown below, but even this one not working.
I can not found any documentation. I even tried to watch Wireshark packages, and i can not find any pattern. Any help will appreciated.
$ echo -en "\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a" |nc localhost 1883|hd
00000000 20 02 01 00 | ...|
00000004
Had a similar usecase like you and this is how I managed to decode this message and create my own connect request.
echo -en "\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a" |nc localhost 1883
The above message is broken down as follows:
nc localhost 1883, opens a tcp socket to the mqtt port 1883 (to the broker (on localhost) listening to port 1883)
\x10\x0d\x00\x04MQTT\x04\x00\x00\x00\x00\x01a is the connect packet sent to the socket that was opened. This connect packet can be broken down as follows:
\x10: MQTT connect packet (Constitutes the control field with the 1st 4 bits representing the command type "0001" and the 2nd 4 bits the control flag.
\x0d: Remaining length: is the total length of both the variable header and the payload. This needs to be set after the whole payload is complete.
*Variable header = Protocol Name + Protocol level + Connect Flag byte + Keep Alive
Payload = ClientId, username, password, e.t.c, each entry is provided in the format lengthOfEntry + Entry e.g for a clientId and username and password, this gives: lengthOfClientId + ClientId + lengthOfUsername + Username + lengthOfPassword + Password. *Note: the length is always provided as two bytes.
\x00\x04MQTT: Represents the protocol name "MQTT". The first two bytes 0x00 and 0x04 are the protocol length and MQTT is then the protocol. These give a total of 6 bytes.
\x04: is the protocol level. From MQTT specification, the value of the Protocol Level field for the version 3.1.1 of the protocol is 4 (0x04)
\0x00: is the connect flag byte. each byte is represented as follows:
UsernameFlag|PasswordFlag|Will Retain|Will|QoS|Will Flag|CleanSession|Reserved. 0x00 means none of the flags are set and persistent sessions will be used. Hence the payload doesn't require a username or password as will be seen in the payload bytes.
\0x00\0x00: two bytes that represent the keepAlive time. In this case 0 is provided which means that the server is not required to disconnect the client on grounds of inactivity. keepalive mechanism is turned off
The next bytes are already the payload. Since the connect flag was set such that no username or password is required, then we only have to provide the client id. In the above example the clientid is "a". This is of length 1 byte.
\0x00\0x01: represents the length of the clientId. since we have the character "a" as the clientId we have just the length 1. The length is always given in 16bit (2 bytes).
a: the clientId.
If we count all the bytes, we come up to a total of 13 bytes which gives us our remaining le0gnth as 0x0d.
If you need to add a username and password, set the connect flag accordingly 0b11000000 = 0xC0. The username and password are added to the payload right after the clientId in the order lengthofClientId, clientId, lengthOfUSername, Username, LengthofPassword, Password. The reminaining length needs to be adjusted to reflect this.
Tip: The variable header has a fixed number of bytes of 10. The payload length always depends on the data provided. the length of each element is always 2 bytes. so for the above case, we have a clientid of 1 byte and the length which take 2 bytes to give a total of 10 + 2 + 1 = 13.
I hope this helps you out and answers your question.
Useful links:
https://openlabpro.com/guide/mqtt-packet-format/
http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718024
I am running a few micro service instances that are functioning as edge routers and have the #EnableZuulProxy annotation. I have written a number of filters and these control the flow of requests into the system.
What I would like to do is get the circuit stats from what is going on under the covers. I see that there is a underlying netflix class DynamicServerListLoadBalancer that has some of the sts I would like to see. Is it possible to get an instance of it and at specific time get the stats form it>
I can see it has stuff like this: (I formatted a log statement that I saw in my logs)
c.n.l.DynamicServerListLoadBalancer : DynamicServerListLoadBalancer for client authserver initialized:
DynamicServerListLoadBalancer:{
NFLoadBalancer:
name=authserver,current
list of Servers=[127.0.0.1:9999],
Load balancer stats=
Zone stats: {
defaultzone=[
Zone:defaultzone;
Instance count:1;
Active connections count: 0;
Circuit breaker tripped count: 0;
Active connections per server: 0.0;]
},
Server stats:
[[
Server:127.0.0.1:9999;
Zone:defaultZone;
Total Requests:0;
Successive connection failure:0;
Total blackout seconds:0;
Last connection made:Wed Dec 31 19:00:00 EST 1969;
First connection made: Wed Dec 31 19:00:00 EST 1969;
Active Connections:0;
total failure count in last (1000) msecs:0;
average resp time:0.0; 9
0 percentile resp time:0.0;
95 percentile resp time:0.0;
min resp time:0.0;
max resp time:0.0;
stddev resp time:0.0
]]
}
ServerList:org.springframework.cloud.netflix.ribbon.eureka.DomainExtractingServerList#5b1b78aa
All of this would be valuable to get and act on. Mostly the acting would be to feed usage heuristics back to the system.
Ok, like most of these things, I figured it out myself.
So here you go.
HystrixCommandKey hystrixCommandKey = HystrixCommandKey.Factory.asKey("what you are looking for");
HystrixCommandMetrics hystrixCommandMetrics = HystrixCommandMetrics.getInstance(hystrixCommandKey);
HystrixCommandProperties properties = hystrixCommandMetrics.getProperties();
long maxConnections = properties.executionIsolationSemaphoreMaxConcurrentRequests().get().longValue();
boolean circuitOpen = properties.circuitBreakerForceOpen().get().booleanValue();
int currentConnections = hystrixCommandMetrics.getCurrentConcurrentExecutionCount();
So in this example, "what you are looking for" is the hysteria command that you are looking for.
this gets you the properties of the particular hysteria thing you are looking for.
Form this you pull out the max connections, the current connections and whether the circuit was open.
So there you are.
I am new to zabbix. I would like to monitor the ping from my server and I want to activate a trigger if the ping gets unresponsive or ping time exceeds 20 milliseconds.
I don't know how to configure the trigger expression to suit my needs. Please help. Thanks.
I used
type -> Simple check
key -> icmppingsec
Type of information -> Numeric(Float)
Units -> s
Flexible intervial -> 10secs, from 7:00-24:00
This is the trigger expression.
And a graph I created.
According to simple check documentation, icmppingsec item returns ping time in seconds or 0 if the host is not reachable. Therefore, your trigger can be as follows:
{Template ICMP Ping:icmppingsec.avg(5m)} > 0.020 |
{Template ICMP Ping:icmppingsec.max(5m)} = 0
If you are using at least Zabbix 2.4, you should use or instead of | (see What's new in Zabbix 2.4.0).
Note also that there is no point in using "1-7,00:00-24:00" flexible interval. You can just put "10" into "Updated interval (in sec)" field.
When I run httperf with following options, the output is easy to understand.
Options: Make total 10 connections (num-conns) at rate of 10 (rate) connections/second with 2 request calls per connection (num-calls).
Output: 10 connections with 20 request calls
httperf -v --server www.example.com --wlog=n,$HOME/tmp/reqs.txt_httperf --rate=10 --num-conns=10 --num-calls=2 --hog
Total: connections 10 requests 20 replies 10 test-duration 1.575 s
However, when I use following options, httperf output, output is confusing.
Options: Make total 4 connections (num-conns) at rate of 10 (rate) connections/second with 6 request calls per connection (num-calls).
httperf -v --server www.example.com --wlog=n,$HOME/tmp/reqs.txt_httperf --rate=10 --num-conns=4 --num-calls=6 --hog
Total: connections 4 requests 8 replies 4 test-duration 0.455 s
It seems like when num-calls is greater than num-conns, number of requests made are 2*num-conns.
I am not following why num-calls be greater than num-conns. Am I missing anything?
The reason why num-calls is greater than num-conns: on each connection, you can make multiple HTTP transactions (a.k.a "calls"). If num-conns = 4, on each connection, you make 2 transactions, then num-calls would be 8.
Hope this helps.
tracert returns requested time out. What I understand from this is the packets lost some where on the network.
Does it mean the issue is with the ISP or with the hosting provider or my windows system?
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
The first 9 were successful.
I can't see the first 9 hops but if they are all the same then you may have a firewall configuration issue that prevents the packets from either getting out or getting back.
Try again turning off your firewall (temporarily!). The other option is that your ISP may drop ICMP traffic as a matter of course, or only when they are busy with other traffic.
ICMP (the protocol used by traceroute) is of the lowest priority, and when higher priority traffic is ongoing the router may be configured to simply drop ICMP packets. There is also the possibility that the ISP drops all ICMP packets as a matter of security since many DOS (Denial of Service) attacks are based on probing done with ICMP packets.
Some routers view all pings as a Port-Scan, and block for that reason. (as the first step in any attack is determining which ports are open.) However, blocking ping packets / tracert packets, etc. is only partially effective at mitigating a Denial-of-service attack, as such an attack could use ANY PROTOCHOL it wanted (such as by using TCP or UDP packets, etc.) So long as there is an open port to receive the packet on the machine targeted for Denial-Of-Service. For example, if we wanted to target an http server, we only need use an intercepting proxy to repeatedly send a null TCP packet to the server on port 80 or port 8080, since we know that these are the two most common ports for http. Likewise, if the target machine is running an IRCd, we know the port is most likely 6667 (unless the server is using SSL), which would be the most common port for that kind of service. Therefore, dropping ping packets does not prevent a DdOS attack- it just makes that type of attack a bit more difficult.
This is what I found from the Wireshark documentation(I had the same problem):
"The tracert program provided with Windows does not allow one
to change the size of the ICMP message sent by tracert. So it won’t be
possible to use a Windows machine to generate ICMP messages that are large
enough to force IP fragmentation. However, you can use tracert to generate
small,fixed-length packets"
https://danielgraham.files.wordpress.com/2021/09/wireshark_ip_v8.1-2.pdf
use tracert -h 1
this will limit the number of times it tries a particular ip address to 1 try. h = hops. I had written a batch script a while back to scan my entire network to get a list of ips and computer networks and it would waste time on the fire wall that wouldnt answer and ip addresses that weren't assigned to any computers. Wicked annoying!!!!!! so I added the -h 1 to the script!! I runs through and makes a list in a text file. I hope to improve it in the future by running arp -a first to get a quck list of ips, then feeding that list into a script similar to this one. that way it doesn't waste time on unassigned IP's.
enter code here#echo off
enter code hereset trace=tracert
enter code hereset /a byte1=222
enter code hereset /a byte2=222
enter code hereset /a byte3=222
enter code hereset /a byte4=100
enter code hereset loop=0
enter code here:loop
enter code here#echo
enter code here%trace% %byte1%.%byte2%.%byte3%.%byte4%>>ips.txt
enter code hereset /a loop=%loop% + 1
enter code hereset /a byte4=%byte4% + 1
enter code here#echo %byte4%
enter code hereif %loop%==255 goto next
enter code heregoto loop
enter code here:next
enter code hereend
Your antivirus blocks the incoming packets , and in no case this option can be turned off because its the basic property of an antivirus i.e to block packets to prevent computer from normal as well as DOS (Denial of Service) attacks .