How do I pump traffic using tcpreplay at 100 MBps, 500 MBps and 1Gbps speeds? - network-programming

I used the -R and -K option but it doesnt seem to be working as I captured the pumped traffic using tcpdump and the number of packets that I see there dont seem to match the number of packets that I expect in the time frame.

First of all make sure you are using the latest version, available here. You will want to use the -K and --mbps (or -M) options, for example:
# tcpreplay -i eth7 -K --mbps 1000 smallFlows.pcap
File Cache is enabled
Actual: 14261 packets (9216531 bytes) sent in 0.073761 seconds.
Rated: 124951275.0 Bps, 999.61 Mbps, 193340.65 pps
Flows: 1209 flows, 16390.77 fps, 14243 flow packets, 18 non-flow
Statistics for network device: eth7
Attempted packets: 14261
Successful packets: 14261
Failed packets: 0
Truncated packets: 0
Retried packets (ENOBUFS): 0
Retried packets (EAGAIN): 0
When you attempt to move to higher speeds (e.g. 10GigE) you may need to generate a bigger block of data by using the --loop option. Also with Tcpreplay version 4.0 there are the more advanced --netmap and --unique-ip options which on a properly set up system, will achieve near wire rate and very high flows/sec. More information available at Tcpreplay How To. Here is an example:
# tcpreplay -i eth7 -K --mbps 9500 --loop 100 --netmap --unique-ip smallFlows.pcap
Switching network driver for eth7 to netmap bypass mode... done!
File Cache is enabled
Actual: 1426100 packets (921653100 bytes) sent in 0.776133 seconds.
Rated: 1187493767.1 Bps, 9499.95 Mbps, 1837442.80 pps
Flows: 120900 flows, 155772.27 fps, 1424300 flow packets, 1800 non-flow
Statistics for network device: eth7
Attempted packets: 1426100
Successful packets: 1426100
Failed packets: 0
Truncated packets: 0
Retried packets (ENOBUFS): 0
Retried packets (EAGAIN): 0
Switching network driver for eth7 to normal mode... done!

Related

uwsgi log format: seeing uwsgi req: N/M where N > M

I have a uwsgi process running a flask application. There is haproxy (running in mode http) sitting between the client and the application.
I am seeing occational haproxy termination state as "SD--" and the Tc = 0 and Tr = -1, and the returned http code is -1. This means that the haproxy encountered a explicit tcp disconnection from the uwsgi server.
Looking at the uwsgi logs, I found that the server was normally processing requests at the same time. But the affected request never reached the server.
Only thing strange about the uwsgi logs at that point of time is that
the Number of requests managed by the current uwsgi worker is greater than the sum total of requests managed by the whole uwsgi app.
like this:
[pid: 22759|app: 0|req: **47188**/**47178**] * POST * => generated 84 bytes in 970 msecs (HTTP/1.1 200) 2 headers in 71 bytes (3 switches on core 98)
I am wondering if this is abnormal, or what what scenarios can these counters be so?

JenkinAPI error with server.get_jobs "max retries exceeded with url"

From the beginning of the new year, i cannot extract jobs info from my Jenkins, using the classic example that can be found here: https://jenkinsapi.readthedocs.io/en/latest/using_jenkinsapi.html#example-2-get-details-of-jobs-running-on-jenkins-server. In replace of them, i get the following error:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host=,
port=): Max retries exceeded with url:
/job/<JOB_NAME>/api/python?tree=allBuilds%5Bnumber%2Curl%5D (Caused by
ReadTimeoutError("HTTPSConnectionPool(host=, port=): Read
timed out. (read timeout=10)"))
Thanks for any advice.
You will get this error if the host you are connecting to is not available.
I would suggest trying a few network tests to ensure the host is visible:
where:
ReadTimeoutError("HTTPSConnectionPool(host='myhost.something.com', port=1234)
try netcat or tracepath:
-> nc -w 5 -vz myhost.something.com 1234
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 10.11.12.13:1234.
Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.
-> tracepath -p 33434 myhost.something.com

tcpreplay, different packet number with pcap file

I have a pcap file, captured with Wireshark,with 2690 packets
I use tcpreplay to replay it on an interface, but the problem is that the number of Attempted packets in tcpreplay is different(less) with number of packets showing in wireshark
#tcpreplay -i eth1 test.pcap
sending out eth1
processing file: /tmp/reloadtest4.pcap
Actual: 1826 packets (1634597 bytes) sent in 58.06 seconds.
Rated: 28153.6 bps, 0.21 Mbps, 31.45 pps
Statistics for network device: eth1
Attempted packets: 1826
Successful packets: 1826
Failed packets: 0
Retried packets (ENOBUFS): 0
Retried packets (EAGAIN): 0
It may be a bug in Tcpreplay, or it may be an issue with your capture. I suggest logging the issue here. Include the output of tcpreplay -V. Also rename the packet capture with a suffix of .txt and you will be able to upload it to the issue.

Nginx + passenger serving 4 requests/s for a static page

I deployed my app to Digitalocean with Passenger and Nginx. I used apache bench to see how many requests per second I can get on a static page (simple hello world rails view), but I am only getting 4 requests/s.
ab -n 100 http://107.170.100.242/fo
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 107.170.100.242 (be patient).....done
Server Software: nginx/1.8.0
Server Hostname: 107.170.100.242
Server Port: 80
Document Path: /fo
Document Length: 5506 bytes
Concurrency Level: 1
Time taken for tests: 22.662 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 632600 bytes
HTML transferred: 550600 bytes
Requests per second: 4.41 [#/sec] (mean)
Time per request: 226.617 [ms] (mean)
Time per request: 226.617 [ms] (mean, across all concurrent requests)
Transfer rate: 27.26 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 181 226 65.4 204 445
Waiting: 181 226 65.4 204 445
Total: 181 227 65.4 204 446
It should be literally thousands per second as I am using Nginx. I have been researching this for the entire day without results, can someone please direct me to the right path to solve this?
This would be the nginx config directive that will cause it to bypass the app server and serve the static files directly:
root /var/www/my_app/public;
Are you sure that is right?

How to determine what is blocking a connection to website?

I need to connect, let's say a script, from an IP to a website and I can't, because:
wget: domain.com…
--2014-11-10 11:20:53-- domain.com…
recognized: domain.com... <IP>
Connecting with: domain.com|<IP>... error: Connection Timeout.
Connect again
I asked the admin of the server I host the domain on to give me ping and tracert to that IP and here it is:
pat#daz:/# ping 188.127.246.50
PING 188.127.246.50 (188.127.246.50) 56(84) bytes of data.
^C
--- 188.127.246.50 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1009ms
pat#daz:/# traceroute 188.127.246.50
traceroute to 188.127.246.50 (188.127.246.50), 30 hops max, 40 byte packets
1 79.133.193.65 (79.133.193.65) 1.348 ms 0.994 ms 0.796 ms
2 ae1-JunM10.etop.pl (79.133.201.129) 72.054 ms 56.309 ms 52.532 ms
3 frm-k90-cr1.rascom.ru (80.81.192.166) 20.489 ms 20.356 ms 20.388 ms
4 * * *
5 oversun.w-ix.net (193.106.112.195) 63.164 ms 59.526 ms 124.898 ms
6 * * *
7 * *^C
The website's .htaccess is clean, robots.txt too, hosting's admin says there's nothing from their side (firewall, blacklist etc) that could be blocking that IP from connecting.
What could it be then? How can I check it?

Resources