How does k6 interpret the following case:
I have two requests which I run in parallel with batch() and command line switch -u 4, that is 4 users.
In that case is k6 running the 2 requests per user? that is 2x4 = 8 times ?
And is that one user after the other in sequential order?
Your first assumption is correct, k6 will make 8 requests, but this will happen concurrently, not sequentially, since VUs are also run in parallel.
Note that it would be 8 requests if you limited the number of total iterations with -i 4, otherwise if you specified a --duration it would run as many iterations, and thus requests, as possible.
Consider posting on the k6 community forum if you need further help. (Disclaimer: I'm a k6 maintainer.)
Related
I'm using Locust to load test my web servers. I'm running Locust in distributed mode. The worker nodes are written in Java, and use the Locust/Java port using locust4j. The master node and the worker nodes are containerized, our orchestrator is Kubernetes. When I want to spin up more workers, I'm doing it from there.
The problem that I'm running into is that no matter how many users I add, or worker nodes I add, I can't seem to generate more than ~8000 RPM. This is confirmed by the Locust web frontend, as well as the metrics I'm collecting from my web server.
Does anyone have any ideas why this is happening?
I've attached an image of timings I've collected. The snapshots are from running the load test for 60 seconds, I'm timing it from a stopwatch.
The usual culprit in these kinds of situations is your servers can't handle more than that. In my experience, the behavior you'll see client side as the servers get overwhelmed is you'll start to see a slow but steady increase in response times. This is one big reason why Locust includes those in the metrics it shows you.
Based on what I'm seeing in your screenshots, this is most likely the case for you. You have some very low minimum times but your average, median, and 90%iles are a lot higher than your minimums; your maximums are very significantly higher than those. Without seeing your charts I can't know for sure but that's a big red flag.
For more things to look out for, check out this question in the FAQ (especially see the list of server stats to investigate):
https://github.com/locustio/locust/wiki/FAQ#increase-my-request-raterps
I wanted to start erlang while varying the number of cores
in order to test the scalability of my program. I expect that running the program on more cores should be faster than running on less cores.
How can I specify the core limits?
In fact, I have tried with smp -disable (and I supposed that it will run on 1 core? isn't it?) But the execution time still the same as with more cores.
I tried also to put +S 1:1 (assuming 1 scheduler so as to run on 1 core? as well as other scheduler numbers), but it seems nothing has changed.
Was that because of characteristic of my program or did I do something wrong on specifying the core limits?
And if possible could someone give some tips on how to scale your Erlang programs.
Thank you very much.
If I have a 3 node cluster. I need to run a specific Quartz job as follows:
There is at a given time, many (say 30) of these jobs that need to be run.
Limit the number of a that Quartz job running on all clusters combined at the same time (to 10, because of system resources)
Limit the number of a that Quartz job running on a single server at the same time (to 5, because of CPU load)
How do I limit both the total number of simultaneous job instances to 10, and the number running on any one host to 5? Is this even possible?
Note that I cannot limit the number of threads as I have other jobs that need to run on the same servers at the same time, and those need threads as well.
Thanks.
While not exactly limiting the consecutive job count, you can limit the maximum thread count with the thread pool configuration. See Quartz Configuration Reference.
The Grails Quartz plugin comes with a handy script for installing the config file:
grails install-quartz-config
org.quartz.threadPool.threadCount
Can be any positive integer, although you should realize that only
numbers between 1 and 100 are very practical. This is the number of
threads that are available for concurrent execution of jobs. If you
only have a few jobs that fire a few times a day, then 1 thread is
plenty! If you have tens of thousands of jobs, with many firing every
minute, then you probably want a thread count more like 50 or 100
(this highly depends on the nature of the work that your jobs perform,
and your systems resources!).
I think that I found the answer.
The answer is to run two (or more) separate Quartz schedulers. A job in the first scheduler would schedule the job for the second scheduler, and the second would run them. The second scheduler could then be limited to (in this case) 5 threads, although the first scheduler could have more.
Some information about this can be found in
http://quartz-scheduler.org/documentation/quartz-2.2.x/cookbook/MultipleSchedulers
However I do not know how to implement two separate Quartz Schedulers in Grails. If anyone could help with that I would appreciate it. There is an existing Stack Overflow question about this though, although it is unanswered.
I'm using savon gem to interact with a soap api. I'm trying to send three parallel request to the api using parallel gem. Normally each request takes around 13 seconds to complete so for three requests it takes around 39 seconds. After using parallel gem and sending three parallel requests using 3 threads it takes around 23 seconds to complete all three requests which is really nice but I'm not able to figure out why its not completing it in like 14-15 seconds. I really need to lower the total time as it directly affects the response time of my website. Any ideas on why it is happening? Are network requests blocking in nature?
I'm sending the requests as follows
Parallel.map(["GDSSpecialReturn", "Normal", "LCCSpecialReturn"], :in_threads => 3){ |promo_plan| self.search_request(promo_plan) }
I tried using multiple processes also but no use.
I have 2 theories:
Part of the work load can't run in parallel, so you don't see 3x speedup, but a bit less than that. It's very rare to see multithreaded tasks speedup 100% proportionally to the number of CPUs used, because there are always a few bits that have to run one at a time. See Amdahl's Law, which provides equations to describe this, and states that:
The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program
Disk I/O is involved, and this runs slower in parallel because of disk seek time, limiting the IO per second. Remember that unless you're on an SSD, the disk has to make part of a physical rotation every time you look for something different on it. With 3 requests at once, the disk is skipping repeatedly over the disk to try to fulfill I/O requests in 3 different places. This is why random I/O on hard drives is much slower than sequential I/O. Even on an SSD, random I/O can be a bit slower, especially if small-block read-write is involved.
I think option 2 is the culprit if you're running your database on the same system. The problem is that when the SOAP calls hit the DB, it gets hit on both of these factors. Even blazing-fast 15000 RPM server hard drives can only manage ~200 IO operations per second. SSDs will do 10,000-100,000+ IO/s. See figures on Wikipedia for ballparks. Though, most databases do some clever memory caching to mitigate the problems.
A clever way to test if it's factor 2 is to run an H2 Database in-memory DB and test SOAP calls using this. They'll complete much faster, probably, and you should see similar execution time for 1,3, or $CPU-COUNT requests at once.
That's actually is big question, it depends on many factors.
1. Ruby language implementation
It could be different between MRI, Rubinus, JRuby. Tho I am not sure if the parallel gem
support Rubinus and JRuby.
2. Your Machine
How many CPU cores do you have in your machine, you can leverage this using parallel process? Have you tried using process do this if you have multiple cores?
Parallel.map(["GDSSpecialReturn", "Normal", "LCCSpecialReturn"]){ |promo_plan| self.search_request(promo_plan) } # by default it will use [number] of processes if you have [number] of CPUs
3. What happened underline self.search_request?
If you running this in MRI env, cause the GIL, it actually running your code not concurrently. Or put it precisely, the IO call won't block(MRI implementation), so only the network call part will be running concurrently, but not all others. That's why I am interesting about what other works you did inside self.search_request, cause that would have impact on the overall performance.
So I recommend you can test your code in different environments and different machines(it could be different between your local machine and the real production machine, so please do try tune and benchmark) to get the best result.
Btw, if you want to know more about the threads/process in ruby, highly recommend Jesse Storimer's Working with ruby threads, he did a pretty good job explaining all this things.
Hope it helps, thanks.
I have a test pack consisting of more than 3000 scenarios.now the problem is that when i run the scenarios in one shot ..it takes approx 10 hours to complete,i want to divide the scenarios in 4 blocks,each of approx 750 scenarios and wanted to run them parallel in different windows/terminal(VMware).is there a workaround ???
This question has a selenium tag, so (assuming that's accurate) Selenium-Grid would be an option for setting up a distributed parallel testing environment.
orde mentions Selenium Grid and that's one piece of the puzzle. The main benefit you get out of that is that you only have to identify your Server Hub in your code when creating a new selenium instance.
The next thing would be to actually execute your 4 blocks of 750 scenarios at once. I'd recommend using a CI tool like Jenkins to accomplish that and you'll have your results together on Jenkins' web gui.