I am having a fully fledged suite for automated tests written using ruby with Appium for mobile automation.
I am running these suites in one simulator in one machine and it takes up a lot of time, around 1 hour to run 56 test cases(We have system test cases where multiple checks like database/Api/functional are integrated). We have more additional test cases adding to our way.
We have implemented running our tests across 3 mac machines currently with running different cucumber tags integrated to Jenkins. However, more addition of tests is only going to take us more time or more mac's
With xcode 9 we can initiate multiple simulators on one machine at the same time, and I had like to know, if there is any sample scenario or documentation on how to implement distributed tests across simulators in one mac machine
I had tried loading two or three different desired capabilities with different platform version, but it only loads the tests in sequential order.
I had gone through a lot of material online that has only the steps to make this possible in android. Does iOS support it?
Or could anyone possibly provide links that would help me? to
1. Implement distributed tests across various simulators in one mac
2. Use cucumber tags to distribute tests creating instance for each desired capability
Update:
I had tried implementing the multithread option and tried to initiate the tests to specific simulator creating instance with each thread. However, I find the tests not running in parallel but sequential.
This is my code:
def start_app(device_id, wdalocalport)
caps_config = {
platformName: "iOS",
wdaLocalPort: wdalocalport,
deviceName: "iPhone Simulator", #update device as per your need
app: (File.join(File.dirname(__FILE__), "Sequoia.app")),
bundleId: "something",
automationName: "XCUITest",
xcodeOrgId: "something",
xcodeSigningId: "iPhone Developer",
#platformVersion: "10.2",
noReset: "true",
fullReset: "false",
showIOSLog: "true",
autoAcceptAlerts: "true",
showXcodeLog: "true",
useNewWDA: "true",
resetOnSessionStartOnly: "true",
udid: device_id }
appium_lib_config={ port: 4723 }
$opts={ caps: caps_config, appium_lib: appium_lib_config }
setup
end
def setup
#appium = Appium::Driver.new($opts)
#appium.start_driver
#Makes all appium_lib methods accessible from steps
#Starts appium driver before the tests begin
end
def test(device1,device2)
threads = []
threads << Thread.new {
start_app(device1, '8100')
}
threads << Thread.new {
start_app(device2, '8200')
}
threads.each(&:join)
end
end
I am calling the launch tests using the test method passing the udid's. The simulators launch at the same time, and also installs the application at the same time, but the tests aren't parallel.
Any help to improvise this case?
I was able to use the rake to parallel run, but I still find this approach runs the tests in sequential manner or doesnt run at all
PFB the code
def run_cucumber(cucumber_options)
Cucumber::Rake::Task.new do |t|
t.cucumber_opts = cucumber_options
end
Rake::Task[:cucumber].invoke
end
task :iphone_7 do |t|
ENV['DEVICE'] = 'iphone7'
run_cucumber('-r features features/test.feature --format pretty --tags #slave1')
end
task :iphone_8 do |t|
ENV['DEVICE'] = 'iphone8'
run_cucumber('-r features features/test.feature --format pretty --tags #slave2')
end
multitask :all => [:iphone_7,:iphone_8]
My hooks.rb
Before do
check
end
def check
if ENV['DEVICE'] == 'iphone7'
start_app('iPhone6','port','udid')
elsif ENV['DEVICE'] == 'iphone8'
start_app('iphone6','port','udid')
else
puts "Device not"
end
end
I have been getting DEVICE NOT. Not sure what am I missing.
You need to create a method which invokes cucumber rake first in the Rakefile as follows:
def run_cucumber(cucumber_options)
Cucumber::Rake::Task.new do |t|
t.cucumber_opts = cucumber_options
end
Rake::Task[:cucumber].invoke
end
This method will need cucumber_options. You can pass same options as you pass on the command line like this:
-r features features/test.feature --format pretty --tags #test1
After this, define rake tasks for a particular configuration in the same Rakefile:
task :iphone_7 do |t|
ENV["DEVICE"] = "iphone 7"
run_cucumber('-r features features/test.feature --format pretty --tags #test1')
end
task :iphone_8 do |t|
ENV["DEVICE"] = "iphone 8"
run_cucumber('-r features features/test.feature --format pretty --tags #test2')
end
Here, ENV["DEVICE"] is used set to device config. You can use this variable in your env.rb in Before block to initialize the test. In Before block you can initialize devices based on this environment variable.
You can create rake tasks like this as many as you need.
Then you can invoke multiple tasks at the same time by using ruby's parallel gem. Examples are in this link -> Ruby Parallel gem
Just pass different types of rake task as the arguments in the Parallel object.
Eg:
task :run_parallel do |t|
Parallel.map([:iphone_7,:iphone_8]) {|task| Rake::Task[task].invoke }
end
And then trigger the run_parallel rake task to execute the whole code.
Eg:
bundle exec rake run_parallel
Related
I have a few tests that download files and assert data inside. The problem I am facing is that the tests run in parallel so I can't delete the download directory after each test or else they delete each others files. The issue with not doing so however is that the filename includes timestamp(unique identifier) which is not known to the test so not possible to know which file to open. Is there a way to change default_directory for a given test in the middle of test run? The idea is to be able to tell capybara that for certain tests override the download path to be another path temporarily? I guess there could be cross wiring here too if it was possible as other tests could still be running expecting the original path to be set?
Alternatively, any suggestions on handling this?
My capybara config looks like this
options = Selenium::WebDriver::Chrome::Options.new
preferences = {
prompt_for_download: false,
credentials_enable_service: false,
default_directory: DownloadUtil::PATH
}
options.add_preference(:download, preferences)
options.add_argument('--disable-infobars')
options.add_argument('--headless')
Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)
end
Most multi-process parallel test setups provide you with an environment variable you can use to configure things that need to be different between each instance of test runner (DB name, ports, etc). In the case of parallel_rspec that is TEST_ENV_NUMBER. Using that you can configure the selenium/chrome instance in each test runner to use a different download directory - something like
preferences = {
prompt_for_download: false,
credentials_enable_service: false,
default_directory: DownloadUtil::PATH + ENV['TEST_ENV_NUMBER']
}
Currently, I'm trying to migrate a large data set from an IBM DB2 database. I found a quick way of doing queries using imb_db2 gem (version 4.0.0).
The main goal is to have a script that runs in parallel and imports independent data.
My import script works fine sequential, but when trying to execute the script into multiple background processes, it crashes the process.
I tried Sidekiq with multiple processes, each process with its own queue.
For example, I try the following approach
DemoWorker.set(queue: :que_1).perform_async(1, 10)
Then I do
DemoWorker.set(queue: :que_2).perform_async(11, 20)
Each queue has its independent Sidekiq process, with concurrency 1. E.g.
bundle exec sidekiq -e development -C config/sidekiq.yml -q que_1
And I execute the following code with a query as a parameter
def get_by_sql(sql)
data = []
stmt = IBM_DB.exec(self.conn, sql.squish)
while row = IBM_DB.fetch_assoc(stmt)
data << row
end
data
end
And from time to time I get the following crash error.
[BUG] object allocation during garbage collection phase
I have no idea why it fails, could you advise? Thank you very much!
And here is the crash report: https://ufile.io/fqmfh0y8
EDIT:
Crash Report uploaded to GIST: https://gist.github.com/RaresSalcudean/db8467bc4cd3ea2164bcaab5da86780e
Crash 2 Report: https://gist.github.com/RaresSalcudean/6f7bb7f74e34a537b687c776d806bf04
I have the following thor command:
require 'highline'
class Import < Thor
desc "files", "Import files into the database"
method_option "path", :required => true, :desc => "Path to folder containing new files", :aliases => "-p", :type => :string
def files
require './config/environment'
line = HighLine.new
line.say(line.color("Identified files as Version 15 (English)", :green))
if line.agree(line.color("Are you sure you want to import?", :yellow))
line.say(line.color("Finished. Imported 70,114 items", :green))
else
line.say(line.color("Aborting...", :red))
end
end
end
Now, obviously, at the moment this is just outputting some language to the screen. However, what I need to do is write a test for the command that tests the output is as I would expect, and that when I start hooking in the heavy lifting that I can stub that stuff out.
I've had a look at Aruba, but this doesn't appear to like interactivity for some reason, and it's not clear why.
Therefore, does anyone have any ideas on how this might be testable (with RSpec)?
Aruba is a pretty complete set of steps for testing command line apps. If it's not working for you it might be because aruba defaults all file operations into tmp/aruba.
But neimOo is right about how to write the scenario with aruba
When I run `thor import` interactively
And I type "yes"
Here is how you can do this with Aruba
Scenario: Test import
When I run `thor import` interactively
And I type "yes"
Then the stdout should contain "Finished. Imported 70,114 items"
Here you can find a lot of aruba examples
https://github.com/cucumber/aruba/blob/master/features/interactive.feature
And here is implementation itself
https://github.com/cucumber/aruba/blob/master/lib/aruba/cucumber.rb
I would like to be able to perform some tests that require a worker to be running.
In order to accomplish this created this test helper method:
def with_delayed_jobs
t=Thread.new {Delayed::Worker.new.start}
sleep(5)
yield
t.exit
end
So I can write in my tests
with_delayed_jobs {
___test_content___
}
Unfortunately, the worker doesn't seem to run this way. Maybe I can do it with processes. Does anybody have an idea on how to accomplish this?
You can run the jobs that are currently in your jobs table with this:
Delayed::Worker.new(
:max_priority => nil,
:min_priority => nil,
:quiet => true
).work_off
Docs are here, although they are sparse.
I'm adding a test task in my Rakefile, similar to this:
namespace :test do
desc "Test lib source"
Rake::TestTask.new(:lib) do |t|
t.libs << "test"
t.pattern = 'test/lib/**/*_test.rb'
t.verbose = true
end
end
and then adding (have also done using "enhance" with the same result:
task :test => [ 'test:lib' ]
My problem is that if there is an error encountered in test:lib, the suite stops running. That's not a terrible thing, but ideally it would go on to run the rest of the suite to let me know that there are more issues later in the suite.
Anyone know how to make it report the errors/failures in test:lib but go on to run the full suite?
Thanks!
Used something along these lines (minus the "on the fly" bit):
http://toolmantim.com/articles/creating_rake_testtasks_on_the_fly