I am using Snapshot from the FastLane suite.
For my purposes I am calling the various tools from scripts and pass in the appropriate environment variables I am using as the inputs.
I am having trouble when I call my script in Jenkins vs from the command line. When I call the script within a Build Step in Jenkins the result is a message from Snapshot saying the process has timed out after waiting 120 seconds for the simulator to boot. If I run this same script from the terminal Snapshot runs as expected without error.
Example:
snapshot \
--workspace "MyWorkspace.xcworkspace" \
--scheme "MyScheme" \
--output_directory "MyOutputDirectory" \
--clear_previous_screenshots \
--stop_after_first_error
(--devices --languages can be found in ./Snapfile)
Snapfile:
devices([
"iPhone 4s"
])
languages([
"en-US"
])
Am I missing something here?
Configuring Jenkins to work for iOS testing and automation is not a simple task, there are a lot of gotchas.
Jenkins the result is a message from Snapshot saying the process has timed out after waiting 120 seconds for the simulator to boot.
This suggests that your Jenkins machine is not able to run the Simulator. This can happen if the jenkins user is not able to start a UI session.
These two posts have useful information on how to configure Jenkins for iOS development:
https://blog.pivotal.io/labs/labs/ios-ci-jenkins
http://staxmanade.com/2015/01/setting-jenkins-up-to-run-xctool-and-xcode-simulator-tests/
The second in particular addresses the issue of Jenkins not running as a GUI user.
Good luck.
Related
I'm trying to get a Jenkins job to run sfdx force:data:soql:query commands in order to migrate configuration data sets between our production org and our sandboxes after a refresh. Certain configurations do not persist on a refresh so we need a way to move that data.
Running the queries from the command line on the Jenkins server work as expected, however the job when it runs fails with the following error:
'C:\Program' is not recognized as an internal or external command, operable program or batch file.
Build step 'Execute shell' marked build as failure
The job does three things:
Authorizes to the DevHub, lists out the connected orgs, and then performs a SQOL query to just print some data - 16 lines to be exact. Here are the commands in the shell script of the job:
sfdx force:auth:jwt:grant -i ${CONNECTED_APP_CONSUMER_KEY} -u ${DEV_HUB} -f ${JENKINS_HOME}/certs/prod/server.key -r [...] -a DevHub
sfdx force:org:list
sfdx force:data:soql:query -u ${DEV_HUB} -q "SELECT Id, Name FROM [...tablename...]" -r human
I am completely stumped on why this is happening. Again, running the SOQL command directly on the server through PowerShell or Command Line works as expected. I would appreciate any help with this.
This one stumped me for a long time but we finally got it figured out.
If you are seeing this error, make sure to check your machine's environmental variables. I saw a TON of other answers pointing to this as the issue where the install of SFDX path name had spaces in it as in C:|P:rogram Files\SFDX\bin but only showed some weird command line FOR loop that made no sense what so ever.
What we did was to completely uninstall all of SFDX making sure none of it was left on the machine and reinstalled into a folder we made where there was no spaces in the path name.
Once we did that, our job worked like it was supposed to. I hope this helps others who run into this same issue.
I am new to Jenkins and have set it up with 4 slaves and trying to run Cucumbers (Capybara+Selenium+Firefox) in vncserver(tried xvfb as well) mode. But some of the slave nodes timeout randomly throwing "execution expired (Timeout::Error)" at the line below
Capybara.current_session.driver.browser.manage.window.resize_to(1600, 900)
and in
page.driver.browser.save_screenshot(screenshot_path)
Starting from here all the remaining tests fail. Have tried using the Xvnc plugin in Jenkins as well but still fails. How do I fix this? Should I use a window manager? Will tightvnc use any default Window Manager? Is it related?
My ~/.vnc/xstartup
#!/bin/sh
xrdb $HOME/.Xresources
xsetroot -solid grey
#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
#x-window-manager &
# Fix to make GNOME work
export XKL_XMODMAP_DISABLE=1
/etc/X11/Xsession
Does someone know what is causing the timeout error?
Looks like this happened because it took time for the browser to open at times so we have to wait or assert till the browser loads. Link here: https://sqa.stackexchange.com/questions/15345/selenium-webdriver-sometimes-fails-to-connect-to-the-browser-when-running-tests
I have calabash running iOS tests on Jenkins. When the job encounters fails I sometimes manually abort the tests by pressing the stop[x] button within the job. The problem is the next test in the feature file begins running even though I aborted. This behavior is not observed when launching the tests through the terminal. When exiting the cucumber test in the terminal the sim returns to home and no other test are launched.
I found a hook that might be useful
After do |s|
# Tell Cucumber to quit after this scenario is done - if it failed.
Cucumber.wants_to_quit = true if s.failed?
end
However, there are times when I don't want it to stop just because one scenario failed. I feel like Jenkins needs to kill all processes and its not.
If someone knows how to kill calabash and its instances manually via terminal after Jenkins has been instructed to abort, I would be interested in that too.
I tried:
ps aux | grep -i instruments | awk {'print $2'} | xargs kill -9
Unfortunately that did not work. Possibly two reasons
greping instruments shows two or more process
20272 ?? S 0:00.00 sh -c xcrun instruments -w "iPhone 5 (8.1 Simulator)...
20273 ?? S 0:00.45 /Applications/Xcode.app/Contents/Developer/usr/bin/instruments -w iPhone 5]...
Should I switch awk to print column 1?
or reason two
I'm not greping the correct process?
Here is some of my version info:
calabash-ios version: 0.11.4
Calabash::Cucumber::MIN_SERVER_VERSION: 0.11.4
Xcode 6.1
You have to let Jenkins to find all forked processes. Depending on the Job type you have to pass different environment entries into the forked process. This question is about the other way (so how to make Jenkins NOT to stop processes), but the names of the possible environment entries are there. Just pass these environment entries below to each forked process and then the process tree killer will find them:
BUILD_ID
HUDSON_SERVER_COOKIE
JENKINS_COOKIE
JENKINS_SERVER_COOKIE
HUDSON_COOKIE
I am currently trying to run JSCover in web server mode to determine the coverage of my Jasmine tests that are executed in the PhantomJS headless browser. I am also using grunt+nodejs to kick off the tests.
The code I use in my gruntfile to start the JSCover server and execute phantomJS is:
// Start JSCover Server
var childProcess = require('child_process'),
var JSCOVER_PORT = "43287";
var JAVA_HOME = process.env.JAVA_HOME;
var jsCoverChildArgs = [
"-jar", "src/js/test/tools/JSCover-all.jar",
"-ws",
"--branch",
"--port="+JSCOVER_PORT,
"--document-root=./",
"--report-dir=target/",
"--no-instrument=src/js/lib/",
"--no-instrument=src/js/test/",
"--no-instrument=src/js/test/lib/"
];
var jsCoverProc = childProcess.spawn(JAVA_HOME + "/bin/java", jsCoverChildArgs);
// Start PhantomJS
var phantomjs = require('phantomjs'),
var binPath = phantomjs.path,
var childArgs = [
'src/js/test/lib/phantomjs_jasminexml_runner.js',
'http://localhost:'+JSCOVER_PORT+'/src/js/test/SpecRunner.html',
'target/surefire-reports'
];
runner = childProcess.execFile(binPath, childArgs);
runner.on('exit', function (code) {
// Tests have finished, so clean up the process
var success = (code === 0) ? true : false;
jsCoverProc.kill(); // kill the JSCover server now that we are done with it
done(success);
});
However, when I run the web server on a Jenkins node in cloudbees and then run phantomjs against it, I get one of the following errors:
Some tests start to run, but then the process fails:
A spec : should be able to have a mock lo-dash ...
Warning: Task "test" failed. Use --force to continue.
Aborted due to warnings.
Build step 'Execute shell' marked build as failure
Recording test results
Finished: FAILURE
PhantomJS is unable to access the JSCover server:
Running "test" task
phantomjs> Could not load 'http://127.0.0.1:43287/src/js/test/SpecRunner.html'.
Warning: Task "test" failed. Use --force to continue.
For the second error, I have tried to use different ports and hostnames that I set (e.g. 127.0.0.1 or localhost for hostnames, and 4327, 43287, etc. for ports). The ports are not being dynamically set at build time - I have them hardcoded in my grunt script.
Any thoughts on why the errors above might be occurring or why I am having issues running and accessing the JSCover server on a Cloudbees Jenkins node (but never on my local machine)?
So when you execute JSCover with any process, it takes time to be up. If we expect it to be up earlier that it is, the errors are bound to come.
Quoting from the great article: http://blog.johnryding.com/post/46757192364/javascript-code-coverage-with-phantomjs-jasmine-and
Now that I had a code coverage tool that met all of my requirements,
the last part was to get this code to run as part of our Jenkins build
(which utilizes a grunt script). This was easy to get running, but I
encountered two errors that consistently broke my builds:
Sometimes phantomJS would fail to connect to the JSCover server
Sometimes phantomJS would connect to the server, but then give up executing my tests at a random point during the run.
These were really weird issues that only occurred on my team’s Jenkins nodes and were hard to diagnose - even though they turned out to be simple fixes.
For issue 1, that error was the result of my grunt script not waiting for JSCover to start before I executed phantomJS.
For the second issue, it turns out that my team was using a special jasmine test runner to help with producing XML files after tests completed. The problem with this file was that it had a function that waited for Jasmine to complete its execution, but utilized an extremely short timeout before it gave up running the tests. This was a problem with Jenkins + JSCover because it took a longer time for the tests to load and run now that they had to be loaded from a web server instead of straight from the file system. Fortunately, this fix was as easy as increasing the timeout.
I would say that you need to wait for a while after spawning JSCover - in the past I have done things with webdriver when I have spawned, and then waited for it to be available (ideally you can look for a response and sleep, repeat, until the spawned process is ready).
Ie look for a valid http reponse from 127.0.0.1:43287 before continuing (whatever "valid" means that the server is up).
For a while now I've been trying to work out how to run UIAutomation tests from Jenkins - every time I run the build, it builds fine, then it runs my instruments command (using the same command as detailed here ( Can Instruments be used using the command line?) and jenkins just hangs, well the whole machine does, and when I look at activity monitor I can see an instruments process using 2gb of memory.
When I set up jenkins, I original ran it as from a hidden user - this presented some challenges with jenkins being a deamon and not being able to access the window server. I then decided to change the jenkins account to a normal user, logged in and ran instruments from the command line - this worked fine.. but still had no luck with running it from jenkins.
I have set the jenkins account as a developer - no admin though
Please let me know if there's anything else that I could try, or if anyone has got this running successful your guidance would be much appreciated - Thanks
Jenkins on OS X is started from a launchd script and will run as "daemon" by default. The thing to do is change the user in the launched script.
First, get Jenkins ready to shutdown (in "Manage Jenkins" in the GUI).
Then unload the job from launchd, like so:
$ sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
Then edit the "UserName" property in the launchd plist, using the user which you want to run jenkins. There's also a GroupName property, which you may want/need to adjust accordingly with your user's group.
Finally, reload Jenkins with:
$ sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist
Hope that helps!
So if you run it as a daemon, first thing to check what happens if you run Jenkins in the foreground The simplest way to do it is with java -jar jenkins.war [other options] command (see this document).
Maybe you can use this https://github.com/houlianpi/robot4ios.
Then in jenkins execute shell:
sh setup.sh
sh runTests.sh ./sample/alltests.js "/Users/komejun/Library/Application Support/iPhone Simulator/5.0/Applications/1622F505-8C07-47E0-B0F0-3A125A88B329/Recipes.app/"
and the report will be auto create in ./ynmsk-report/test.xml