Automating frank from jenkins - jenkins

I am trying to automate a test session from jenkins using Frank.
this is the error message I receive.
(Frank::Cucumber::FrankNetworkError)
./step_definitions/test_steps.rb:30:in `/^I navigate to button V\303\244der$/'
test.feature:41:in `When I navigate to <link>'
| Radar | Norrland |
*********************************************
Oh dear. Your app fell over and can't get up.
*********************************************
Jenkins checks out the code from git, besides this I have added a shell command as follows.
cd ios #<--this is so that I go to the root folder, the one with the .xcodeproj project
frank setup
frank build
frank launch
mv features/*.feature Frank/features/. #<--- this is the testscript
mv features/step_definitions/*.rb Frank/features/step_definitions/. #<--here it is moved to the newly created frank/features & Frank/features/step_definitions folder
cd Frank/features
cucumber test.feature
Everything is built the way it should and if I go to the server
and manually type the last row from my shell command will the tests be executed.
Best Regards

I have personally had many problems running Jenkins on Mac. Especially if you used the Jenskins Mac image installer, Jenkins is always run under the "jenkins" user that is created during installation.
This has given me many headaches with running cucumber from inside jobs or for starting the iOS simulator.
I have finally learned to start Jenkins under my own user, like this:
"nohup java -jar /Applications/Jenkins/jenkins.war --httpPort=8080
Since then, I was able to run cucumber without any problems. Hope this helps.

This happens every time your application crashes. When the application crashes, frank stops receiving events and cucumber ends with the error you see.
There are two possible reasons:
You have a bug in your app that made the app crash
Frank has a bug that made your app crash
You should inspect the crash/application log to see the exact reason.
I learned that is very helpful to capture the application log by the jenkins job, e.g.
function grab_log_and_fail {
APP_NAME = "MyApplication"
# get the PID of the last process with the given name
PID=$( cat /var/log/system.log | grep "$APP_NAME\[" | tail -n 1 | sed -e "s/^.*$APP_NAME\ [\([^\]*\)\].*/\1/g" )
# grab all the messages from the process with the PID
cat /var/log/system.log | grep "$APP_NAME\[$PID\]" >"$WORKSPACE/$APP_NAME.log"
#fail the job
exit 1
}
You can call it using
cucumber test.feature || grab_log_and_fail
(will grab the log only if cucumber ends with an error)

Related

Have Jenkins force quit cucumber script if new build is pushed

So I am using Ruby/Cucumber and Appium to run automated tests after each of our builds on our Jenkins server.
Essentially I have a job set up that runs our regression suite after every build that is successful. My problem comes in when my cucumber tests are running and a new build is successful.
At the moment Jenkins will start a new series of tests with the cucumber script, but won't force the old script to quit. This leads to all of my cucumber reports showing multiple false negatives. Is there a way to stipulate that Jenkins run a script that quits cucumber before starting a new set of tests?
Thanks a ton for any help you can give me. If you have any questions about how I have Jenkins set up specifically just ask and I'll give you the more intimate details.
Technically Jenkins Job will wait for 1 instance of job to complete until it starts another one. So you dont have to worry about starting another build before first one finishes...
Now coming to your question how to kill old/existing test cases if a set of new ones want to start.
I would suggest to have a cleanup script run before your actual tests are triggered, your tests would be triggering using a rake or direct cucumber command, I am guessing. So before this happens, have a cleanup.sh executed which will do the following:
1) Restart appium server.
2) Kill running ruby processes.
3) Kill running cucumber processes.
The script should look something like this:
kill -9 $(ps -ef | grep \[a]ppium | awk '{print $2}')
kill -9 $(ps -ef | grep \[r]uby | awk '{print $2}')
kill -9 $(ps -ef | grep \[c]ucumber | awk '{print $2}')
# restart appium again
appium &
Hope it helps!! Let me know in comments if you run into issues.. :)

Jenkins does not stop cucumber tests when aborting (pressing the stop[x] button)

I have calabash running iOS tests on Jenkins. When the job encounters fails I sometimes manually abort the tests by pressing the stop[x] button within the job. The problem is the next test in the feature file begins running even though I aborted. This behavior is not observed when launching the tests through the terminal. When exiting the cucumber test in the terminal the sim returns to home and no other test are launched.
I found a hook that might be useful
After do |s|
# Tell Cucumber to quit after this scenario is done - if it failed.
Cucumber.wants_to_quit = true if s.failed?
end
However, there are times when I don't want it to stop just because one scenario failed. I feel like Jenkins needs to kill all processes and its not.
If someone knows how to kill calabash and its instances manually via terminal after Jenkins has been instructed to abort, I would be interested in that too.
I tried:
ps aux | grep -i instruments | awk {'print $2'} | xargs kill -9
Unfortunately that did not work. Possibly two reasons
greping instruments shows two or more process
20272 ?? S 0:00.00 sh -c xcrun instruments -w "iPhone 5 (8.1 Simulator)...
20273 ?? S 0:00.45 /Applications/Xcode.app/Contents/Developer/usr/bin/instruments -w iPhone 5]...
Should I switch awk to print column 1?
or reason two
I'm not greping the correct process?
Here is some of my version info:
calabash-ios version: 0.11.4
Calabash::Cucumber::MIN_SERVER_VERSION: 0.11.4
Xcode 6.1
You have to let Jenkins to find all forked processes. Depending on the Job type you have to pass different environment entries into the forked process. This question is about the other way (so how to make Jenkins NOT to stop processes), but the names of the possible environment entries are there. Just pass these environment entries below to each forked process and then the process tree killer will find them:
BUILD_ID
HUDSON_SERVER_COOKIE
JENKINS_COOKIE
JENKINS_SERVER_COOKIE
HUDSON_COOKIE

Error when attempting to run UI automation script from jenkins

I am using Xcode 6.1. And i need to run a UI automation script from jenkins as a post build action. The command that i use is shown below.
instruments -t $TRACETEMPLATE -w $DEVICE $APP_PATH -e UIASCRIPT $SCRIPT -e UIARESULTSPATH $RESULTS_PATH | grep "<" > test-reports/test-results.xml
When i run that the following error is thrown by jenkins.
Failed to authorize rights (0x1) with status: -60007.
2014-11-12 16:31:30.685 instruments[488:2607] -[XRSimulatorDevice prepareConnection:]: Unable to authorize simulated daemon (99637): 8
Instruments Trace Error : Target failed to run: Permission to debug com.test.app was denied.
Any help is much appreciated.
The user that invokes Instruments must have developer permissions. The user must be in the _developer group.
Security permissions allowing the user to access Instruments must be set. See the man page for DevToolsSecurity
The user must be logged in to a window server to use the simulator. How to do this will depend somewhat on your Jenkins and OS configuration for that user. In older versions of MacOS creating an SSH connection back into the machine and running Instruments through that connection typically worked. YMMV.
Note that any of the above steps escalates the rights for the Jenkins user, which was security implications.
I got an almost identical error message to yours, and it seemed to be because when Jenkins is launched as a Launch Daemon, it doesn't have access to the screen, even if you log in as "jenkins".
I found the solution from reading this discussion: https://issues.jenkins-ci.org/browse/JENKINS-14421
You can either launch Jenkins from Terminal with java -jar jenkins.war, or you can create a Launch Agent, which runs as the logged-in user and has access to the user's screen.
I achieved the latter solution by moving /Library/LaunchDaemons/org.jenkins-ci.plist to /Library/LaunchAgents/org.jenkins-ci.plist and removing the UserName key and value from the plist. Now Jenkins doesn't start automatically when I boot the machine, but it does start when I log into the desktop, which is required for running UI Automation tests anyway.
It appears that Jenkins still can't actually launch the Simulator app, but if Simulator is already running, the UI Automation scripts proceed just fine.
I also got this error message. Moving service from LaunchDaemons to LaunchAgents didn't solve the problem. My solution was as following:
remove the service by deleting /Library/LaunchAgents/org.jenkins-ci.plist file
create an iOS application using Automator tool. (Which I think exists by default):
I chose "Library" -> "Utilities" -> "Shell Script".
Enter the content of /Library/Application Support/Jenkins/jenkins-runner.sh.
Put export JENKINS_HOME=/path/to/jenkins at the top.
Save the app somewhere
Go to Preferences -> Users & Groups -> choose your user -> Add the saved app.
This way, jenkins is run as an application after login and it has all the privilages of any other application.

Register for messages from collabd like XCSBuildService to receive Xcode Bots integration number

During an Xcode Bots build each run gets an integration number assigned. This number does not show in the build logs but would be convenient to create http links back to the individual Xcode Bots build in an CI environment or enterprise app store.
In the /Library/Server/Xcode/Logs/xcsbuildd.log I found that XCSBuildService gets a message/event from collabd with a structure that contains the integration number.
Is there a way to register to the same event in an own (OSX) program and receive this message/event?
The only ugly way I found so far to get the Xcode Bots integration number was by parsing the xcsbuildd.log file with the drawback that I can't be sure that the latest integration number corresponds with my current build when several builds are executing in parallel.
This log file is also located in an read protected folder/file so that I have to either change the permissions (ugh!) or use sudo (I don't really want to do that)!?
Example:
sudo grep -r "integration =" /Library/Server/Xcode/Logs/xcsbuildd.log | tail -1 | cut -d'=' -f 2| cut -d';' -f 1 |tr -d '\040\011\012\015'
Gives me the latest integration number stripped from whitespace ...
Edit:
Just found out that if you actually include the following script in your scheme as Build-post-action it will create a file (e.g. /Library/Server/Xcode/Data/BotRuns/Cache/22016b1e-2f91-4757-b3b8-322233e87587/source/integration_number.txt) with the integration number without requiring sudo. Xcode Bots seems to serialize the different builds so that they are not executed in parallel and so the created integration number in the file could be used.
grep -r "integration =" /Library/Server/Xcode/Logs/xcsbuildd.log | tail -1 | cut -d'=' -f 2| cut -d';' -f 1 |tr -d '\040\011\012\015' > ${PROJECT_DIR}/integration_number.txt
For specifically the case of getting the integration build number, I've added a simple answer here.
To repeat myself (and keep this from being down voted as a 1-line answer):
I implemented this using a Shell Script Build Phase in my Xcode project. In my case, I used the integration number to set the internal version of my built product. My script looks like this:
if [ "the$XCS_INTEGRATION_NUMBER" == "the" ]; then
echo "Not an integration build…"
xcrun agvtool new-version "10.13"
else
echo "Setting integration build number: $XCS_INTEGRATION_NUMBER"
xcrun agvtool new-version "$XCS_INTEGRATION_NUMBER"
fi
Note that XCS_INTEGRATION_NUMBER exists by default in the Xcode Server build environment. If you want to simulate an integration build (for the purposes of this script), you can simply add it to your build settings as a custom variable.

IntelliJ 11 hangs when executing grails command

I am using IntelliJ 11 with Grails 2.0.0 under Ubuntu. When IntelliJ executes any grails command it hangs straight away. I am not able to migrate my project to 2.0.0 (from 1.3.7) or even create a new grails project.
No exceptions thrown in the logs, hangs after clicking on create-app using grails.
Anyone have an idea what could be the problem here?
Thanks,
For me, the freeze only occurs when starting idea from a terminal window using & (ampersand for running in background, like this:
/path/to/your/ideahome/bin/idea.sh &
IDEA then starts correctly, but as soon as any grails command is run, the process enters "stopped" state and the IDEA GUI appears to be frozen.
When doing "fg" on the process, the application wakes up again and actually runs the grails command.
So, the workaround is of course to not run IDEA in background, for example by creating a desktop icon using the following as the "command":
bash -c "export JAVA_HOME=/path/to/your/javahome;/path/to/your/ideahome/bin/idea.sh"
I have found the following to 'unstick' the process though not every time.
when Intellij hangs due to some grails command or another, I run the following to see which processes are running (I have an alias set with the name 'idea' you would use whatever the command is to run idea)
ps -ef | grep idea
There are usually 3 or 4 processes, but the first one looks like:
username 19349 14977 0 10:41 pts/1 00:00:00 /bin/sh /usr/local/bin/idea
I run the following command to kill it
kill -9 {processId}
For example:
kill -9 19349
We use -9 to force full kill the process.
Intellij begins processing as normal. Sometimes I get the prompt about whether I am sure I want to exit to which I reply no.
Sometimes it does not work and I have to kill intelliJ entirely then start over. Most times it works.
Try 11.0.2 RC from http://confluence.jetbrains.net/display/IDEADEV/IDEA+11+EAP. If it doesn't help, file a bug at http://youtrack.jetbrains.net/issues/IDEA with a thread dump attached, refer to http://www.jetbrains.net/devnet/docs/DOC-260 for details.

Resources