I am using Apache Giraph version 1.0 upon Hadoop version 0.20.203. It executes ConnectedComponentsVertex and SimpleShortetPathsVertex, examples of apache giraph, jobs successfully, but there exists a problem. After a job is finished memory is not deallocated. As System monitor shows, java processes that are created for the job are still live. I don’t understand why this problem is occurred. Is it a Giraph's bug or I am doing something wrong???
I'm using Ubuntu 11.10 and java 1.6. Any help would be appreciated.
Thanks
Yes, I believe it's a bug (see my question on the user lists: https://mail-archives.apache.org/mod_mbox/giraph-user/201403.mbox/%3COF416E2CF4.1613A751-ON86257C9F.00498FA5-86257C9F.0049D454#us.ibm.com%3E). The way I'm getting around it is by sending a kill to all the workers after a job is done:
ssh worker-name "kill -9 \$(ps aux | grep \"[j]obcache/job_[0-9]\{12\}_[0-9]\{4\}/\" | awk '{print \$2}')"
Related
I'm currently using knife zero to help me provision the servers and the commands are executed through Jenkins.
But recently I noticed that the process will stuck halfway through without any error and it is always at the part where it start executing the Java recipe.
So every time the process got stuck, I will have to reboot my system to get it running again.
The Java cookbook i'm using is from Chef supermarket.
https://supermarket.chef.io/cookbooks/java/versions/1.50.0
How should I debug this issue?
I would agree with #coderanger that this sounds like a symptom of a process that is waiting for user input before it can continue. Is it possible that you're trying to install Java 7 and haven't included the attribute accept_oracle_download_terms?
From the docs for that cookbook, this is explicitly noted as being required:
To install Oracle Java 7 (note that when installing Oracle JDK,
accept_oracle_download_terms attribute must be set
After much investigation, I found out that this is due to the lack of available ram in the system when chef is running.
I use the sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' command to clear up the caches at the start of each run and my script no longer get stuck anymore.
http://linuxinsight.com/proc_sys_vm_drop_caches.html
So I am using Ruby/Cucumber and Appium to run automated tests after each of our builds on our Jenkins server.
Essentially I have a job set up that runs our regression suite after every build that is successful. My problem comes in when my cucumber tests are running and a new build is successful.
At the moment Jenkins will start a new series of tests with the cucumber script, but won't force the old script to quit. This leads to all of my cucumber reports showing multiple false negatives. Is there a way to stipulate that Jenkins run a script that quits cucumber before starting a new set of tests?
Thanks a ton for any help you can give me. If you have any questions about how I have Jenkins set up specifically just ask and I'll give you the more intimate details.
Technically Jenkins Job will wait for 1 instance of job to complete until it starts another one. So you dont have to worry about starting another build before first one finishes...
Now coming to your question how to kill old/existing test cases if a set of new ones want to start.
I would suggest to have a cleanup script run before your actual tests are triggered, your tests would be triggering using a rake or direct cucumber command, I am guessing. So before this happens, have a cleanup.sh executed which will do the following:
1) Restart appium server.
2) Kill running ruby processes.
3) Kill running cucumber processes.
The script should look something like this:
kill -9 $(ps -ef | grep \[a]ppium | awk '{print $2}')
kill -9 $(ps -ef | grep \[r]uby | awk '{print $2}')
kill -9 $(ps -ef | grep \[c]ucumber | awk '{print $2}')
# restart appium again
appium &
Hope it helps!! Let me know in comments if you run into issues.. :)
I have calabash running iOS tests on Jenkins. When the job encounters fails I sometimes manually abort the tests by pressing the stop[x] button within the job. The problem is the next test in the feature file begins running even though I aborted. This behavior is not observed when launching the tests through the terminal. When exiting the cucumber test in the terminal the sim returns to home and no other test are launched.
I found a hook that might be useful
After do |s|
# Tell Cucumber to quit after this scenario is done - if it failed.
Cucumber.wants_to_quit = true if s.failed?
end
However, there are times when I don't want it to stop just because one scenario failed. I feel like Jenkins needs to kill all processes and its not.
If someone knows how to kill calabash and its instances manually via terminal after Jenkins has been instructed to abort, I would be interested in that too.
I tried:
ps aux | grep -i instruments | awk {'print $2'} | xargs kill -9
Unfortunately that did not work. Possibly two reasons
greping instruments shows two or more process
20272 ?? S 0:00.00 sh -c xcrun instruments -w "iPhone 5 (8.1 Simulator)...
20273 ?? S 0:00.45 /Applications/Xcode.app/Contents/Developer/usr/bin/instruments -w iPhone 5]...
Should I switch awk to print column 1?
or reason two
I'm not greping the correct process?
Here is some of my version info:
calabash-ios version: 0.11.4
Calabash::Cucumber::MIN_SERVER_VERSION: 0.11.4
Xcode 6.1
You have to let Jenkins to find all forked processes. Depending on the Job type you have to pass different environment entries into the forked process. This question is about the other way (so how to make Jenkins NOT to stop processes), but the names of the possible environment entries are there. Just pass these environment entries below to each forked process and then the process tree killer will find them:
BUILD_ID
HUDSON_SERVER_COOKIE
JENKINS_COOKIE
JENKINS_SERVER_COOKIE
HUDSON_COOKIE
I am trying to automate a test session from jenkins using Frank.
this is the error message I receive.
(Frank::Cucumber::FrankNetworkError)
./step_definitions/test_steps.rb:30:in `/^I navigate to button V\303\244der$/'
test.feature:41:in `When I navigate to <link>'
| Radar | Norrland |
*********************************************
Oh dear. Your app fell over and can't get up.
*********************************************
Jenkins checks out the code from git, besides this I have added a shell command as follows.
cd ios #<--this is so that I go to the root folder, the one with the .xcodeproj project
frank setup
frank build
frank launch
mv features/*.feature Frank/features/. #<--- this is the testscript
mv features/step_definitions/*.rb Frank/features/step_definitions/. #<--here it is moved to the newly created frank/features & Frank/features/step_definitions folder
cd Frank/features
cucumber test.feature
Everything is built the way it should and if I go to the server
and manually type the last row from my shell command will the tests be executed.
Best Regards
I have personally had many problems running Jenkins on Mac. Especially if you used the Jenskins Mac image installer, Jenkins is always run under the "jenkins" user that is created during installation.
This has given me many headaches with running cucumber from inside jobs or for starting the iOS simulator.
I have finally learned to start Jenkins under my own user, like this:
"nohup java -jar /Applications/Jenkins/jenkins.war --httpPort=8080
Since then, I was able to run cucumber without any problems. Hope this helps.
This happens every time your application crashes. When the application crashes, frank stops receiving events and cucumber ends with the error you see.
There are two possible reasons:
You have a bug in your app that made the app crash
Frank has a bug that made your app crash
You should inspect the crash/application log to see the exact reason.
I learned that is very helpful to capture the application log by the jenkins job, e.g.
function grab_log_and_fail {
APP_NAME = "MyApplication"
# get the PID of the last process with the given name
PID=$( cat /var/log/system.log | grep "$APP_NAME\[" | tail -n 1 | sed -e "s/^.*$APP_NAME\ [\([^\]*\)\].*/\1/g" )
# grab all the messages from the process with the PID
cat /var/log/system.log | grep "$APP_NAME\[$PID\]" >"$WORKSPACE/$APP_NAME.log"
#fail the job
exit 1
}
You can call it using
cucumber test.feature || grab_log_and_fail
(will grab the log only if cucumber ends with an error)
I am using IntelliJ 11 with Grails 2.0.0 under Ubuntu. When IntelliJ executes any grails command it hangs straight away. I am not able to migrate my project to 2.0.0 (from 1.3.7) or even create a new grails project.
No exceptions thrown in the logs, hangs after clicking on create-app using grails.
Anyone have an idea what could be the problem here?
Thanks,
For me, the freeze only occurs when starting idea from a terminal window using & (ampersand for running in background, like this:
/path/to/your/ideahome/bin/idea.sh &
IDEA then starts correctly, but as soon as any grails command is run, the process enters "stopped" state and the IDEA GUI appears to be frozen.
When doing "fg" on the process, the application wakes up again and actually runs the grails command.
So, the workaround is of course to not run IDEA in background, for example by creating a desktop icon using the following as the "command":
bash -c "export JAVA_HOME=/path/to/your/javahome;/path/to/your/ideahome/bin/idea.sh"
I have found the following to 'unstick' the process though not every time.
when Intellij hangs due to some grails command or another, I run the following to see which processes are running (I have an alias set with the name 'idea' you would use whatever the command is to run idea)
ps -ef | grep idea
There are usually 3 or 4 processes, but the first one looks like:
username 19349 14977 0 10:41 pts/1 00:00:00 /bin/sh /usr/local/bin/idea
I run the following command to kill it
kill -9 {processId}
For example:
kill -9 19349
We use -9 to force full kill the process.
Intellij begins processing as normal. Sometimes I get the prompt about whether I am sure I want to exit to which I reply no.
Sometimes it does not work and I have to kill intelliJ entirely then start over. Most times it works.
Try 11.0.2 RC from http://confluence.jetbrains.net/display/IDEADEV/IDEA+11+EAP. If it doesn't help, file a bug at http://youtrack.jetbrains.net/issues/IDEA with a thread dump attached, refer to http://www.jetbrains.net/devnet/docs/DOC-260 for details.