I have some doubts about the correct configuration of Jenkins to ensure the continuous integration of a Drupal project but I arrive at some contradictions.
Let me explain: the deployment, after all, consists in executing:
cd / path / to / web / root
pull from git
drush config:import
drush cache:rebuild
The tests are launched with the command
../vendor/bin/phpunit --verbose --log-junit ../tests_output/phpunit.xml -c ../phpunit.xml
The contradiction is that I do not understand when to run the tests.
Before the pull does not make sense because the last changes are missing, after the pull if any test goes wrong I should be able to restore the situation before the pull (but I'm not sure it's a safe action).
I'm trying to run the tests directly in the workspace of jenkins and to do this I also created a separate database, but at the moment I get the error:
Drupal\Tests\field_example\Functional\TextWidgetTest::testSingleValueField
Behat\Mink\Exception\ElementNotFoundException: Button with id|name|label|value "Log in" not found
What could be the best strategy to follow?
So, your order seems ok - pull first then run tests.
However, you can have 2 Jenkins jobs. First runs, your tasks. 2nd runs ONLY if your first job completes without failure.
There are ways to get exit status from scripts - see following plugins/notes about that.
How to mark Jenkins builds as SUCCESS only on specific error exit values (other than 0)?
How to mark a build unstable in Jenkins when running shell scripts
Related
I am trying to run my protractor tests in Jenkins and its working. The problem is, running the tests on a docker container and use multiple headless browsers doesnt work, because my test is using actions like hovering an element. My idea was to use multiple test suites:
local: not headless, for presentations
external: headless (test cases without actions like hovering an element), Jenkins
I added the suites in the protractor.conf.js file.
Typing
protractor protractor.conf.js --suite local using the terminal works fine and as expected, but in my Jenkinsfile.feature it says
npm run e2e..., which runs also the testsuite I dont want to be executed.
So my Jenkins tests fail, just because the wrong testsuite is executed.
Replacing npm run e2e... with
protractor protractor.conf.js --suite local in the Jenkinsfile.feature doesnt work neither. I hope there is a way to tell Jenkins what suite to run. Thank you
Even though your question sounds like 'how can one develop a website' I'll try to walk you through the steps. But your question it too broad especially given that you didn't provide your error stack and the content of package.json, config file, docker file and jenkins job you're running
You gotta break your task into multiple layers (and this relates to any parameter you want, headless or multiple suites) and resolve them one by one
Make sure your protractor can take this parameter, by passing from the CLI. For example
protractor protractor.conf.js --suite local
But don't forget to add default value if nothing is passed
Once 1 is tested and works, go to the next layer, which should be docker in your case. Open Dockerfile where you declare the image, and add environment variables that you will be passing to protractor. Define the command to run your script once a container is spun up from your image
When you have your docker image working as expected, find out how to pass variables to that image when spinning up a container. Normally it should be like following
docker run -e SUITE=“regression” protractor_image
When you have the image in Jenkins master or slave, and you can run tests from CLI, you can work on your job. Again, depends on what you're doing pipeline job or regular freelance job, your steps maybe different. But the logic remains the same. You need to add a job, and make it run tests by hardcoding parameters
When the job works, add input parameters and ensure that can be used
I'm sure you'll have questions about each item or the list, so I'd suggest to open a new question for each and provide more details for them. Good luck
I'm using maven to deploy my jobs on Google Cloud Dataflow, with the folowing command :
mvn compile exec:java -Dexec.mainClass=org.beam.StreamerRunner --Dexec.args="\
...
--runner=DataflowRunner \
..."
It deploys successfully, and it is pulling the log from the dataflow job and printing them on the output. I'm wondering if it is possible to tell the deployment to not pull and just returns.
Indeed, my CI tool (TeamCity) I'm using to deploy my job, is also waiting never ending.
I obviously can run the maven command in a nohup, but maybe an option does exist to exit the command after the deploy is complete.
As Alex pointed out I was calling waitUntilFinish in my code, so it dit exactly what I asked it to do.
It was fixed as soon as I removed the calle to
waitUntilFinish()
I'm beginning with Jenkins.
I want, that each time I do a git commit (or push?), that the jasmine test of my ionic project was executed and must work before the commit can be done.
In reality, it has 2 questions:
How execute jasmine test with Jenkins?
In this moment I execute the test with:
npm test
How can I do for executing this tests with a commit (or a push)?
Thanks
Best regards
You have two ways to achieve the task.
GIT Hook: From GIT after commit or push execute the Jasmine test
Jenkins Trigger with GIT Hook: From Jenkins check the repo and execute the Jasmine test
Hooks from GIT
Look for the hidden directory in your git repo, you'll find a directory called "hooks" and inside it many examples of hooks:
First list the content of your repo main directory:
ls -ltra
You should see something like:
m.ortiz.montealegre#CPX-XYR3G1DTHBU ~/-argentina/.git
$ vim hooks/
applypatch-msg.sample pre-applypatch.sample pre-push.sample update.sample
commit-msg.sample pre-commit.sample pre-rebase.sample
post-update.sample prepare-commit-msg.sample pre-receive.sample
You have a whole guide of how to setup hooks here.
In your case maybe update would do the thing:
update The update script is very similar to the pre-receive script,
except that it’s run once for each branch the pusher is trying to
update.
Triggers from Jenkins with GIT Hooks
In this one you'll setup your Jenkins project build trigger with "Poll SCM" but do not specify a schedule.
Then with a post-receive hook from GIT notify the Jenkins Job about the changes:
http://yourserver/jenkins/git/notifyCommit?url=<URL of the Git repository>?token=<get token from git to build remotely>
I found that example here.
Run the Jasmine tests
I don't know which O.S you're using but I hope it's a beautiful Linux box.
You can achieve pretty much the same with Jenkins. You need to consider the user (your user) and its permissions and check if the user which runs the Jenkins instance is allowed to execute the same.
Just create a new Jenkins Project and add a shell execution step with the test just like you said:
npm test
There are many questions regarding your particular environment, but I think that this will be a good guide for you.
There was a ticket about adding this functionality.
Finally the ticket was closed
slackersoft commented on 2 Dec 2016
At this point, I think it makes more sense to leave the code to do the watching of your specs and production code to one of the many external libraries that are built specifically for that.
The relate external library can be:
jasmine-node
nodemon
mochajs
gaze
Inside the .travis.yml configuration file what is the practical difference between before_install, install, before_script and script options?
I have found no documentation explaining the differences between these options.
You don't need to use these sections, but if you do, you communicate the intent of what you're doing:
before_install:
# execute all of the commands which need to be executed
# before installing dependencies
- composer self-update
- composer validate
install:
# install all of the dependencies you need here
- composer install --prefer-dist
before_script:
# execute all of the commands which need to be executed
# before running actual tests
- mysql -u root -e 'CREATE DATABASE test'
- bin/doctrine-migrations migrations:migrate
script:
# execute all of the commands which
# should make the build pass or fail
- vendor/bin/phpunit
- vendor/bin/php-cs-fixer fix --verbose --diff --dry-run
See, for example, https://github.com/localheinz/composer-normalize/blob/0.8.0/.travis.yml.
The difference is in the state of the job when something goes wrong.
Git 2.17 (Q2 2018) illustrates that in commit 3c93b82 (08 Jan 2018) by SZEDER Gábor (szeder).
(Merged by Junio C Hamano -- gitster -- in commit c710d18, 08 Mar 2018)
That illustrates the practical difference between before_install, install, before_script and script options
travis-ci: build Git during the 'script' phase
Ever since we started building and testing Git on Travis CI (522354d: Add Travis CI support, 2015-11-27, Git v2.7.0-rc0), we build Git in the
'before_script' phase and run the test suite in the 'script' phase
(except in the later introduced 32 bit Linux and Windows build jobs,
where we build in the 'script' phase').
Contrarily, the Travis CI practice is to build and test in the
'script' phase; indeed Travis CI's default build command for the
'script' phase of C/C++ projects is:
./configure && make && make test
The reason why Travis CI does it this way and why it's a better
approach than ours lies in how unsuccessful build jobs are
categorized. After something went wrong in a build job, its state can
be:
'failed', if a command in the 'script' phase returned an error.
This is indicated by a red 'X' on the Travis CI web interface.
'errored', if a command in the 'before_install', 'install', or
'before_script' phase returned an error, or the build job exceeded
the time limit.
This is shown as a red '!' on the web interface.
This makes it easier, both for humans looking at the Travis CI web
interface and for automated tools querying the Travis CI API, to
decide when an unsuccessful build is our responsibility requiring
human attention, i.e. when a build job 'failed' because of a compiler
error or a test failure, and when it's caused by something beyond our
control and might be fixed by restarting the build job, e.g. when a
build job 'errored' because a dependency couldn't be installed due to
a temporary network error or because the OSX build job exceeded its
time limit.
The drawback of building Git in the 'before_script' phase is that one
has to check the trace log of all 'errored' build jobs, too, to see
what caused the error, as it might have been caused by a compiler
error.
This requires additional clicks and page loads on the web interface and additional complexity and API requests in automated tools.
Therefore, move building Git from the 'before_script' phase to the
'script' phase, updating the script's name accordingly as well.
'ci/run-builds.sh' now becomes basically empty, remove it.
Several of our build job configurations override our default 'before_script' to do nothing; with this change our default 'before_script' won't do
anything, either, so remove those overriding directives as well.
Here is something I want to achieve:
I have a jenkins project which has 4 upstream projects. But I don't want to trigger this project when the upstream jobs are done building, but I want the trigger the project via remote API, which then waits on upstream projects until they are done building, if these projects are building.
Lets say all the 4 upstream projects can build the source code from any branch passed via API, but I want the downstream project to start only when a specific branch is passed to these upstream projects.
Scenario:
Lets say I have two clusters A and B, for the sake of this question, I want to deploy my code to cluster A, i.e front end and backend code. Now I have a project to build front end and 1 project to build backend (these two projects can build code for cluster A and B, based on the branch passed). Now, I have two deploy projects for cluster A which will deploy front end and backend. So, when I pass a branch to build code for cluster A, it will trigger the build projects. But now I only want these two deploy projects to start when this specific branch was passed.
If you want to control the builds remotely then use the Jerkins cli - I have found it very useful http://jenkinshost:8080/cli
You need to get the ssh key config right, add the public key of the user running the cli to the user you want to run the job in Jenkins using the Jenkins user configuration (not on the command line
Test key setup with
java -jar jenkins=cli.jar -s http://jenkinshost:8080 who-am-i
This should then report which user will be used to run the build in Jenkins
But I think you can use the Conditional Build Step plugin for your problem
https://wiki.jenkins-ci.org/display/JENKINS/Conditional+BuildStep+Plugin
This will allow you to put a conditional wrapper around a build step i.e.
if branch==branchA then
trigger step - deploy to clusterA
if branch==branchB then
trigger step - deploy to clusterB
Personally I find this plugin a bit clunky and it makes the job config page a little messy
Another solution I came up with was to always call the child job and then let it decide if it runs.
So I have a script step at the start of the child job to see if it should run
if [${branch}="Not the right branch name" ] ; then
echo "EXIT_GREEN"
exit 1
fi
You have now failed this job which would cause the parent job to go red but by using the Groovy Postbuild plugin https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Postbuild+Plugin you can add a post build step like this
if (manager.logContains(".*EXIT_GREEN.*")) {
manager.addBadge("info.gif","This job had nothing to do")
manager.build.#result = hudson.model.Result.SUCCESS
}
Child job has run green (with an info icon against the build) but has actually not done anything. Obviously if the branch is one you want deploy then the first script step does not run the exit 1 and the job continues as normal