I'm using a self-hosted GitLab CI server (community edition v8.9.5) and gitlab-ci-multi-runner 1.2.0 to build a project. One of my pipeline stages (test) takes a while to run and I get the following erm:
ERROR: Build failed: execution took longer than 3600 seconds
Where do I put the override for this timeout? Can I apply it to just the test pipeline stage?
You can set a global timeout in "Project settings -> CI/CD Pipelines -> Timeout" or "Project settings -> Builds -> Timeout" in older versions.
As of version 12.3, you can set a timeout per stage in your CI .yml file using timeout:
timeout allows you to configure a timeout for a specific job. For example:
build:
script: build.sh
timeout: 3 hours 30 minutes
test:
script: rspec
timeout: 3h 30m
The job-level timeout can exceed the project-level timeout but can’t exceed the Runner-specific timeout.
There are two timeout can be set: project timeout and runner timeout.
Project timeout:
It is possible to set timeout per job from Settings -> CI/CD -> General pipelines
Runner timeout:
runner's timeout can be set from Settings -> CI/CD -> Runners, select the runner from Runners activated for this project and edit the Maximum job timeout from the runner edit form.
Be aware that these two types of timeout can be overridden by each other. Refer the docs.
A job-specific timeout was introduced in Gitlab 12.3
https://docs.gitlab.com/ce/ci/yaml/README.html#timeout
build:
script: build.sh
timeout: 3 hours 30 minutes
test:
script: rspec
timeout: 3h 30m
If running it at Gitlab, then the above stated answers has already told the right way. But if you would like to run the Gitlab CI locally that too with tour own runner, then runners timeout you need to pass with command.
gitlab-runner exec docker job_name --timeout 3600
here it is setting runners timeout as 3600 seconds
Related
Can you please help, I have the following scenario and I went through many videos, blogs but could not find anything matching with my use-case
Requirement:
To write a CI\CD pipeline in GitLab, which can facilitate the following stages in this order
- verify # unit test, sonarqube, pages
- build # package
- publish # copy artifact in repository
- deploy # Deploy artifact on runtime in an test environment
- integration # run postman\integration tests
All other stages are fine and working but for the deploy stage, because of a few restrictions I have to submit an existing Jenkins job using Jenkin remote API with the following script but the problem that script returns an asynchronous response and start the Jenkins job and deploy stage completes and it moves to next stage (integration).
Run Jenkins Job:
image: maven:3-jdk-8
tags:
- java
environment: development
stage: deploy
script:
- artifact_no=$(grep -m1 '<version>' pom.xml | grep -oP '(?<=>).*(?=<)')
- curl -X POST http://myhost:8081/job/fpp/view/categorized/job/fpp_PREP_party/build --user mkumar:1121053c6b6d19bf0b3c1d6ab604f22867 --data-urlencode json="{\"parameter\":[{\"name\":\"app_version\",\"value\":\"$artifact_no\"}]}"
Note: Using GitLab CE edition and Jenkins CI project service is not available.
I am looking for a possible way of triggering the Jenkins job from the pipeline and only on successful completion of the Jenkins job my integration stage starts executing.
Thanks for the help!
Retrieving the status of a Jenkins job that is triggered programmatically through the remote access API is notorious for not being quite convoluted.
Normally you would expect to receive in the response header, under the Location attribute, a url that you can poll to get the status of your request, but unfortunately there are some in-between steps to reach that point. You can find a guide in this post. You may also have a look in this older post.
Once you have the url, you can pool and parse the status job and either sh "exit 1" or sh "exit 0" in your script to force the job that is invoking the external job to fail or succeed, depending on how you want to assert the result of the remote job
I'm using the circleCI CLI locally to test my .circleci/config.yml. This is what it looks like:
version: 2.1
jobs:
test:
docker:
- image: circleci/node:4.8.2
steps:
- checkout
- run: echo 'test step'
workflows:
version: 2
workflow:
jobs:
- test
This fails with the following error:
* Cannot find a job named build to run in the jobs: section of your configuration file.
If you expected a workflow to run, check your config contains a top-level key called 'workflows:'
The 'hello world' workflow from the CLI docs works fine.
What am I missing here?
In the same CircleCI CLI documentation mentioned above it has in the 'limitations' section:
The CLI tool does not provide support for running workflows. By nature, workflows leverage running jobs concurrently on multiple machines allowing you to achieve faster, more complex builds. Because the CLI is only running on your machine, it can only run single jobs (which make up parts of a workflow).
So I guess running workflows with orbs works (as in the 'hello world' example), but running workflows with your own jobs does not work with the CLI.
Testing Jobs Locally
If you're looking to test your config locally like I was, you can still execute your individual jobs locally. In the same documentation linked above, under the title 'Running a Job' when using config with version 2.1+ you can explicitly call one of your jobs like so:
circleci config process .circleci/config.yml > process.yml
circleci local execute -c process.yml --job JOB_NAME
I use CircleCI and the pipeline is as follows:
build
test
build app & nginx Docker images and push them to a GitLab registry
deploy Docker stack to the development server (currently the Swarm manager)
I just pushed my develop branch to my repository and faced a "Symfony4 new Controller page" on the development server after a successful message from CircleCI.
I logged via SSH in it and executed (with output for the application service):
docker stack ps my-development-stack --format "{{.Name}} {{.Image}} {{.CurrentState}}"
my-stack_app.1 gitlab-image:latest-develop Running 33 minutes ago
On my GitLab repository's registry, the application image has been "Last Updated" 41 minutes ago. The service's image has apparently been refreshed before with the last version.
Is it a common issue/error ?
How could (or should) I fix this timing issue ?
Can CircleCI help about this ?
Perhaps it is best ( though not ideal ) to introduce a delay between build and deploy , you can refer to this example here CircelCI Delay Between Jobs
I found a workaround using a CircleCI scheduled workflow triggered by a CRON. I scheduled a nightly build workflow which will run every day at midnight.
Sample of my config.yml file
# Beginning of the config.yml
# ...
workflows:
version: 2
# Push workflow
# ...
# Nightly build workflow
nightly-dev-deploy:
triggers:
- schedule:
cron: "0 0 * * *"
filters:
branches:
only:
- develop
jobs:
- build
- test:
requires:
- build
- deploy-dev:
requires:
- test
Read more about scheduled workflow with a nightly build example in the CircleCI official documentation
Looks more like a workaround to me. I'd be glad to hear how do you avoid this issue, which could lead to a better answer to the question.
I have jenkins deployment process that uses Ansible playbook for deployment, sometimes that job runs on 100 hosts.
I have a downstream job that runs on a successful run of the deployment job.
If 1 host is failed for some reason, it fails all the deployment and the downstream is not running, I want to mark the deployment job as a successful run even if 10% of the hosts are failed.
How can I achieve that?
Thanks.
You can use, as the documenatation of the meta module states
- meta: clear_host_errors
as the last task in your playbook. This will clear all errors from failed hosts. Additionally you can deal with the Maximum Failure Percentage.
max_fail_percentage: 10
Put it all together in a play:
- hosts: my_host_group
max_fail_percentage: 10
tasks:
- name: "Do something"
command: "uname -a"
- name: "Do something other"
command: "id -a"
- meta: clear_host_errors
I have a jenkins jobs that executes with 'Publish over SSH'. The job connects to the remote server, transfers files and runs and ansible playbook.
The playbook runs as intended, confirmed by the logs. However at the end of the job an error is returned, failing the job. It's causing problems as it's preventing the pipeline from working correctly.
SSH: EXEC: completed after 402,593 ms
SSH: Disconnecting configuration [server] ...
ERROR: Exception when publishing, exception message [Exec exit status not zero. Status [2]]
Build step 'Send files or execute commands over SSH' changed build result to UNSTABLE
[Run Playbook] $ /bin/sh -xe /tmp/jenkins1528195779014969962.sh
+ echo Finished
Finished
Finished: UNSTABLE
Is there a setting missing to allow this to pass?
never used the 'Publish over SSH' you are referingto, but I can recommend Jenkins Ansible Plugin. I am running several playbooks in pipeline stages here successfully from labeled build slaves (have one dedicated slave that has Ansible installed) targeting Linux hosts on cloud infrastructure via SSH.
Especially in combination with the ANSI color plugin the output very readable.
If you cannot try that plugin, check whats the return code of the playbook run shell call.