I have a following GitLab CI/CD job defined in my gitlab-ci.yml:
image: docker:stable
stages:
- ...
...
build-dependencies-init:
stage: checks
tags:
- ...
needs:
- check-registry
script:
#- wget --spider --server-response http://$DOCKER_REGISTRY_URL/v2/repo/image/manifests/latest 2>&1 | grep -m 1 "HTTP/" | awk '{print $2}'
- STATUS=$(wget --spider --server-response http://$DOCKER_REGISTRY_URL/v2/repo/image/manifests/latest 2>&1 | grep -m 1 "HTTP/" | awk '{print $2}')
- if [ $STATUS -eq 200 ]; then
- echo "Image is available"
- else
- echo "Building dependencies"
- docker build -t $DOCKER_REGISTRY_URL/repo/image:latest .
- echo "Pushing dependencies"
- docker push $DOCKER_REGISTRY_URL/repo/image:latest
- fi
I would expect, that it would store the status code into the variable STATUS which I can then use to decide if building new image is necessary. I dont want to build new image every time.
The problem is that when this thing fails (STATUS stores 404 for example) the runner decides to stop execution and starts cleaning up... it does not evaluate the if/then/else code that follows.
Why this happens? How do I prevent GitlabRunner from stopping early? The commented out code prints out 404 as it should (if uncommented), but i do not expect it to terminate at this point in execution.
I am using containerised GitlabRunner in Docker, with a mounted docker.sock.
Related
I am trying to implement a script in gitlab CI to trigger a smoke test via Jenkins and then get the results.
So far I am able to trigger the job successfully and I am trying to follow this to implement a monitoring stage and then get the result once the job finishes.
my issue is that I implemented a while loop to monitor if the Jenkins job has finished so far the script is giving either syntax errors (when copied in gitlab) or if run in the terminal I get:
job is building? true
waiting...
parse error: Invalid numeric literal at line 2, column 0
job is still building?
job is building?
This is what I am using so far:
#!/bin/bash
running="true"
while [ "$running" != "false" ]
do
echo "job is building? ${running}"
echo "waiting...";
sleep 2;
running=$(curl -s --user ${EMAIL}:${TOKEN} ${URL}/${var}/lastBuild/api/json | jq .'building')
echo "job is still building? ${running}"
done
echo "Done!"
buildNumber=$(curl -s --user $EMAIL:$TOKEN ${URL}/$ENV-${var}/lastBuild/api/json | jq ".url" | awk -F "/" '{print $(NF-1)}')
echo "getting results for build ${buildNumber}"
curl -s --user ${EMAIL}:${TOKEN} ${URL}/${ENV}-${var}/lastBuild/api/json | jq ".url" | awk -F "/" '{print $(NF-1)}'
curl -v --silent --user ${EMAIL}:${TOKEN} ${URL}/${ENV}-${var}/lastBuild/consoleText 2>&1 | grep -i "finished:"
UPDATE
the script is running now ok in my local terminal
the change was
running="true"
while [ "$running" != "false" ]
do
echo "job is building? ${running}"
echo "waiting..."
sleep 2
curl -s --user $EMAIL:$TOKEN $URL/$ENV-${var}/lastBuild/api/json --output now.txt
running=$(jq .'building' now.txt)
echo "job is still building? ${running}"
done
The problem is still on Gitlab CI as after copy pasting this script I get in the pipeline
/bin/sh: eval: line 149: syntax error: unexpected "done"
I'm guessing that the error
parse error: Invalid numeric literal at line 2, column 0
comes from the line that does the curl
running=$(curl -s --user ${EMAIL}:${TOKEN} ${URL}/${var}/lastBuild/api/json | jq .'building')`
I'm assuming that the error is thrown by jq when it tries to parse the json and fails. Since this line has an error, the variable running never gets properly updated, meaning that the rest of the script doesn't work as intended.
If you fix this line, the rest of your pipeline should work. Consider looking into this question which has a similar problem and some solutions.
My question is similar to this post where the Docker build stage is failing in Codebuild. However, the buildspec.yml file is a bit different since I'm including additional credentials. Here is my buildspec file:
version: 0.2
phases:
pre_build:
commands:
- echo Logging in to Amazon ECR...
- aws --version
- aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin XXXXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com
- REPOSITORY_URI=XXXXXXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/wp-api
- COMMIT_HASH=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
- IMAGE_TAG=build-$(echo $CODEBUILD_BUILD_ID | awk -F":" '{print $2}')
build:
commands:
- echo Build started on `date`
- echo Building the Docker image...
- docker build -t app:latest --secret id=awscredentials,src=$HOME/.aws/credentials .
- docker tag app:latest REPOSITORY_URI:latest
post_build:
commands:
- echo Build completed on `date`
- echo Pushing the Docker images...
- docker push $REPOSITORY_URI:latest
The error I get is:
[Container] 2022/07/11 18:37:24 Command did not exit successfully docker build -t app:latest --secret id=awscredentials,src=$HOME/.aws/credentials . exit status 1
[Container] 2022/07/11 18:37:24 Phase complete: BUILD State: FAILED
[Container] 2022/07/11 18:37:24 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: docker build -t app:latest --secret id=awscredentials,src=$HOME/.aws/credentials .. Reason: exit status 1
[Container] 2022/07/11 18:37:24 Entering phase POST_BUILD
[Container] 2022/07/11 18:37:24 Running command echo Build completed on `date`
Build completed on Mon Jul 11 18:37:24 UTC 2022
Am I missing something? Note that a more basic buildfile runs fine in the same project environment (one that does not have extra credentials or anything).
I am running script in my gitlab ci:
pre_build:
stage: get_info
script:
- printenv
- cd ./scripts
- ./pre_build.sh
this script has the following line:
todays_date=$(date +"%d-%b-%Y")
latest=$(curl -i ${artifactory_url} | grep develop | grep -i ${todays_date} | grep ${last_ok_build_number} | grep -Eoi '<a [^>]+>' | grep -Eo 'href="[^\"]+"')
I believe this is failing due to regex in grep, any idea how to run this in GitLab ci?
Above script works perfectly well on my local machine
EDIT: Above codes works it was timezone issue
I've just started setting up a Github-actions workflow for one of project.I attempted to run the workflow steps inside a container with this workflow definition:
name: TMT-Charts-CI
on:
push:
branches:
- master
- actions-ci
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://alpine/helm:2.13.0
steps:
- name: Checkout Code
uses: actions/checkout#v1
- name: Validate and Upload Chart to Chart Museum
run: |
echo "Hello, world!"
export PAGER=$(git diff-tree --no-commit-id --name-only -r HEAD)
echo "Changed Components are => $PAGER"
export COMPONENT="NOTSET"
for CHANGE in $PAGER; do ENV_DIR=${CHANGE%%/*}; done
for CHANGE in $PAGER; do if [[ "$CHANGE" != .* ]] && [[ "$ENV_DIR" == "${CHANGE%%/*}" ]]; then export COMPONENT="$CHANGE"; elif [[ "$CHANGE" == .* ]]; then echo "Not a Valid Dir for Helm Chart" ; else echo "Only one component per PR should be changed" && exit 1; fi; done
if [ "$COMPONENT" == "NOTSET" ]; then echo "No component is changed!" && exit 1; fi
echo "Initializing Component => $COMPONENT"
echo $COMPONENT | cut -f1 -d"/"
export COMPONENT_DIR="${COMPONENT%%/*}"
echo "Changed Dir => $COMPONENT_DIR"
cd $COMPONENT_DIR
echo "Install Helm and Upload Chart If Exists"
curl -L https://git.io/get_helm.sh | bash
helm init --client-only
But Workflow fails stating the container stopped due immediately.
I have tried many images including "alpine:3.8" image described in official documentation, but container stops.
According to Workflow syntax for GitHub Actions, in the Container section: "A container to run any steps in a job that don't already specify a container." My assumption is that the container would be started and the steps would be run inside the Docker container.
We can achieve this my making custom docker images, Actually Github runners somehow stops the running container after executing the entrypoint command, I made docker image with entrypoint the make container alive, so container doesn't die after start.
Here is the custom Dockerfile (https://github.com/rizwan937/Helm-Image)
You can publish this image to dockerhub and use it in workflow file like
container:
image: docker://rizwan937/helm
You can add this entrypoint to any docker image so that It remains alive for further steps execution.
This is a temporary solution, if anyone have better one, let me know.
I'm doing the productive deploy in gitlab manually. I'm using docker container.
Clicking on the 'Play'-Button in the pipeline list should do the deploy.
But how do I get the version of the selected container? Doing this script is always trying to pull the latest version, which should not be. I want to pull the 'selected' container.
deploy_prod:
stage: deploy
script:
- docker pull $CI_REGISTRY_IMAGE # here selected version is missing
# ...
when: manual
environment:
name: productive
url: https://example.com
only:
- master
As mentioned in the comments to your question, simply use the same script you used to push the image, to pull it in the deploy stage.
Here's an example pull.sh script:
#!/bin/sh
args=("$#")
CI_REGISTRY_IMAGE=${args[0]}
PACKAGE_VERSION=$(cat package.json \
| grep version \
| head -1 \
| awk -F: '{ print $2 }' \
| sed 's/[",]//g' \
| tr -d '[[:space:]]')
CONTAINER_RELEASE_IMAGE=$CI_REGISTRY_IMAGE\:$PACKAGE_VERSION
docker pull $CONTAINER_RELEASE_IMAGE
Notice the pull instead of the push in the last line.
Then modify your deploy job like this:
deploy_prod:
stage: deploy
script:
- ./pull.sh $CI_REGISTRY_IMAGE
# ...
when: manual
environment:
name: productive
url: https://example.com
only:
- master