I am new to Azure Devops and am having some difficulty building my first pipeline. So far I have three steps that work just fine:
Maven build from POM, successfully packages my war file
Copy files to $(system.defaultworkingdirectory), copying the files I want from the target folder
Successful Publish of the artifact to a private Azure package repository
My 4th step runs a DevOps Docker Task to build a Docker image to be used to deploy the web app. This has been a challenge because my dockerfile COPY commands are failing. I can't locate the target folder, the one that step 3 just used to build the war file! In an effort to locate the target folder I added this command to my dockerfile:
RUN ls -R -la /
It appears to have dumped the entire file system and the target folder is nowhere to be found in the listing.
Any thoughts regarding where I can find my target files?
I am very close to making this work the way I want. If I comment out the COPY command it builds a fundamentally empty image which my 5th step successfully pushes to my private Docker repository. Of course the image is useless without the web app.
Any help you might offer will be greatly appreciated.
After a lot of trial and error I came up with the following azure-pipelines.yml file:
trigger:
- master
jobs:
- job: build
pool:
vmImage: 'Ubuntu-16.04'
steps:
- script: |
echo Starting the build
env
java -version
./mvnw clean package -Dmaven.test.failure.ignore=true -e -U
ls -la *
displayName: 'Build with Maven'
- task: Docker#0
displayName: 'Build an image'
inputs:
azureSubscription: 'Visual Studio Enterprise (******)'
azureContainerRegistry: '{"loginServer":"testingcontainerregistry******.azurecr.io", "id" : "/subscriptions/******/resourceGroups/******/providers/Microsoft.ContainerRegistry/registries/testingContainerRegistry******"}'
action: 'Build an image'
- task: Docker#0
displayName: 'Push an image'
inputs:
azureSubscription: 'Visual Studio Enterprise (******)'
azureContainerRegistry: '{"loginServer":"testingcontainerregistry******.azurecr.io", "id" : "/subscriptions/******/resourceGroups/******/providers/Microsoft.ContainerRegistry/registries/testingContainerRegistry******"}'
action: 'Push an image'
- job: test
dependsOn: build
condition: succeeded()
pool:
vmImage: 'Ubuntu-16.04'
steps:
- script: |
echo Performing tests
env
ls -la
displayName: 'Running integration tests'
The testing job is not doing anything useful yet, but you can see that the Maven build and the Docker build&push are done by the same job.
Essentially I struggled with the same issue as you did. I created a GitHub ticket to make them aware of the fact that the basic concepts are not easy to understand from the current documentation: https://github.com/MicrosoftDocs/vsts-docs/issues/2851
Related
I have build the ios-x86-64 build using the azure pipeline on the mac vmimage.
Now i want to run the unit test on that using Ctest.
I have given the following command in my .yml file
steps:
- task: CmdLine#2
displayName: 'Run tests ${{parameters.platform}}-$(Configuration)'
inputs:
script: 'ctest -j14 -C $(Configuration) -T test --output-on-failure'
workingDirectory: '$(Build.SourcesDirectory)/out/build/${{parameters.platform}}-$(Configuration)'
After running the command my build is getting generated but on running above script i am getting the following error
Could not find executable /Users/runner/azure-pipelines-agent/_work/3/s/out/build/iOS-x86_64-Release/cpp/src/Microsoft.AugLoop.Client/tests/Release${EFFECTIVE_PLATFORM_NAME}/Microsoft.AugLoop.Client.Tests.app/Microsoft.AugLoop.Client.Tests
2021-09-20T19:33:32.6663830Z Looked in the following places:
2021-09-20T19:33:32.6665690Z
I have created a React app and setup a fully working pipeline in Azure DevOps using Docker to build, test and publish my application.
Snippet from my pipeline yaml file:
#----------------------------------------------------------------------
# Build the docker image
#----------------------------------------------------------------------
- task: Docker#2
inputs:
containerRegistry: 'XXXcontainerrepo'
repository: 'XXX'
command: 'build'
Dockerfile: '**/production.Dockerfile'
tags: |
$(Build.BuildNumber)
arguments: '--build-arg buildNumber="$(Build.BuildNumber)"'
displayName: 'Docker build'
#----------------------------------------------------------------------
# Export and publish test results
#----------------------------------------------------------------------
- script: |
export id=$(docker images --filter "label=test=$(Build.BuildNumber)" -q | head -1)
echo "Container ID: ${id}"
docker create --name testcontainer $id
docker cp testcontainer:/app/coverage ./coverage
docker cp testcontainer:/app/junit.xml ./junit.xml
docker rm testcontainer
displayName: 'Copy test results and code coverage reports'
- task: PublishCodeCoverageResults#1
inputs:
codeCoverageTool: 'cobertura'
summaryFileLocation: '$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml'
displayName: 'Publish code coverage reports'
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/junit.xml'
mergeTestResults: true
failTaskOnFailedTests: true
testRunTitle: 'Jest Unit Tests'
displayName: 'Publish test results'
Snippet from the Docker file:
# run tests
LABEL test=$buildNumber
RUN npm run test -- --reporters=default --reporters=jest-junit --coverage --watchAll=false
When all the test cases pass a test report is generated which I then copy out from my Docker container and publish using a task in the Azure pipeline.
But when a test fails I get an error in the Docker container and my build fails without any report being created. Of course I can see the failing test in the pipeline build log, but I don't get the report available in Azure DevOps since it is never created and thus not available to publish.
When I run the same command locally I get a test report created in both cases, when all test pass or when one or more fail.
So my question is, is it possible to create the report even if one or more test fail using Docker? As I understand the number of failing tests is the exit code, and when the exit code != 0 the Docker container fails.
I would still like the Docker build step to fail after the report has been created, but in case that is not possible I have set a flag in the test result publish task to fail the build if one or more test fail.
failTaskOnFailedTests: true
Update
When adding the continueOnError: true flag to the Docker build step the build continues to the next step, but the problem still exists. It seems like the test run is halted before it can create the test and coverage reports, probably due to the non zero exit code it produces that makes the Docker step exit.
In the copy test results step I then get this output:
Error: No such container:path: testcontainer:/app/coverage
Error: No such container:path: testcontainer:/app/junit.xml
Which tells me the test run didn't create a report due to the non zero exit code which stopped the Docker build step to exit.
Solution
I ended up delaying the exit code so that the container with the test label finished and thus was available to the next step to extract the report files.
RUN npm run test -- --reporters=default --reporters=jest-junit --coverage --
watchAll=false; \
echo $? > /npm.exitcode;
# if the npm command failed - fail the build here so that we can get the
test-report files regardless of exit code
RUN exit $(cat /npm.exitcode)
I also added the condition: succeededOrFailed() to the copy test results step and the publish code coverage step so that they always run, even if the build step "fails".
You may try the workaround in the following links:
https://github.com/MicrosoftDocs/azure-devops-docs/issues/2183
https://github.com/microsoft/vstest/issues/1848
Try adding "exit 0" to the end of this line as this: RUN dotnet test MyProject.Tests.csproj -c Release --logger "trx;LogFileName=testresults.trx"; exit 0
You can set continueOnError field on docker step which enable next steps to run. However, in this way your build ends with status SucceededWithIssues. If this is ok for you, all is done. If not you can put at the end step with condition on Agent.JobStatus like
- pwsh: exit 1
condition eq(variables['Agent.JobStatus'], 'SucceededWithIssues')
to fail a build when it happens.
I'm trying to run my build inside a container with azure-pipelines in Azure DevOps Server(onprem). Following the official guide https://learn.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops-2019
I do have a self-hosted linux agent with ubuntu18.04 installed.
My azure-pipelines.yml
pool: linux-container-build
container: ubuntu:16.04
steps:
- script: whoami
The container initialization works fine and creates the container properly. Afterwards the checkout steps fails without much information.
Picture of pipeline: pipeline
Checkout step just does this:
##[section]Starting: Checkout ***** to s
==============================================================================
Task : Get sources
Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
Version : 1.0.0
Author : Microsoft
Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)
==============================================================================
##[error]Collection was modified; enumeration operation may not execute.
##[section]Finishing: Checkout **** to s
I updated my task definition to:
- checkout: none
This will skip the checkout step and the 'whoami' step succeeds with proper output inside the container
It seems I need git inside my container? ..also probably all other packages..
Can I somehow add git and all required applications to the _work folder or to externels because this will get mounted in the docker volume?
Circle CI does these and more, before it gets to your actual Java/Ruby/Python/etc build:
Starting the build
Start container
Enable SSH
Is there a way of inserting a custom command at that early stage? Circle.yml seems to be where you would specify things like that, but the documentation doesn't detail things that can be inserted into the earlier stages/steps.
In CircleCI 1.0, there's several phases, and steps in those phases, that you can run commands.
The absolute earliest step is:
machine:
pre:
- echo "Some command"
Many components in the build aren't ready by this step though, including git clone. In that case, you can also run commands after the machine phase and before dependencies.
machine:
post:
- echo "Probably the better place to run early commands."
dependencies:
pre:
- echo "Run commands before any inferred dependency commands."
More on how phases in circle.yml work can be found in the CircleCI 1.0 Configuration Doc.
-Ricardo N Feliciano
Developer Evangelist, CircleCI
I move to GitLab and use all his tools that comes with it.
I installed GitLab v8.0.4, on my CentOs7 with Tomcat. I create a project and push a grails example to the git project.
Now I'd like to be able, every time I push a file to the project, to fire up a deploy. In jenkis I was able to pull the project, compile it with grails cmd tool, and deploy the war to the Tomcat.
I'm trying to do the same but I really feel lost. Does anybody have never try this, and can show me how to do?
If the deployment script is in the same repository as the project itself, you can have a build stage and a deploy stage. If the the build stage succeeds, it will start the deployment stage. The .gitlab-ci.yml could look like this:
stages:
- build
- deploy
build_grails:
stage: build
script:
- build-script_of_grails_cmd
deploy_to_tomcat:
stage: deploy
script:
- deploy_script_with_capistrano_or_whatever
If your deployment code is in another project you can trigger this project to start the deployment when the build stage has finished. The deployment repo should have a trigger setup. This can be done in the continuous integration menu of the deployment project. After setting up a trigger, GitLab generates a triggering curl snippet you can paste in the yml-file. The grails app gitlab-ci.yml will look like this:
stages:
- build
- deploy
build_grails:
stage: build
script:
- build-script_of_grails_cmd
trigger:
type: deploy
script:
- curl -X POST -F token=4579a6f10c51f0a4b7bdbd384f6e53 https://gitlab-comewhere.com/ci/api/v1/projects/5/refs/master/trigger
The gitlab-ci.yml in the deployment project will look like this:
stages:
- deploy
deploy_to_tomcat:
stage: deploy
script:
- deploy_script_with_capistrano_or_whatever