How to run the liquibase docker image in Jenkins containerized docker environment - docker

Question

I'm doing POC with liquibase docker image,
I would like to run the liquibase docker image in docker with Jenkins kubernetes POD template. unfortunately unable to make it.
And also I have attached the Jenkins file and my observation.
Jenkins File
def workspace_dir = "/home/jenkins/agent/workspace/${env.JOB_BASE_NAME}"
def project_name = "master-chart"
def isDeployerJob = (env.JOB_BASE_NAME).contains("deploy") ? "true" : "false"
// These variables come from the build parameters in the Jenkins job
def git_branch = git_branch
def release_version
if (isDeployerJob == "true") {
// Extracting the release version from the branch
def temp = git_branch.split("/")
release_version = temp[temp.length - 1]
switch(environment) {
case "dev":
hs_jdbc_url="jdbc:postgresql://40.xx.xx.xx:5432/dbname"
db_username="username"
db_password="pwd"
break
default:
break
}
}
pipeline {
agent {
kubernetes {
cloud 'eks-tools-13'
yaml """
apiVersion: v1
kind: Pod
spec:
containers:
- name: azcli-kubectl-helm
image: internal.docker.cioxhealth.com/azcli-kubectl-helm
command:
- cat
tty: true
- name: docker
image: docker
command:
- cat
tty: true
privileged: true
volumeMounts:
- name: dockersock
mountPath: /var/run/docker.sock
volumes:
- name: dockersock
hostPath:
path: /var/run/docker.sock
"""
}
}
stages {
stage('Install Database Scripts') {
when {
expression {
"${isDeployerJob}" == "true"
}
}
steps {
container('docker') {
sh """
docker run --rm --network="host" -v ${workspace_dir}/db:/liquibase/changelog liquibase/liquibase --url=${hs_jdbc_url} --changeLogFile=db.changelog-master.yaml --driver=org.postgresql.Driver --username=${db_username} --password=${db_password} --logLevel=info update
"""
}
}
}
}
}
For verifying the files, I have getting into running container
Jenkins Master Node:
ls -ltr /home/jenkins/agent/workspace/master-chart-deploy/db
total 4
drwxr-xr-x 3 1000 1000 21 Nov 6 04:35 sql
drwxr-xr-x 3 1000 1000 21 Nov 6 04:35 rollback
drwxr-xr-x 4 1000 1000 35 Nov 6 04:35 migration
-rw-r--r-- 1 1000 1000 154 Nov 6 04:35 db-master-changelog.yaml
drwxr-xr-x 2 1000 1000 38 Nov 6 04:35 changelog
Docker container on master-chart-deploy-259-qxrn5-nqq7j-hhlb8
ls -ltr /home/jenkins/agent/workspace/master-chart-deploy/db
total 4
drwxr-xr-x 3 1000 1000 21 Nov 6 04:35 sql
drwxr-xr-x 3 1000 1000 21 Nov 6 04:35 rollback
drwxr-xr-x 4 1000 1000 35 Nov 6 04:35 migration
-rw-r--r-- 1 1000 1000 154 Nov 6 04:35 db-master-changelog.yaml
drwxr-xr-x 2 1000 1000 38 Nov 6 04:35 changelog
Liquibase Container
docker run --rm '--network=host' -v /home/jenkins/agent/workspace/master-chart-deploy/db:/liquibase/changelog liquibase/liquibase -- ls -ltr /liquibase/changelog
total 0
Files are not available in the liquibase running container. due this the following error has been occurred.
Error:
Starting Liquibase at 14:50:38 (**version 4.1.1** #10 built at 2020-10-12 19:24+0000)
[2020-11-05 14:50:38] INFO [liquibase.lockservice] Successfully acquired change log lock
[2020-11-05 14:50:38] INFO [liquibase.lockservice] Successfully released change log lock
Unexpected error running Liquibase: db-master-changelog.yaml does not exist
For more information, please use the --logLevel flag
[2020-11-05 14:50:38] SEVERE [liquibase.integration] Unexpected error running Liquibase: db-master-changelog.yaml does not exist
liquibase.exception.ChangeLogParseException: db-master-changelog.yaml does not exist
at liquibase.parser.core.yaml.YamlChangeLogParser.parse(YamlChangeLogParser.java:27)
at liquibase.Liquibase.getDatabaseChangeLog(Liquibase.java:337)
at liquibase.Liquibase.lambda$update$1(Liquibase.java:229)
at liquibase.Scope.lambda$child$0(Scope.java:160)
at liquibase.Scope.child(Scope.java:169)
at liquibase.Scope.child(Scope.java:159)
at liquibase.Scope.child(Scope.java:138)
at liquibase.Liquibase.runInScope(Liquibase.java:2277)
at liquibase.Liquibase.update(Liquibase.java:215)
at liquibase.Liquibase.update(Liquibase.java:201)
at liquibase.integration.commandline.Main.doMigration(Main.java:1760)
at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:361)
at liquibase.Scope.lambda$child$0(Scope.java:160)
May I know, What did I do wrong in this case? and why files are not available in liquibase running container?
Is this a problem, because of file permissions due to Docker in Docker case?
Is there any other way I can achieve this?
Thank you in advance for the help.

Answer 1

I think you are somehow messing with docker configuration. From documentation it looks like liquibase expects that you mount everything inside /liquibase/changelog directory.
And in your command you are mapping your changelogs to /app/liquibase:
docker run --rm --network="host" -v ${workspace_dir}/db:/app/liquibase liquibase/liquibase --url=${hs_jdbc_url} --changeLogFile=db.changelog-master.yaml --classpath=/app/liquibase --driver=org.postgresql.Driver --username=${db_username} --password=${db_password} --logLevel=info update
so instead of that I'd use this:
docker run --rm --network="host" -v ${workspace_dir}/db:/liquibase/changelog liquibase/liquibase --url=${hs_jdbc_url} --changeLogFile=db.changelog-master.yaml --driver=org.postgresql.Driver --username=${db_username} --password=${db_password} --logLevel=info update
note: I've removed --classpath=/app/liquibase if you rely on it, if you've added some additional driver or something else you should probably include it again, but try to read about it at first. I think the documentation is pretty good.

Answer 2

you must specify full path when you use docker run in jekins pipeline:
--changeLogFile=/app/liquibase/db.changelog-master.yaml
Define in jenkins pipeline :
environment {
HOME = '.'
}

Answer 3

Change log file is the main point from where Liquibase looks for configuration. If we do not define change log file path in Spring Boot, it considers db/changelog/db.changelog-master.yaml as default path for YAML format. As we will go with XML format, we need to set spring.liquibase.change-log=classpath:/db/changelog/changelog-master.xml for change log file path in application.properties file. You can set logging level of liquibase logs by setting log level in logging.level.liquibase property. Other properties in given below properties file are for H2 database configuration.