I am unable to read Docker environment variables in xml file.
I have below docker setup.
docker-compose.yaml (part)
spark:
build:
context: .
dockerfile: some.Dockerfile
environment:
- user_name=testuser
ports:
- 'xxxx:xxxx'
some.Dockerfile:
FROM some_image
COPY site.xml /opt/xxxx/xxxx/conf/
site.xml (part)
?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.username</name>
<value>"${env.user_name}"</value>
</property>
when I am running docker-compose up I am not amble to get the proper value for ${env.user_name}
I am expecting testuser but I am getting "${env.user_name}" inside the container. I am not able to find the mistake that I am doing to retrieve the env variable in my xml file.
I am able to acheive this using the below command. I am replacing the value in xml file, after xml file copied to docker container. Here's how my modified files look like.
docker-compose.yaml (part)
spark:
build:
context: .
dockerfile: some.Dockerfile
args:
HOST_NAME=localhost
environment:
- user_name=testuser
ports:
- 'xxxx:xxxx'
some.Dockerfile:
FROM some_image
COPY site.xml /opt/xxxx/xxxx/conf/
ARG HOST_NAME
#Replacing the host_name value with build ARG value.
RUN sed -i -r "s/host_name/"${HOST_NAME}"/g" /opt/xxxx/xxxx/conf/site.xml
site.xml (part)
?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.username</name>
<value>"host_name"</value>
</property>
Related
Ι have this Dockerfile:
FROM openjdk:11
ENV JAR_FILE=target/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
The structure of my application is:
Demo:
--deployment:
--Dockerfile
--src/
--docker-compose.yaml
--target:
--app.jar
Code snippet from docker-compose file:
api:
container_name: backend
image: backend
build:
context: deployment/
ports:
- "8080:8080"
When I am putting the Dockerfile in the same directory with the docker-compose and I am changing the docker compose to:
api:
container_name: backend
image: backend
build: .
ports:
- "8080:8080"
Is running as expected. But I want to put the Dockerfile into the deployment folder, since there I have the helm chart and others docker-comopose files which are using this Dockerfile.
My question is:
How I can specify the correct path of the target folder in the Dockerfile?
You cannot copy anything which is out of the build context. If you want to keep the current project structure a solution would be in your compose file for the api service:
build:
context: .
dockerfile: deployment/Dockerfile
I'm working on a group project involving Docker that has a .env file, which looks like this:
DATABASE_URL=xxx
DJANGO_SETTINGS_MODULE=xxx
SECRET_KEY=xxx
Couldn't this just be declared inside the Dockerfile? If so, what is the advantage of making a .env file?
Not sure if I'm going in the right direction with this, but this Docker Docs page says (emphasis my own):
Your configuration options can contain environment variables. Compose
uses the variable values from the shell environment in which
docker-compose is run. For example, suppose the shell contains
POSTGRES_VERSION=9.3 and you supply this configuration:
db:
`image: "postgres:${POSTGRES_VERSION}"`
When you run docker-compose up with this configuration, Compose looks for the POSTGRES_VERSION environment variable in the shell and substitutes its value in. For this example, Compose resolves the image to postgres:9.3 before running the configuration.
If an environment variable is not set, Compose substitutes with an empty string. In the example above, if POSTGRES_VERSION is not set, the value for the image option is postgres:.
You can set default values for environment variables using a .env file, which Compose automatically looks for. Values set in the shell environment override those set in the .env file.
If we're using a .env file, then wouldn't I see some ${...} syntax in our docker-compose.yml file? I don't see anything like that, though.
Here's our docker-compose.yml file:
version: '3'
services:
server:
build:
context: ./server
dockerfile: Dockerfile
env_file: .env.dev
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./server:/app
ports:
- "8500:8000"
depends_on:
- db
stdin_open: true
tty: true
db:
image: postgres
client:
build:
context: ./client
dockerfile: Dockerfile
command: bash -c "npm install; npm run start"
volumes:
- ./client:/app
- /app/node_modules
ports:
- "3000:3000"
depends_on:
- server
Idea there is probably to have a place to keep secrets separated from docker-compose.yml, which you then can keep in VCS and/or share.
I am packing my application into jar and run it with Docker using bash script (which executes jar). I would like to change neo4j database url configuration as environment variable in docker-compose file, however, I still get this error:
Exception in thread "main" com.typesafe.config.ConfigException$NotResolved: neo4j.url has not been resolved, you need to call Config#resolve(), see API docs for Config#resolve()
How can I solve this problem and where should I add some configurations for that?
I have only set up url variable in configurations file:
neo4j{
url= "bolt://localhost:7687"
url = ${?HOSTNAME}
user = "user"
password = "password"
}
Also, I use this variable in configurations method:
def getNeo4jConfig(configName: String) = {
val neo4jLocalConfig = ConfigFactory.parseFile(new File("configs/local_neo4j.conf"))
neo4jLocalConfig.resolve()
val driver = configName match {
case "neo4j_local" => GraphDatabase.driver(neo4jLocalConfig.getString("neo4j.url"),
AuthTokens.basic(neo4jLocalConfig.getString("neo4j.user"), neo4jLocalConfig.getString("neo4j.password")))
case _ => GraphDatabase.driver("url", AuthTokens.basic("user", "password"))
}
driver.session
}
In docker-compose.yml I defined the value of the hostname:
version: '3.3'
services:
neo4j_db:
image: neo4j:latest
ports:
- "7474:7474"
- "7473:7473"
- "7687:7687"
volumes:
- $HOME/neo4j/import:/var/lib/neo4j/import
- $HOME/neo4j/data:/neo4j/data
- $HOME/neo4j/conf:/neo4j/conf
- $HOME/neo4j/logs:/neo4j/logs
environment:
- NEO4J_dbms_active__database=graphImport.db
benchmarks:
image: "container"
volumes:
- ./:/workdir1
working_dir: /workdir1
links:
- neo4j_db
environment:
- HOSTNAME=myhoat
Also, bash script looks like this:
#!/usr/bin/env bash
for run in {1..2}
do
java -cp "target/scala-2.11/benchmarking.jar" benchmarks/Main $1 $2
done
Set these env in a .env file in the same location than your
docker-compose.yml file:
.env
VAR1=value
VAR2=2.0
VAR3=`awk -F ':' '{if ($3 == 1000) {print $1}}' /etc/passwd` <-- any bash command, for example
...
And for example, use them in your compose file sections:
docker-compose.yml
...
build:
dockerfile: Dockerfile_${VAR2}
...
I defined hostname as arg in Dockerfile and then defined it in docker-compose file for the application build. That solved the problem!
Docker-compose file:
version: '3.3'
services:
benchmarks-app:
build:
context: .
dockerfile: Dockerfile
args:
- HOST=neo4jdb
volumes:
- ./:/workdir
working_dir: /workdir
Dockerfile args definition:
FROM java:8
ARG HOST
ENV SCALA_VERSION 2.11.8
ENV SBT_VERSION 1.1.1
ENV SPARK_VERSION 2.2.0
ENV SPARK_DIST spark-$SPARK_VERSION-bin-hadoop2.6
ENV SPARK_ARCH $SPARK_DIST.tgz
ENV NEO4J_CONFIG $DB_CONFIG
ENV BENCHMARK $BENCHMARK_NAME
ENV HOSTNAME bolt://$HOST:7687
...
I am having an issue with accessing rest service with docker as http://192.168.99.100:8080/rest/clients.I am getting HTTP Status 404 – Not Found. While without docker I have access the rest service like http://localhost:8080/rest/clients.Please anybody helps on my issue.
here is my docker-compose file,
version: '3'
services:
tomcat:
container_name: tomcat_docker
image: tomcat:latest
environment:
- TOMCAT_USERNAME=admin
- TOMCAT_PASSWORD=secret
ports:
- 8080:8080
volumes:
- /docker/AW317/TaskmanagerDocker-03212018/tomcat-users.xml:/usr/local/tomcat/conf/tomcat-users.xml
- /docker/AW317/TaskmanagerDocker-03212018/TaskManager.war:/usr/local/tomcat/webapps/TaskManager.war
command: ["/usr/local/tomcat/bin/catalina.sh", "run"]
links:
- db:mysql
db:
container_name: mysql_docker
image: mysql
ports:
- 53306:3306
restart: always
environment:
MYSQL_ROOT_PASSWORD: root
I have tried to overwrite tomcat-users.xml but won't help.
can any anybody help me on this issue? what am I missing?
I have just overwrite context.xml file in docker using volume path and issue resolved,
<?xml version="1.0" encoding="UTF-8"?>
<Context antiResourceLocking="true" privileged="true" >
<!--<Valve className="org.apache.catalina.valves.RemoteAddrValve"
allow="127\.\d+\.\d+\.\d+|::1|0:0:0:0:0:0:0:1" />-->
</Context>
Now I can access the rest service.
Edit:
you can overwrite context.xml in the docker-compose file as bellow,
volumes:
- <path to docker whaere it is located>/context.xml:/usr/local/tomcat/webapps/manager/META-INF/context.xml
you can put context.xml file in your project directory.
I have written a Dockerfile which uses two arguments:
FROM jessie
MAINTAINER Zeinab Abbasimazar
#Build Arguments
ARG REP_USER
ARG REP_PASS
# Build
RUN echo 'REP_USER:'$REP_USER', REP_PASS:'$REP_PASS
I wrote a docker-compose.yml for build:
version: "2"
services:
ui:
build:
context: .
dockerfile: Dockerfile
args:
REP_USER: $REP_USER
REP_PASS: $REP_PASS
I don't want to define these arguments directly in the compose file, so I tried to send them during docker compose build:
REP_USER=myusername REP_PASS=mypassword docker-compose build
Which didn't work. I changed my Dockerfile to use these arguments as environment variables; so I removed ARG lines:
FROM jessie
MAINTAINER Zeinab Abbasimazar
# Build
RUN echo 'REP_USER:'$REP_USER', REP_PASS:'$REP_PASS
And docker-compose.yml:
version: "2"
services:
ui:
build:
context: .
dockerfile: Dockerfile
And ran REP_USER=myusername REP_PASS=mypassword docker-compose build; still no result.
I also tried to save these information into an env file:
version: "2"
services:
ui:
build:
context: .
dockerfile: Dockerfile
env_file:
- myenv.env
But it seems env files doesn't affect at build time; they are just take part into run time.
EDIT 1:
Docker version is 1.12.6 which doesn't support passing arguments with --build-arg.
EDIT 2:
I tried using .env file as described here:
cat .env
REP_USER=myusername
REP_PASS=mypassword
I then called docker-compose config which returned:
networks: {}
services:
ui:
build:
args:
REP_PASS: mypassword
REP_USER: myusername
context: /home/zeinab/Workspace/ZiZi-Docker/Test/test-exec-1
dockerfile: Dockerfile
version: '2.0'
volumes: {}
Which means this resolved my issue.
EDIT 3:
I also tried third section of docker-compose arg documentation in my docker-compose.yml file:
version: "2"
services:
ui:
build:
context: .
dockerfile: Dockerfile
args:
- REP_USER
- REP_PASS
And executed:
export REP_USER=myusername;export REP_PASS=mypassword;sudo docker-compose build --no-cache
Still not getting what I wanted.
You can set build arguments with docker compose as described here:
docker-compose build [--build-arg key=val...]
docker-compose build --build-arg REP_USER=myusername --build-arg REP_PASS=mypassword
Btw, AFAIK build arguments are a compromise between usability and deterministic building. Docker aims to build in a deterministic fashion. That is, wherever you execute the build the produced image should be the same. Therefore, it appears logical that the client ignores the environment (variables) it is executed in.
The correct syntax for variable substitution in a docker-compose file is ${VARNAME}.
Try with this one:
version: "2"
services:
ui:
build:
context: .
dockerfile: Dockerfile
args:
REP_USER: ${REP_USER}
REP_PASS: ${REP_PASS}
I finally found the solution. I mentioned it in the question too. I first tried it with fail, then I found out that I had a typo naming .env file; it was .evn.
I tried using .env file as described here:
cat .env
REP_USER=myusername
REP_PASS=mypassword
I then called docker-compose config which returned:
networks: {}
services:
ui:
build:
args:
REP_PASS: mypassword
REP_USER: myusername
context: /home/zeinab/Workspace/ZiZi-Docker/Test/test-exec-1
dockerfile: Dockerfile
version: '2.0'
volumes: {}
Which means this resolved my issue. I should mention that this answer was really helpful.