I'm running a Symfony app inside a Docker container.
I have an env variable declared:
root#2665da58cc0e:/var/www/project# printenv | grep DATABASE
DATABASE_URL=test
And if I run php -a inside the container I can dump $_ENV:
php > var_dump($_ENV['DATABASE_URL']);
string(4) "test"
And my config looks like:
doctrine:
dbal:
default_connection: default
connections:
default:
driver: pdo_sqlite
charset: utf8mb4
url: '%env(DATABASE_URL)%'
I have no mention of DATABASE_URL in my .env file, yet I get
PHP message: PHP Fatal error: Uncaught Symfony\Component\DependencyInjection\Exception\EnvNotFoundException: Environment variable not found: "DATABASE_URL"
Why does it happen? Symfony should have taken DATABASE_URL from the system env variables, it does not happen.
Related
I am using the following command to start the MLflow server:
mlflow server --backend-store-uri postgresql://mlflow_user:mlflow#localhost/mlflow --artifacts-destination <S3 bucket location> --serve-artifacts -h 0.0.0.0 -p 8000
Before production deployment, we have a requirement that we need to print or fetch the under what configurations the server is running. For example, the above command uses localhost postgres connection and S3 bucket.
Is there a way to achieve this?
Also, how do I set the server's environment as "production"? So finally I should see a log like this:
[LOG] Started MLflow server:
Env: production
postgres: localhost:5432
S3: <S3 bucket path>
You can wrap it in a bash script or in a Makefile script, e.g.
start_mlflow_production_server:
#echo "Started MLflow server:"
#echo "Env: production"
#echo "postgres: localhost:5432"
#echo "S3: <S3 bucket path>"
#mlflow server --backend-store-uri postgresql://mlflow_user:mlflow#localhost/mlflow --artifacts-destination <S3 bucket location> --serve-artifacts -h 0.0.0.0 -p 8000
Additionally, it you can set and use environment variables specific to that server and print and use those in the command.
I have my Symfony application run perfectly fine on my development machine, on several servers except for one where it doesn't read the .env file.
The env file contains this variable:
root#719c10cf645d:/app# grep application_version .env
application_version=3.0.3-64-g642ed7f
The shell environment doesn't:
root#719c10cf645d:/app# env | grep application_version
root#719c10cf645d:/app# set | grep application_version
Symfony doesn't read the .env file:
root#719c10cf645d:/app# ./bin/console >/dev/null
[WARNING] Some commands could not be registered:
In EnvVarProcessor.php line 131:
Environment variable not found: "application_version".
[WARNING] Some commands could not be registered:
In EnvVarProcessor.php line 131:
Environment variable not found: "application_version".
On another server I have:
root#4535704c813a:/app# env | grep application_version
root#4535704c813a:/app# set | grep application_version
root#4535704c813a:/app# grep application_version .env
application_version=3.0.3-64-g642ed7f
root#4535704c813a:/app# ./bin/console >/dev/null
root#4535704c813a:/app#
I'm using the same docker version (19.03.5, build 633a0ea838) on both servers, same OS (Debian), and same docker images (different registry, but same SHA1). So everything should be perfectly identical.
What could be wrong here?
The environment on the two servers were different, and my config/bootstrap.php file didn't load .env files on one machine.
The bottom line is that I didn't follow these updates: https://symfony.com/doc/current/configuration/dot-env-changes.html
I don't know how my app survived without these changes for one year.
I am using Phusion Passenger 5.1.8 and have set the following in my .zshrc
export SECRET_KEY_BASE='secure_key_base'
export DATABASE_NAME='db_production'
export DATABASE_PASSWORD='secure_db_pass'
I then did source ~/.zshrc and restart nginx with sudo service nginx restart. However, my application is complaining that it cannot find the SECRET_KEY_BASE and fails to start up. If I manually put these in config/secrets.yml then everything works well.
My config/secrets.yml has the following:
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
and my config/database.yml file has:
production:
<<: *default
database: <%= ENV['DATABASE_NAME'] %>
password: <%= ENV['DATABASE_PASSWORD'] %>
Can someone please explain how I'd go about using zsh environment variables with Phusion Passenger?
Thanks!
So the problem is that you are using your environment variable in .zshrc and you run nginx from systemd. Both have no connection to each other.
What you need is that your nginx should have these variable, which is run through a Systemd service. You need to use what is called a Drop-in
mkdir -p /etc/systemd/system/nginx.service.d
cat > /etc/systemd/system/nginx.service.d/90-nginx-myapp.conf <<EOF
[Service]
Environment=SECRET_KEY_BASE=XYZ
Environment=SECRET_KEY_BASE2=XYZ2
EnvironmentFile=-/etc/myapp/environment
EOF
You can either use Environment= to declare a variable. Or you can use a file with environment variables.
Once you have added the drop-in, you need to reload systemd
$ systemctl daemon-reload
$ systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; disabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/nginx.service.d
└─90-nginx-myapp.conf
Active: inactive (dead)
And then use systemctl restart nginx
Now nginx should get the variables you want it to have.
Edit-1
If you need to use variables in both NGINX and your shell. The create a file with variables like beow
SECRET_KEY_BASE=XYZ
SECRET_KEY_BASE2=XYZ2
In your drop-in use the EnvironmentFile= and in your .bashrc or .bash_profile or .zshrc add the below lines
# set -a will make sure that X=Y is equivalent to export X=Y
set -a
source /etc/myapp/environment
# disable auto export of variable
set +a
I'm trying to pass values from docker-compose.yml file to Wildfly configuration dynamically.
I want to have flexibility of mail configuration - just for quick change of addres, or username, or port..
In this case, I tried to do that by forwarding environment variables from docker-compose.yml, by dockerfile as arguments "-Dargumentname=$environmentvariable.
Currently wildfly interupts on start with error:
[org.jboss.as.controller.management-operation] (ServerService Thread
Pool -- 45) WFLYCTL0013: Operation ("add") failed - address: ([
("subsystem" => "mail"),
("mail-session" => "default") ]) - failure description: "WFLYCTL0097: Wrong type for ssl. Expected [BOOLEAN] but was STRING"
Same situation, if I try to pass PORT as value in outbound-socket-binding block.
I have no idea how to pass integers/booleans from docker-compose file to Wildfly configuration.
docker-compose.yml (part)
...
services:
some_service:
image: image_name:tag
environment:
- USERNAME=some_username#...
- PASSWORD=some_password
- SSL=true // I also tried with value 1
- HOST=smtp.gmail.com
- PORT=465 // also doesn't work
...
Dockerfile:
FROM some_wildfly_base_image
# install cgroup-bin package
USER root
RUN apt-get update
RUN apt-get install -y cgroup-bin
RUN apt-get install -y bc
USER jboss
ADD standalone-myapp.xml /opt/jboss/wildfly/standalone/configuration/
ADD standalone.conf /opt/jboss/wildfly/bin/
ADD modules/ /opt/jboss/wildfly/modules/
RUN wildfly/bin/add-user.sh usr usr --silent
# Set the default command to run on boot
# This will boot WildFly in the standalone mode and bind to all interface
CMD [ "/opt/jboss/wildfly/bin/standalone.sh", "-c", "standalone-myapp.xml", "-Dmail.username=$USERNAME", "-Dmail.password=$PASSWORD", "-Dmail.ssl=$SSL", "-Drm.host=$HOST", "-Drm.port=$PORT" ]
standalone-myapp.xml:
...
<subsystem xmlns="urn:jboss:domain:mail:2.0">
<mail-session name="default" jndi-name="java:jboss/mail/Default">
<smtp-server password="${mail.password}" username="${mail.username}" ssl="${mail.ssl}" outbound-socket-binding-ref="mail-smtp"/>
</mail-session>
</subsystem>
...
<outbound-socket-binding name="mail-smtp">
<remote-destination host="${rm.host}" port="465"/>
</outbound-socket-binding>
...
Almost there. In your docker file, you have defined environmental variables therefore you need to reference them as environmental variables in your wildfly config. The easiest way is to prefix your env var with env. prefix. So in your example, you have env variables HOST, SSL, USERNAME... which you can reference in standalone.xml like this:
<smtp-server password="${env.PASSWORD}" username="${env.USERNAME}" ssl="${env.SSL}" outbound-socket-binding-ref="mail-smtp"/> </mail-session>
Without env. prefix, jboss/wildfly will try to resolve the expression as jvm property, which you'd have to specify as jvm -D flag.
You can also use default value fallback in your expressions such as:
ssl="${env.SSL:true}"
This way, the ssl will be set the the value of environmental variable named SSL, and if such var does not exist, server will fallback to true.
Happy hacking
I'm creating a docker image for our fluentd.
The image contains a file called http_forward.conf
It contains:
<store>
type http
endpoint_url ENDPOINTPLACEHOLDER
http_method post # default: post
serializer json # default: form
rate_limit_msec 100 # default: 0 = no rate limiting
raise_on_error true # default: true
authentication none # default: none
username xxx # default: ''
password xxx # default: '', secret: true
</store>
So this is in our image. But we want to use the image for all our environments. Specified with environment variables.
So we create an environment variable for our environment:
ISSUE_SERVICE_URL = http://xxx.dev.xxx.xx/api/fluentdIssue
This env variable contains dev on our dev environment, uat on uat etc.
Than we want to replace our ENDPOINTPLACEHOLDER with the value of our env variable. In bash we can use:
sed -i -- 's/ENDPOINTPLACEHOLDER/'"$ISSUE_SERVICE_URL"'/g' .../http_forward.conf
But how/when do we have to execute this command if we want to use this in our docker container? (we don't want to mount this file)
We did that via ansible coding.
Put the file http_forward.conf as template, and deploy the change depend on the environment, then mount the folder (include the conf file) to docker container.
ISSUE_SERVICE_URL = http://xxx.{{ environment }}.xxx.xx/api/fluentdIssue
playbook will be something like this, I don't test it.
- template: src=http_forward.conf.j2 dest=/config/http_forward.conf mode=0644
- docker:
name: "fluentd"
image: "xxx/fluentd"
restart_policy: always
volumes:
- /config:/etc/fluent
In your DockerFile you should have a line starting with CMD somewhere. You should add it there.
Or you can do it cleaner: set the CMD line to call a script instead. For example CMD ./startup.sh. The file startup.sh will then contain your sed command followed by the command to start your fluentd (I assume that is currently the CMD).