Kartoza postgis docker image: Psql says role does not exist - docker

I'm trying to run the docker image kartoza/postgis locally on MacOS. It works perfectly when I run it on an Ubuntu instance on DigitalOcean, but when I try to interface with the local container using psql command line interface, I get the following:
Mac:~ User$ sudo psql -h localhost -U docker -p 5432 -l
psql: FATAL: role "docker" does not exist
Where docker is the username I specified when spinning up the container.
Below are the command I used to spin up the container, as well as the logs from the container:
Mac:~ User$ docker run --name "postgis" -p 5432:5432 -e ALLOW_IP_RANGE=0.0.0.0/0 -d -t kartoza/postgis
Logs:
Mac:~ User$ docker logs postgis
Add rule to pg_hba: 0.0.0.0/0
Add rule to pg_hba: replication user
Setup master database
2019-03-11 11:40:40.625 UTC [28] LOG: listening on IPv4 address "127.0.0.1", port 5432
2019-03-11 11:40:40.628 UTC [28] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-03-11 11:40:40.644 UTC [33] LOG: database system was shut down at 2019-02-01 14:24:17 UTC
2019-03-11 11:40:40.649 UTC [28] LOG: database system is ready to accept connections
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+---------+-----------------------
postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
(3 rows)
postgres ready
Postgis is missing, installing now
Creating template postgis
Enabling template_postgis as a template
UPDATE 1
Loading postgis extension
CREATE EXTENSION
Enabling hstore in the template
CREATE EXTENSION
Enabling topology in the template
CREATE EXTENSION
Loading legacy sql
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
CREATE OPERATOR CLASS
Setup postgres User:Password
CREATE ROLE
Check default db exists
Create default db gis
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privilege
s
------------------+----------+----------+---------+---------+-------------------
----
gis | docker | UTF8 | C.UTF-8 | C.UTF-8 |
postgres | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
template0 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres
+
| | | | | postgres=CTc/postg
res
template1 | postgres | UTF8 | C.UTF-8 | C.UTF-8 | =c/postgres
+
| | | | | postgres=CTc/postg
res
template_postgis | postgres | UTF8 | C.UTF-8 | C.UTF-8 |
(5 rows)
2019-03-11 11:40:43.331 UTC [28] LOG: received smart shutdown request
2019-03-11 11:40:43.339 UTC [28] LOG: background worker "logical replication launcher" (PID 39) exited with exit code 1
2019-03-11 11:40:43.340 UTC [34] LOG: shutting down
2019-03-11 11:40:43.362 UTC [28] LOG: database system is shut down
/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
Postgres initialisation process completed .... restarting in foreground
2019-03-11 11:40:44.382 UTC [212] LOG: listening on IPv4 address "0.0.0.0", port 5432
2019-03-11 11:40:44.382 UTC [212] LOG: listening on IPv6 address "::", port 5432
2019-03-11 11:40:44.387 UTC [212] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-03-11 11:40:44.406 UTC [215] LOG: database system was shut down at 2019-03-11 11:40:43 UTC
2019-03-11 11:40:44.414 UTC [212] LOG: database system is ready to accept connections
As far as I can see the user "docker" is the owner of the database "gis".

Can you set the database "gis":
psql -d gis -U username -h localhost
Then you can give your password.

Related

PostgreSQL installation issue Character Type

I installed PostgreSQL 9.2 with default database postgres.
It shows Collation and Character Type as en_GB.UTF-8.
Is there any difference between en_GB and en_US. Can I change to en_US without uninstalling.
First option is while creating a new database you may provide the Collation and Ctype like
postgres=# CREATE DATABASE test WITH LC_COLLATE='en_US' Encoding='LATIN1' lc_ctype='en_US' TEMPLATE=template0;;
CREATE DATABASE
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
postgres | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 |
template0 | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
test | postgres | LATIN1 | en_US | en_US |
(4 rows)
Second option is to do the fresh initdb and provide the -E encoding and --locale=locale. Sets the default locale for the database cluster. If this option is not specified, the locale is inherited from the environment that initdb runs in
See more details about initdb with --locale at postgres initdb

Docker not properly installing python packages using pip install -r requirements.txt

I am pretty new to Docker and Django. I am trying to set up a Django project for a REST-ful API running in a Docker container. I am trying to import the relavent python packages from a RUN command in the dockerfile, however, not all the packages are successfully installing.
Here are the files I'm using and the error I am getting.
Dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
COPY . .
docker-compose.yml:
version: '3'
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: password
web:
build: .
# command: bash -c "pip install -r requirements.txt && python manage.py runserver 0.0.0.0:8000"
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
requirements.txt
djangorestframework
django-filter
markdown
Django
psycopg2
When I execute docker-compose up I get this output
Starting apiTest_db_1 ... done
Recreating apiTest_web_1 ... done
Attaching to apiTest_db_1, apiTest_web_1
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2020-04-17 21:35:57.022 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2020-04-17 21:35:57.023 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-04-17 21:35:57.023 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2020-04-17 21:35:57.028 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-04-17 21:35:57.075 UTC [27] LOG: database system was shut down at 2020-04-17 21:34:34 UTC
db_1 | 2020-04-17 21:35:57.100 UTC [1] LOG: database system is ready to accept connections
web_1 | Watching for file changes with StatReloader
web_1 | Exception in thread django-main-thread:
web_1 | Traceback (most recent call last):
web_1 | File "/usr/local/lib/python3.8/threading.py", line 932, in _bootstrap_inner
web_1 | self.run()
web_1 | File "/usr/local/lib/python3.8/threading.py", line 870, in run
web_1 | self._target(*self._args, **self._kwargs)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
web_1 | fn(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/commands/runserver.py", line 109, in inner_run
web_1 | autoreload.raise_last_exception()
web_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 76, in raise_last_exception
web_1 | raise _exception[1]
web_1 | File "/usr/local/lib/python3.8/site-packages/django/core/management/__init__.py", line 357, in execute
web_1 | autoreload.check_errors(django.setup)()
web_1 | File "/usr/local/lib/python3.8/site-packages/django/utils/autoreload.py", line 53, in wrapper
web_1 | fn(*args, **kwargs)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
web_1 | apps.populate(settings.INSTALLED_APPS)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate
web_1 | app_config = AppConfig.create(entry)
web_1 | File "/usr/local/lib/python3.8/site-packages/django/apps/config.py", line 90, in create
web_1 | module = import_module(entry)
web_1 | File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
web_1 | return _bootstrap._gcd_import(name[level:], package, level)
web_1 | File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
web_1 | File "<frozen importlib._bootstrap>", line 991, in _find_and_load
web_1 | File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
web_1 | ModuleNotFoundError: No module named 'rest_framework'
Which indicates that djangorestframework has not been installed by pip.
Furthermore, when I switch the comented line in the docker-compose.yml file for the line below it (so that section becomes)
command: bash -c "pip install -r requirements.txt && python manage.py runserver 0.0.0.0:8000"
# command: python manage.py runserver 0.0.0.0:8000
Then when I run docker-compose up I get the following output.
Creating network "apiTest_default" with the default driver
Creating apiTest_db_1 ... done
Creating apiTest_web_1 ... done
Attaching to apiTest_db_1, apiTest_web_1
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting dynamic shared memory implementation ... posix
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting default time zone ... Etc/UTC
db_1 | creating configuration files ... ok
db_1 | running bootstrap script ... ok
db_1 | performing post-bootstrap initialization ... ok
web_1 | Collecting djangorestframework
db_1 | syncing data to disk ... initdb: warning: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 | ok
db_1 |
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | waiting for server to start....2020-04-17 22:47:22.783 UTC [46] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2020-04-17 22:47:22.789 UTC [46] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
web_1 | Downloading djangorestframework-3.11.0-py3-none-any.whl (911 kB)
db_1 | 2020-04-17 22:47:22.823 UTC [47] LOG: database system was shut down at 2020-04-17 22:47:22 UTC
db_1 | 2020-04-17 22:47:22.841 UTC [46] LOG: database system is ready to accept connections
db_1 | done
db_1 | server started
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1 |
db_1 | 2020-04-17 22:47:22.885 UTC [46] LOG: received fast shutdown request
db_1 | waiting for server to shut down....2020-04-17 22:47:22.889 UTC [46] LOG: aborting any active transactions
db_1 | 2020-04-17 22:47:22.908 UTC [46] LOG: background worker "logical replication launcher" (PID 53) exited with exit code 1
db_1 | 2020-04-17 22:47:22.920 UTC [48] LOG: shutting down
db_1 | 2020-04-17 22:47:22.974 UTC [46] LOG: database system is shut down
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | 2020-04-17 22:47:23.021 UTC [1] LOG: starting PostgreSQL 12.2 (Debian 12.2-2.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2020-04-17 22:47:23.022 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2020-04-17 22:47:23.023 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2020-04-17 22:47:23.036 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2020-04-17 22:47:23.063 UTC [55] LOG: database system was shut down at 2020-04-17 22:47:22 UTC
db_1 | 2020-04-17 22:47:23.073 UTC [1] LOG: database system is ready to accept connections
web_1 | Collecting django-filter
web_1 | Downloading django_filter-2.2.0-py3-none-any.whl (69 kB)
web_1 | Collecting markdown
web_1 | Downloading Markdown-3.2.1-py2.py3-none-any.whl (88 kB)
web_1 | Requirement already satisfied: Django in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (3.0.5)
web_1 | Requirement already satisfied: psycopg2 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (2.8.5)
web_1 | Requirement already satisfied: setuptools>=36 in /usr/local/lib/python3.8/site-packages (from markdown->-r requirements.txt (line 3)) (46.1.3)
web_1 | Requirement already satisfied: pytz in /usr/local/lib/python3.8/site-packages (from Django->-r requirements.txt (line 4)) (2019.3)
web_1 | Requirement already satisfied: sqlparse>=0.2.2 in /usr/local/lib/python3.8/site-packages (from Django->-r requirements.txt (line 4)) (0.3.1)
web_1 | Requirement already satisfied: asgiref~=3.2 in /usr/local/lib/python3.8/site-packages (from Django->-r requirements.txt (line 4)) (3.2.7)
web_1 | Installing collected packages: djangorestframework, django-filter, markdown
web_1 | Successfully installed django-filter-2.2.0 djangorestframework-3.11.0 markdown-3.2.1
web_1 | Watching for file changes with StatReloader
web_1 | Performing system checks...
web_1 |
web_1 | System check identified no issues (0 silenced).
web_1 |
web_1 | You have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
web_1 | Run 'python manage.py migrate' to apply them.
web_1 | April 17, 2020 - 22:47:25
web_1 | Django version 3.0.5, using settings 'apiTesting.settings'
web_1 | Starting development server at http://0.0.0.0:8000/
web_1 | Quit the server with CONTROL-C.
Which shows that some packages such as Django have been successfully installed by the Dockerfile but some like djangorestframework, django-filter and markdown have not.
Why is this and what can I do in my Dockerfile to make them correctly install?
Both the main problem and the problem mentioned in the comments of itamar-turner-trauring's answer were solved by instead of running docker-compose up running
docker-compose up --build
Not 100% sure why this fixed it but I'd guess the compose file was loaing up the container from an old image which didn't include the new python packages. So forcing it to rebuild made it include the new python packages.
You are doing two things that potentially conflict:
Inside the image, as part of the build you copy everything in to /code.
In the compose file you mount current working directory into /code.
I am not sure that's the problem, but I suggest removing the volumes bit from the compose.yml and see if that help.

Ident authentication failed for user

I am trying to set up my rails app with a Postgres database which I am new to.
I am using
centos 6
postgresql 9.6.2
pg_hba.conf path => /var/lib/pgsql/data/pg_hba.conf
I had logged in with sudo -u postgres psql and created a user 'X' with password 'password'. Below are the details
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
--------------+--------------+----------+-------------+-------------+-------------------------------
postgres | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
template0 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =c/postgres +
| | | | | postgres=CTc/postgres
trip_staging | X | UTF8 | en_US.UTF-8 | en_US.UTF-8 | =Tc/X +
| | | | | X=CTc/X
(4 rows)
postgres=# \du
List of roles
Role name | Attributes | Member of
--------------+------------------------------------------------------------+-----------
X | | {}
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
When I run command:
RAILS_ENV=staging bundle exec rake db:create
I get the below error:
FATAL: Ident authentication failed for user "X"
Couldn't create database for {"database"=>"trip_staging", "adapter"=>"postgresql", "username"=>"X", "password"=>"password", "host"=>"localhost", "port"=>5432, "pool"=>5, "timeout"=>5000}
After this I looked up each possible fix for this. I tried changing the pg_hba.conf with below configuration:
TYPE DATABASE USER ADDRESS METHOD
local all all md5
host all all 0.0.0.0 md5
host all all ::1/128 md5
Also tried, changing METHOD with trust and then restarted the postgresql server with the command: sudo service postgresql-9.6 restart but same error.
Also tried, changing read permissions for path /var/lib/pgsql/data/pg_hba.conf from 600 to 755. To do that I had to login as postgres superuser with sudo su - postgres which is weird. Then restarted the postgresql server
Also tried, changing Method from ident to trust for path /var/lib/pgsql/9.6/data/pg_hba.conf, as apparently there exists a version wise pg_hba.conf too. Then I restarted the PostgreSQL server.
Nothing seemed to have worked so far. Please help find out what is missing.
In the pg_hba.conf the part of 0.0.0.0 needs the mask like this 0.0.0.0/32, in the log says the problem, wich is by default in centos in /var/lib/pgsql/9.6/data/pg_log/

Docker Compose with Rails hanging up on command

I'm following the Docker Rails tutorial via https://docs.docker.com/compose/rails/#build-the-project (using Windows 10 Home)
EDIT: Just a note, I am using docker-toolbox because Docker requires Windows 10 Pro for Hyper-V, and I have Windows 10 Home edition.
I have gone through the tutorial several times, and each time I run docker-compose up, it gets hung up and doesn't mention the local port the app is running on. I have also tried making a new app and changing the port number in docker-compose.yml to see if that would fix the issue.
All of the previous commands in the tutorial have worked properly, and I have edited the config/database.yml file correctly, before running the docker-compose up command.
I have deleted images, and all files and started from scratch several times. I still run into the same issue.
Here are the commands from the tutorial:
docker-compose run web rails new . --force --database=postgresql --skip-bundle
docker-compose build
docker-compose up
docker-compose run web rails db:create
Here is my docker-compose.yml file:
version: '2'
services:
db:
image: postgres
web:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
Here is my Dockerfile:
FROM ruby:2.3.3
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev nodejs
RUN mkdir /myapp
WORKDIR /myapp
ADD Gemfile /myapp/Gemfile
ADD Gemfile.lock /myapp/Gemfile.lock
RUN bundle install
ADD . /myapp
Here is the output from docker-compose up:
$ docker-compose up
mydockerbuild_db_1 is up-to-date
Creating mydockerbuild_web_1
Attaching to mydockerbuild_db_1, mydockerbuild_web_1
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting dynamic shared memory implementation ... posix
db_1 | creating configuration files ... ok
db_1 | running bootstrap script ... ok
db_1 | performing post-bootstrap initialization ... ok
db_1 |
db_1 | WARNING: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 | syncing data to disk ... ok
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | ****************************************************
db_1 | WARNING: No password has been set for the database.
db_1 | This will allow anyone with access to the
db_1 | Postgres port to access your database. In
db_1 | Docker's default configuration, this is
db_1 | effectively any other container on the same
db_1 | system.
db_1 |
db_1 | Use "-e POSTGRES_PASSWORD=password" to set
db_1 | it in "docker run".
db_1 | ****************************************************
db_1 | waiting for server to start....LOG: database system was shut down at 2017-02-14 18:56:05 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
db_1 | done
db_1 | server started
db_1 | ALTER ROLE
db_1 |
db_1 |
db_1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint- initdb.d/*
db_1 |
db_1 | LOG: received fast shutdown request
db_1 | LOG: aborting any active transactions
db_1 | LOG: autovacuum launcher shutting down
db_1 | LOG: shutting down
db_1 | waiting for server to shut down....LOG: database system is shut down
db_1 | done
db_1 | server stopped
db_1 |
db_1 | PostgreSQL init process complete; ready for start up.
db_1 |
db_1 | LOG: database system was shut down at 2017-02-14 18:56:06 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
db_1 | LOG: received smart shutdown request
db_1 | LOG: autovacuum launcher shutting down
db_1 | LOG: shutting down
db_1 | LOG: database system is shut down
db_1 | LOG: database system was shut down at 2017-02-14 19:06:12 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
db_1 | ERROR: database "myapp_development" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_development" ENCODING = 'unicode'
db_1 | ERROR: database "myapp_test" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_test" ENCODING = 'unicode'
db_1 | ERROR: database "myapp_development" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_development" ENCODING = 'unicode'
db_1 | ERROR: database "myapp_test" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_test" ENCODING = 'unicode'
db_1 | ERROR: database "myapp_development" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_development" ENCODING = 'unicode'
db_1 | ERROR: database "myapp_test" already exists
db_1 | STATEMENT: CREATE DATABASE "myapp_test" ENCODING = 'unicode'

docker-compose generating duplicate entries in /etc/hosts

I have a fairly simple docker-compose.yml:
db:
build: docker/db
env_file:
- .env
ports:
- "5432"
web:
build: .
env_file:
- .env
volumes:
- .:/home/app/emerson
ports:
- "80:80"
links:
- db
The web container launches a rails app. Everything goes smoothly, but there is one thing that confuses me. Looking inside /etc/hosts on the web container, I see the following entries:
172.17.0.10 db_1
172.17.0.10 emerson_db_1
172.17.0.10 db
I would expect db, since that's the container I'm linking to the web container, but where did the other guys come from? FYI, here's the output of docker-compose up:
Creating emerson_db_1...
Creating emerson_web_1...
Attaching to emerson_db_1, emerson_web_1
db_1 | The files belonging to this database system will be owned by user "postgres".
db_1 | This user must also own the server process.
db_1 |
db_1 | The database cluster will be initialized with locale "en_US.utf8".
db_1 | The default database encoding has accordingly been set to "UTF8".
db_1 | The default text search configuration will be set to "english".
db_1 |
db_1 | Data page checksums are disabled.
db_1 |
db_1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1 | creating subdirectories ... ok
db_1 | selecting default max_connections ... 100
db_1 | selecting default shared_buffers ... 128MB
db_1 | selecting dynamic shared memory implementation ... posix
db_1 | creating configuration files ... ok
web_1 | *** Running /etc/my_init.d/00_configure_nginx.sh...
web_1 | *** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
web_1 | No SSH host key available. Generating one...
db_1 | ok
db_1 | initializing pg_authid ... ok
web_1 | Creating SSH2 RSA key; this may take some time ...
db_1 | initializing dependencies ... ok
web_1 | Creating SSH2 DSA key; this may take some time ...
web_1 | Creating SSH2 ECDSA key; this may take some time ...
web_1 | Creating SSH2 ED25519 key; this may take some time ...
db_1 | creating system views ... ok
db_1 | loading system objects' descriptions ... ok
db_1 | creating collations ... ok
db_1 | creating conversions ... ok
db_1 | creating dictionaries ... ok
db_1 | setting privileges on built-in objects ... ok
web_1 | invoke-rc.d: policy-rc.d denied execution of restart.
db_1 | creating information schema ... ok
web_1 | *** Running /etc/my_init.d/30_presetup_nginx.sh...
web_1 | *** Running /etc/rc.local...
db_1 | loading PL/pgSQL server-side language ... ok
web_1 | *** Booting runit daemon...
web_1 | *** Runit started as PID 98
db_1 | vacuuming database template1 ... ok
db_1 | copying template1 to template0 ... ok
db_1 | copying template1 to postgres ... ok
web_1 | Apr 24 02:44:26 1d3b7bb27612 syslog-ng[105]: syslog-ng starting up; version='3.5.3'
db_1 | syncing data to disk ... ok
db_1 |
db_1 | WARNING: enabling "trust" authentication for local connections
db_1 | You can change this by editing pg_hba.conf or using the option -A, or
db_1 | --auth-local and --auth-host, the next time you run initdb.
db_1 |
db_1 | Success. You can now start the database server using:
db_1 |
db_1 | postgres -D /var/lib/postgresql/data
db_1 | or
db_1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1 |
db_1 | ****************************************************
db_1 | WARNING: No password has been set for the database.
db_1 | This will allow anyone with access to the
db_1 | Postgres port to access your database. In
db_1 | Docker's default configuration, this is
db_1 | effectively any other container on the same
db_1 | system.
db_1 |
db_1 | Use "-e POSTGRES_PASSWORD=password" to set
db_1 | it in "docker run".
db_1 | ****************************************************
db_1 |
db_1 | PostgreSQL stand-alone backend 9.4.1
db_1 | backend> statement: ALTER USER "postgres" WITH SUPERUSER ;
db_1 |
web_1 | ok: run: /etc/service/nginx-log-forwarder: (pid 118) 0s
db_1 | backend>
db_1 | No PostgreSQL clusters exist; see "man pg_createcluster" ... (warning).
db_1 |
db_1 | backend> *******************************************
db_1 | LOG: database system was shut down at 2015-04-24 02:44:28 UTC
db_1 | LOG: database system is ready to accept connections
db_1 | LOG: autovacuum launcher started
web_1 | [ 2015-04-24 02:44:27.9386 119/7f4c07f13780 agents/Watchdog/Main.cpp:538 ]: Options: { 'analytics_log_user' => 'nobody', 'default_group' => 'nogroup', 'default_python' => 'python', 'default_ruby' => '/usr/bin/ruby', 'default_user' => 'nobody', 'log_level' => '0', 'max_pool_size' => '6', 'passenger_root' => '/usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini', 'passenger_version' => '4.0.58', 'pool_idle_time' => '300', 'temp_dir' => '/tmp', 'union_station_gateway_address' => 'gateway.unionstationapp.com', 'union_station_gateway_port' => '443', 'user_switching' => 'true', 'web_server_passenger_version' => '4.0.58', 'web_server_pid' => '107', 'web_server_type' => 'nginx', 'web_server_worker_gid' => '33', 'web_server_worker_uid' => '33' }
web_1 | [ 2015-04-24 02:44:27.0007 122/7f0c3eb9a780 agents/HelperAgent/Main.cpp:650 ]: PassengerHelperAgent online, listening at unix:/tmp/passenger.1.0.107/generation-0/request
web_1 | [ 2015-04-24 02:44:28.1065 127/7f5e5b4377c0 agents/LoggingAgent/Main.cpp:321 ]: PassengerLoggingAgent online, listening at unix:/tmp/passenger.1.0.107/generation-0/logging
web_1 | [ 2015-04-24 02:44:28.1072 119/7f4c07f13780 agents/Watchdog/Main.cpp:728 ]: All Phusion Passenger agents started!
But there are only two containers docker ps -a outputs:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1d3b7bb27612 emerson_web:latest "/sbin/my_init" About an hour ago Up About an hour 443/tcp, 0.0.0.0:80->80/tcp emerson_web_1
0c047c3ce103 emerson_db:latest "/docker-entrypoint. About an hour ago Up About an hour 0.0.0.0:49156->5432/tcp emerson_db_1
In addition, I also see duplicate environment variables in the web container, corresponding to db, db_1 and emerson_db_1 prefixes.
They are coming from pre-1.0 docker-compose, where multiple db instances where named after _1, _2 pattern.
PR 364 introduced link name (by default, the name of the linked service) as the hostname to connect to, instead of using environment variable.
There are still aliases with _x added for each container instances, and that can be an issue (Issue 472: Hostnames with underscore fails in ruby URI validation
The current answer is:
You can use the name of the service in the docker-compose.yml as the hostname. It doesn't contain any underscores.
You can also add an alias to your link to the container, which should allow you to access it as just the alias.
In the 1.3 release of compose there should be support for naming your container as anything you want, which will make this more obvious.

Resources