I have this .yml file:
name: CasDB
on: push
env:
CARGO_TERM_COLOR: always
jobs:
test:
runs-on: ubuntu-latest
services:
cassandra:
image: cassandra
ports:
- 9042:9042
options: --health-cmd "cqlsh --debug" --health-interval 5s --health-retries 10
steps:
- run: docker ps
- run: docker exec ${{ job.services.cassandra.id }} cqlsh --debug localhost:9042 --execute="use somekeyspace;"
I want in my Github actions to spin up a Cassandra database and than execute some queries. The Cassandra database is running, but when I want to execute a query ("use somekeyspace"), it fails with this error message:
Using CQL driver: <module ‘cassandra’ from
‘/opt/cassandra/bin/…/lib/cassandra-driver-internal-only-3.11.0-bb96859b.zip/cassandra-driver-3.11.0-bb96859b/cassandra/init.py’> Using connect timeout: 5 seconds Using ‘utf-8’ encoding Using ssl:
False Traceback (most recent call last): File
“/opt/cassandra/bin/cqlsh.py”, line 2459, in
main(*read_options(sys.argv[1:], os.environ)) File
“/opt/cassandra/bin/cqlsh.py”, line 2437, in main
encoding=options.encoding) File “/opt/cassandra/bin/cqlsh.py”, line
485, in init
load_balancing_policy=WhiteListRoundRobinPolicy([self.hostname]), File
“/opt/cassandra/bin/…/lib/cassandra-driver-internal-only-3.11.0-bb96859b.zip/cassandra-driver-3.11.0-bb96859b/cassandra/policies.py”, line 417, in init socket.gaierror: [Errno -2] Name or service not
known
##[error]Process completed with exit code 1.
What things I need to change in my .yml to:
Execute a .sql script (multiple database scripts)
Execute a single cqlsh statement
Thanks
You are simply running cqlsh with wrong parameters: You cannot give it localhost:9042 as the hostname. The hostname and port are separate parameters. Try:
cqlsh localhost 9042
Instead of cqlsh localhost:9042.
Related
I am currently working on a CI pipeline for a project and just started setting up a Github Action for running integration tests but I can't get it to work.
My action looks like this:
name: Integration Tests
on:
push:
branches:
- main
workflow_dispatch:
jobs:
integration-tests:
runs-on: ubuntu-latest
services:
selenium:
image: selenium/standalone-chrome:latest
ports:
- 4444:4444
options: --shm-size="2g"
steps:
- uses: actions/checkout#v2
- name: Get IP Address
run: echo "##[set-output name=ip;]$(ifconfig eth0 | grep 'inet [0-9\.]* ' -o | sed 's/[^0-9\.]//g')"
id: ip_addr
- name: Setup Pyhon
uses: actions/setup-python#v2
with:
python-version: '3.8'
- name: Install Python dependencies
uses: py-actions/py-dependency-install#v2
with: /path/to/requirements
- name: Run Tests
run: python3 main.py --backend http://localhost:8080/ --frontend http://${{ steps.ip_addr.outputs.ip }}:4200 --selenium http://localhost:4444/wd/hub
main.py starts two docker containers (that expose their corresponding ports) and runs a suite of selenium tests. It works on my local machine with a container I run using docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-chrome:latest. It takes the url for the backend and the one for the frontend, which gets used by selenium.
I think I have to use the IP address of the runner so selenium can access the site (since it runs locally, localhost doesn't work), but it fails with the following error:
File "main.py", line 57, in <module>
testcase.run()
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_cases/t1.py", line 14, in run
self.web_driver.accept_cookies()
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_utils/parkview_webdriver.py", line 64, in accept_cookies
self.wait_and_click(By.ID, 'confirmCookies')
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_utils/parkview_webdriver.py", line 30, in wait_and_click
WebDriverWait(self.driver, 10).until(
File "/opt/hostedtoolcache/Python/3.8.11/x64/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
It seems to just not load the time, but I don't know to fix this. I guess it could also be that I messed up the networking somehow?
I think the service container is referenced by the service name instead of localhost. i.e. in your example:
--selenium http://localhost:4444/wd/hub
would be:
--selenium http://selenium:4444/wd/hub
as you defined it:
services:
selenium:
My general question is: how to get a running service name in Github workflow?
I have a Keycloak containerset up ass a service and I want to import a realm by executing a script inside Keycloak container, here is a snippet of my workflow:
name: Test Workflow
on:
push:
branches-ignore:
- main
jobs:
test:
name: Test
runs-on: ubuntu-latest
services:
keycloak:
image: quay.io/keycloak/keycloak:12.0.4
env:
KEYCLOAK_USER: "admin"
KEYCLOAK_PASSWORD: "admin"
JAVA_OPTS_APPEND: "-Dkeycloak.profile.feature.upload_scripts=enabled"
ports:
- "8091:8080"
volumes:
- "/workspace/src/main/resources/keycloak:/src/main/resources/keycloak/"
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Setup Java
uses: actions/setup-java#v1
with:
java-version: 14
- name: List running containers
run: docker ps -a
- name: Setup Keycloak realm
run: |
docker exec -it keycloak sh -c
"/opt/jboss/keycloak/bin/kcadm.sh config credentials --server http://localhost:8080/auth --realm master --user admin --password admin &&
/opt/jboss/keycloak/bin/kcadm.sh create realms -s realm=testrealm -s enabled=true &&
/opt/jboss/keycloak/bin/kcadm.sh create partialImport -r testrealm -s ifResourceExists=SKIP -o -f /src/main/resources/keycloak/realm.json"
- name: Gradle Test
run: ./gradlew test
[...]
To connect to a running container, I need its name. A service name keycloak doesn't work as I see in logs of Github actions a list of running containers:
Run docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fdb7e1e05296 quay.io/keycloak/keycloak:12.0.4 "/opt/jboss/tools/do…" 55 seconds ago Up 47 seconds 8443/tcp, 0.0.0.0:8091->8080/tcp 594297e586cd4bdab13cc8fa63b8954d_quayiokeycloakkeycloak1104_1ac754
Is there a way to connect to a running container via running container name?
Two options:
You set the --name in the service object options:
jobs:
test:
name: Test
...
services:
keycloak:
...
options: --name keycloak --hostname keycloak
Possible docker create options.
Workflow syntax documentation.
According to this example, the key of your service object can be used as the hostname. But this seems to be only relevant when running jobs from within containers.
I am using GitHub action to do some automation test and my application was developed in docker.
name: Docker Image CI
on:
push:
branches: [ master]
pull_request:
branches: [ master]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Build the Docker image
run: docker-compose build
- name: up mysql and apache container runs
run: docker-compose up -d
- name: install dependencies
run: docker exec myapp php composer.phar install
- name: show running container
run: docker ps
- name: run unit test
run: docker exec myapp ./vendor/bin/phpunit
At the step 'show running container', I can see that all the containers are running but for the MySQL, the status is (health: starting). Thus, my unit test cases all failed as it requires a connection to MySQL. So may I know if there is a way to start the unit case only when the MySQL container's status is healthy?
I would like to offer a solution, not a smart one but it requires minimum configuration and ready to go, just use the GitHub Action for Sleeping.
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Sleep for 30 seconds
uses: jakejarvis/wait-action#master
with:
time: '30s'
Assumption: your Mysql server will be up and running in 30s.
You can use thegabriele97/dockercompose-health-action
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Check services healthiness
uses: thegabriele97/dockercompose-health-action#main
with:
timeout: '60'
workdir: 'src'
As the documentation states:
To handle this, design your application to attempt to re-establish a connection to the database after a failure. If the application retries the connection, it can eventually connect to the database.
If you can't implement this at the moment, you can write some simple script that will indefinitely try a simple statement on the database. Once the script succeed you exit loop and start your unit tests. Check the documentation link I've provided, you'll find there an example of such script (wait-for-it.sh).
My approach was to use:
in my docker-compose.yml file:
healthcheck:
test: curl --fail http://localhost/ping || exit 1
interval: 2s
retries: 10
start_period: 10s
timeout: 10s
in my Github Actions workflow:
- name: Wait for healthchecks
run: timeout 60s sh -c 'until docker ps | grep <CONTAINER_NAME> | grep -q healthy; do echo "Waiting for container to be healthy..."; sleep 2; done'
As stated in documentation:
On Linux and macOS runners, use the sleep command:
- name: Sleep for 30 seconds
run: sleep 30s
shell: bash
On Windows runners, use the Start-Sleep command:
- name: Sleep for 30 seconds
run: Start-Sleep -s 30
shell: powershell
This is steps I have done
prisma init
I set postgresql for database in my local(not exist).
It created 3 files, datamodel.graphql, docker-compose.yml, prisma.yml
docker-compose up -d
I confirmed it running successfully
But if I call prisma deploy, it shows me error
Could not connect to server at http://localhost:4466. Please check if your server is running.
All I have done is standard operation described in manual and there is no customization in
https://www.prisma.io/docs/tutorials/deploy-prisma-servers/local-(docker)-meemaesh3k
And this is docker-compose.yml
version: '3'
services:
prisma:
image: prismagraphql/prisma:1.11
restart: always
ports:
- "4466:4466"
environment:
PRISMA_CONFIG: |
port: 4466
# uncomment the next line and provide the env var PRISMA_MANAGEMENT_API_SECRET=my-secret to activate cluster security
# managementApiSecret: my-secret
databases:
default:
connector: postgres
host: localhost
port: '5432'
database: databasename
schema: public
user: postgres
password: root
migrations: true
What am I missing?
I found this solution to the same problem i was facing
docker-machine ip default
Use this address and replace the "localhost" with the IP with the above command to look something like this in prisma.yml file
endpoint: http://1xx.1xx.xx.xxx:4466
The answer is referred from this Github Link
The documentation mentions:
docker ps
You should see output similar to this:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b799c529e73 prismagraphql/prisma:1.7 "/bin/sh -c /app/sta…" 17 hours ago Up 7 hours 0.0.0.0:4466->4466/tcp myapp_prisma_1
757dfba212f7 mysql:5.7 "docker-entrypoint.s…" 17 hours ago
(Here shown with mysql, but valid with postgresql too)
The point is: there should be two containers running, not one.
Check docker-compose logs to see why the second one (database) did not start.
instead of docker-compose up -d
USE:
docker-compose up
and keep the window running which will keep localhost:4466 alive.
Note : If u want to connect to connect to the database created in docker, you need to map the port in the following way:
docker run --name <ENTER_NAME> -e POSTGRES_PASSWORD=<ENTER_PASSWORD> -d -p 5433:5432 postgres
In the above case PORT(5433) = HOST_PORT and PORT(5432) = CONTAINER_PORT
I try to deploy docker with ansible. I have one docker database container, and in other container is my web app, and I try to link this two container. The problem is that database container didn't have a time to configure itself and a web container is already started. My ansible playbook look something like:
...
- name: run mysql in docker container
docker:
image: "mysql:5.5"
name: database
env: "MYSQL_ROOT_PASSWORD=password"
state: running
- name: run application containers
docker:
name: "application"
image: "myapp"
ports:
- "8080:8080"
links:
- "database:db"
state: running
How to determine if database is start? I try with wait_for module, but that didn't work. I don't want to set timeout, it's not good option for me.
wait_for does not work for the MySQL docker container because it only checks that the port is connectable (which is true straight away for the Docker container). However, wait_for does not check that the service inside the container listens the port and sends responses to the client.
This is how I am waiting in the ansible playbook for the MySQL service becoming fully operational inside the Docker container:
- name: Start MySQL container
docker:
name: some-name
image: mysql:latest
state: started
ports:
- "8306:3306" # it's important to expose the port for waiting requests
env:
MYSQL_ROOT_PASSWORD: "{{ mysql_root_password }}"
- template: mode="a+rx,o=rwx" src=telnet.sh.j2 dest=/home/ubuntu/telnet.sh
# wait while MySQL is starting
- action: shell /home/ubuntu/telnet.sh
register: result
until: result.stdout.find("mysql_native_password") != -1
retries: 10
delay: 3
And the telnet.sh.j2 is
#!/bin/bash -e
telnet localhost 8306 || true
To avoid the sh and I don't normally have telnet installed...
- name: Wait for database to be available
shell: docker run --rm --link mysql:mysql mysql sh -c 'mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p{{mysql_password}} || true'
register: result
until: result.stderr.find("Can't connect to MySQL") == -1
retries: 10
delay: 3
As etrubenok said:
wait_for does not work for the MySQL docker container because it only checks that the port is connectable (which is true straight away for the Docker container). However, wait_for does not check that the service inside the container listens the port and sends responses to the client.
Using Andy Shinn's suggestion of FreshPow's answer, you can wait without needing a shell script or telnet:
- name: Wait for mariadb
command: >
docker exec {{ container|quote }}
mysqladmin ping -u{{ superuser|quote }} -p{{ superuser_password|quote }}
register: result
until: not result.rc # or result.rc == 0 if you prefer
retries: 20
delay: 3
This runs mysqladmin ping ... until it succeeds (return code 0). Usually superuser is root. I tested using podman instead of docker but I believe the command is the same regardless. |quote does shell escaping, which according to the Ansible docs should also be done when using command:
This works for me just fine:
- name: get mariadb IP address
command: "docker inspect --format '{''{ .NetworkSettings.IPAddress }''}' mariadb-container"
register: mariadb_ip_address
- name: wait for mariadb to become ready
wait_for:
host: "{{ mariadb_ip_address.stdout }}"
port: 3306
state: started
delay: 5
connect_timeout: 15
timeout: 30
Use wait_for module. I'm no expert on MySQL but I assume there would be some port or existence of file or message in some log file etc. you can check to find out if the DB is up or not.
Here are examples of wait_for copied from the link above.
# wait 300 seconds for port 8000 to become open on the host, don't start checking for 10 seconds
- wait_for: port=8000 delay=10
# wait 300 seconds for port 8000 of any IP to close active connections, don't start checking for 10 seconds
- wait_for: host=0.0.0.0 port=8000 delay=10 state=drained
# wait 300 seconds for port 8000 of any IP to close active connections, ignoring connections for specified hosts
- wait_for: host=0.0.0.0 port=8000 state=drained exclude_hosts=10.2.1.2,10.2.1.3
# wait until the file /tmp/foo is present before continuing
- wait_for: path=/tmp/foo
# wait until the string "completed" is in the file /tmp/foo before continuing
- wait_for: path=/tmp/foo search_regex=completed
# wait until the lock file is removed
- wait_for: path=/var/lock/file.lock state=absent
# wait until the process is finished and pid was destroyed
- wait_for: path=/proc/3466/status state=absent
# wait 300 seconds for port 22 to become open and contain "OpenSSH", don't assume the inventory_hostname is resolvable
# and don't start checking for 10 seconds
- local_action: wait_for port=22 host="{{ ansible_ssh_host | default(inventory_hostname) }}" search_regex=OpenSSH delay=10
I was able to use wait_for like this:
- name: "MySQL - Check mysql - Wait for mysql to be up"
wait_for:
host: 127.0.0.1
port: 3306
search_regex: "(mysql_native_password|caching_sha2_password)"
This way it will wait for the port o be up and for the service to send some data.
The drawback is that the output may change with mysql versions and configurations. In the example are the strings for mysql 5.5 and 8.0. Adjust for your use cases.
An alternative, avoiding running wait_for, command or shell, may be to retry some mysql command several times until it succedes:
- name: "MySQL - Check mysql - if it responds"
mysql_info:
login_user: root
login_password: "{{ mysql_root_password }}"
filter:
- version
register: mysql_result
until: mysql_result is not failed
retries: 5
delay: 10