I'm trying to run selenium hub with nodes generated dynamically with docker.
Here is my docker compose file.
node-docker:
image: selenium/node-docker:${SELENIUM_DOCKER_TAG}
container_name: node-docker
volumes:
- ${SELENIUM_ASSETS}:/opt/selenium/assets
- ./config.toml:/opt/bin/config.toml
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
- SE_VNC_VIEW_ONLY=1
- SE_VNC_NO_PASSWORD=1
- TZ=${SELENIUM_TZ}
- NODE_OVERRIDE_MAX_SESSIONS=${SELENIUM_OVERRIDE_MAX_SESSIONS}
- NODE_MAX_SESSIONS=${SELENIUM_MAX_SESSIONS}
restart: always
selenium-hub:
image: selenium/hub:${SELENIUM_DOCKER_TAG}
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
environment:
- SE_OPTS=--allow-cors true
- TZ=${SELENIUM_TZ}
restart: always
And the config.toml file is as follow:
[docker]
# Configs have a mapping between the Docker image to use and the capabilities that need to be matched to
# start a container with the given image.
configs = [
"selenium/standalone-firefox:4.4.0-20220812", "{\"browserName\": \"firefox\"}",
"selenium/standalone-chrome:4.4.0-20220812", "{\"browserName\": \"chrome\"}",
"selenium/standalone-edge:4.4.0-20220812", "{\"browserName\": \"MicrosoftEdge\"}"
]
# URL for connecting to the docker daemon
# Most simple approach, leave it as http://127.0.0.1:2375, and mount /var/run/docker.sock.
# 127.0.0.1 is used because interally the container uses socat when /var/run/docker.sock is mounted
# If var/run/docker.sock is not mounted:
# Windows: make sure Docker Desktop exposes the daemon via tcp, and use http://host.docker.internal:2375.
# macOS: install socat and run the following command, socat -4 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock,
# then use http://host.docker.internal:2375.
# Linux: varies from machine to machine, please mount /var/run/docker.sock. If this does not work, please create an issue.
url = "http://127.0.0.1:2375"
# Docker image used for video recording
video-image = "selenium/video:ffmpeg-4.3.1-20220812"
# Uncomment the following section if you are running the node on a separate VM
# Fill out the placeholders with appropriate values
#[server]
#host = <ip-from-node-machine>
#port = <port-from-node-machine>
My problem is that the container generated dynamically has a UTC timezone instead of my timezone (America/Montreal) set in my environment variables.
To set the timezone for dynamic container we need de set "se:timeZone" in browser option capabilities.
Related
I have running a DDEV-Environment for Magento2, locally on my Mac OSX (Ventura)
https://ddev.readthedocs.io/en/stable/users/quickstart/#magento-2
For testing purpose I included Nifi per docker-compose.yaml inside my ddev project .ddev/docker-compose.nifi.yaml
Below you can see the docker-compose, which is really minimal at this point. Nifi works like expected, because I can login etc, although it is not persistent yet, but thats a different problem
version: '3'
services:
nifi:
image: apache/nifi:latest
container_name: ddev-${DDEV_SITENAME}-nifi
ports:
# HTTP
- "8080:8080"
# HTTPS
- "8443:8443"
volumes:
# - ./nifi/database_repository:/opt/nifi/nifi-current/database_repository
# - ./nifi/flowfile_repository:/opt/nifi/nifi-current/flowfile_repository
# - ./nifi/content_repository:/opt/nifi/nifi-current/content_repository
# - ./nifi/provenance_repository:/opt/nifi/nifi-current/provenance_repository
# - ./nifi/state:/opt/nifi/nifi-current/state
# - ./nifi/logs:/opt/nifi/nifi-current/logs
# - ./nifi/conf/login-identity-providers.xml:/opt/nifi/nifi-current/conf/login-identity-providers.xml
- ".:/mnt/ddev_config"
All I want to do is sending a POST-requst from Nifi to my Magento2 module.
I tried several IPs now, which I got from docker inspect ddev-ddev-magento2-web but I always receive "Connection refused"
My output from docker network ls:
NETWORK ID NAME DRIVER SCOPE
95bea4031396 bridge bridge local
692b58ca294e ddev-ddev-magento2_default bridge local
46be47991abe ddev_default bridge local
7e19ae1626f1 host host local
f8f4f1aeef04 nifi_docker_default bridge local
dbdba30546d7 nifi_docker_mynetwork bridge local
ca12e667b773 none null local
My Magento2-Module is working properly, because sending requests from Postmanto it works fine
You don't want most of what you have. Please remove the ports statement, which you shouldn't need at all; if you need anything, you'll need an expose. But I doubt you need that in this case?
You'll want to look at the docs:
Additional services and add-ons
Additional services with docker-compose
Then create a .ddev/docker-compose.nifi.yaml with something like
services:
nifi:
image: apache/nifi:latest
container_name: ddev-${DDEV_SITENAME}-nifi
container_name: "ddev-${DDEV_SITENAME}-someservice"
labels:
com.ddev.site-name: ${DDEV_SITENAME}
com.ddev.approot: ${DDEV_APPROOT}
expose:
- "8080"
environment:
- VIRTUAL_HOST=$DDEV_HOSTNAME
- HTTP_EXPOSE=8080:8080
- HTTPS_EXPOSE=9999:8080
volumes:
- ".:/mnt/ddev_config"
The name of the "web" container from inside your nifi container will be "web", curl http://web:8080, assuming that you have nifi on port 8080.
I don't know what you're trying to accomplish, but this may get you started. Feel free to come over to the DDEV Discord channel for more interactive help.
I am running automated UI tests using selenium grid 4 (doku)
I am trying to start my webdriver using a fake webcam (a .y4m File), in order to start the driver using the .y4m File, I need the file in my docker container.
My config.toml
[docker]
# Configs have a mapping between the Docker image to use and the capabilities that need to be matched to
# start a container with the given image.
configs = [
"selenium/standalone-firefox:4.1.4-20220427", "{\"browserName\": \"firefox\"}",
"selenium/standalone-chrome:4.1.4-20220427", "{\"browserName\": \"chrome\"}",
"selenium/standalone-edge:4.1.4-20220427", "{\"browserName\": \"MicrosoftEdge\"}"
]
# URL for connecting to the docker daemon
# Most simple approach, leave it as http://127.0.0.1:2375, and mount /var/run/docker.sock.
# 127.0.0.1 is used because interally the container uses socat when /var/run/docker.sock is mounted
# If var/run/docker.sock is not mounted:
# Windows: make sure Docker Desktop exposes the daemon via tcp, and use http://host.docker.internal:2375.
# macOS: install socat and run the following command, socat -4 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock,
# then use http://host.docker.internal:2375.
# Linux: varies from machine to machine, please mount /var/run/docker.sock. If this does not work, please create an issue.
url = "http://127.0.0.1:2375"
# Docker image used for video recording
video-image = "selenium/video:ffmpeg-4.3.1-20220427"
# Uncomment the following section if you are running the node on a separate VM
# Fill out the placeholders with appropriate values
#[server]
#host = <ip-from-node-machine>
#port = <port-from-node-machine>
And my docker-compose.yml
# To execute this docker-compose yml file use `docker-compose -f docker-compose-v3-dynamic-grid.yml up`
# Add the `-d` flag at the end for detached execution
# To stop the execution, hit Ctrl+C, and then `docker-compose -f docker-compose-v3-dynamic-grid.yml down`
version: "3"
services:
node-docker:
image: selenium/node-docker:4.1.4-20220427
volumes:
- ./assets:/opt/selenium/assets
- ./NodeDocker/config.toml:/opt/bin/config.toml
- /var/run/docker.sock:/var/run/docker.sock
depends_on:
- selenium-hub
environment:
- SE_EVENT_BUS_HOST=selenium-hub
- SE_EVENT_BUS_PUBLISH_PORT=4442
- SE_EVENT_BUS_SUBSCRIBE_PORT=4443
selenium-hub:
image: selenium/hub:4.1.4-20220427
container_name: selenium-hub
ports:
- "4442:4442"
- "4443:4443"
- "4444:4444"
those 2 Files are the exact same from the doku.
The grid works like this:
I am connecting a remoteDriver using C# to localhost:4444, the selenium-hub spawns a new selenium/standalone-chrome container using the config.toml.
My problem: I was not able to find any dokumentation on how to map a volume using the config.toml
I'm trying to implement this tutorial. The "docker-compose" content is this :
# WARNING: Do not deploy this tutorial configuration directly to a production environment
#
# The tutorial docker-compose files have not been written for production deployment and will not
# scale. A proper architecture has been sacrificed to keep the narrative focused on the learning
# goals, they are just used to deploy everything onto a single Docker machine. All FIWARE components
# are running at full debug and extra ports have been exposed to allow for direct calls to services.
# They also contain various obvious security flaws - passwords in plain text, no load balancing,
# no use of HTTPS and so on.
#
# This is all to avoid the need of multiple machines, generating certificates, encrypting secrets
# and so on, purely so that a single docker-compose file can be read as an example to build on,
# not use directly.
#
# When deploying to a production environment, please refer to the Helm Repository
# for FIWARE Components in order to scale up to a proper architecture:
#
# see: https://github.com/FIWARE/helm-charts/
#
version: "3.5"
services:
# Orion is the context broker
orion:
image: fiware/orion:latest
hostname: orion
container_name: fiware-orion
depends_on:
- mongo-db
networks:
- default
expose:
- "1026"
ports:
- "1026:1026"
command: -dbhost mongo-db -logLevel DEBUG
healthcheck:
test: curl --fail -s http://orion:1026/version || exit 1
interval: 5s
# Tutorial displays a web app to manipulate the context directly
tutorial:
image: fiware/tutorials.context-provider
hostname: iot-sensors
container_name: fiware-tutorial
networks:
- default
expose:
- "3000"
- "3001"
ports:
- "3000:3000"
- "3001:3001"
environment:
- "DEBUG=tutorial:*"
- "PORT=3000"
- "IOTA_HTTP_HOST=iot-agent"
- "IOTA_HTTP_PORT=7896"
- "DUMMY_DEVICES_PORT=3001"
- "DUMMY_DEVICES_API_KEY=4jggokgpepnvsb2uv4s40d59ov"
- "DUMMY_DEVICES_TRANSPORT=HTTP"
iot-agent:
image: fiware/iotagent-ul:latest
hostname: iot-agent
container_name: fiware-iot-agent
depends_on:
- mongo-db
networks:
- default
expose:
- "4041"
- "7896"
ports:
- "4041:4041"
- "7896:7896"
environment:
- "IOTA_CB_HOST=orion"
- "IOTA_CB_PORT=1026"
- "IOTA_NORTH_PORT=4041"
- "IOTA_REGISTRY_TYPE=mongodb"
- "IOTA_LOG_LEVEL=DEBUG"
- "IOTA_TIMESTAMP=true"
- "IOTA_MONGO_HOST=mongo-db"
- "IOTA_MONGO_PORT=27017"
- "IOTA_MONGO_DB=iotagentul"
- "IOTA_HTTP_PORT=7896"
- "IOTA_PROVIDER_URL=http://iot-agent:4041"
# Database
mongo-db:
image: mongo:3.6
hostname: mongo-db
container_name: db-mongo
expose:
- "27017"
ports:
- "27017:27017"
networks:
- default
command: --bind_ip_all --smallfiles
volumes:
- mongo-db:/data
healthcheck:
test: |
host=`hostname --ip-address || echo '127.0.0.1'`;
mongo --quiet $host/test --eval 'quit(db.runCommand({ ping: 1 }).ok ? 0 : 2)' && echo 0 || echo 1
interval: 5s
networks:
default:
ipam:
config:
- subnet: 172.18.1.0/24
volumes:
mongo-db: ~
But when I run the docker compose with the command "docker-compose up -d" I get this error :
*WARNING: The host variable is not set. Defaulting to a blank string.
Creating network "fiware_default" with the default driver
ERROR: Pool overlaps with other one on this address space*
I also get these networks by running the command "docker network ls" :
*NETWORK ID NAME DRIVER SCOPE
78403834b9bd bridge bridge local
1dc5b7d0534b hadig_default bridge local
4162244c37b0 host host local
ac5a94a89bde none null local*
I see no conflict with the name "fiware_default". where is the problem?
The "pool" the error message refers to is the 172.18.1.0/24 CIDR block that file manually specifies. If something else on your system is using that network space, it won't start up. (Docker might have assigned another Compose file's network to 172.18.0.0/16, for example.)
You don't usually need to manually specify IP addresses in Docker at all, and so you should remove that ipam: block. Having done that, you're telling Compose to configure the default network with default settings, and you can actually remove the entire networks: block at the end of the file.
The exception to this is if your host network environment is using some of the same IP address blocks, and then you do potentially need an override like this. If you run ifconfig or a similar command from the host (or look at your host's network settings from a desktop application) and your host or a VPN is using a 172.18.1.* address, you'll also get this message. In that case, change the network to something else; if you only need a /24 (254 addresses) then setting subnet: 192.168.123.0/24 (where "123" can be any number between 1 and 254) should get you past this.
I have a docker-compose file for my backend service as follows.
backend:
container_name: backend
build:
context: .
dockerfile: docker/dev.Dockerfile
# TODO: Try using cache_from pointed at a Docker Hub image built from master
# Though may need to also use the workaround for local vs remote:
# https://github.com/moby/moby/issues/32612#issuecomment-294055017
# cache_from: TODO
image: treeherder-backend
privileged: true
environment:
# Development/CI-specific environment variables only.
# Those that do not vary across environments should go in `Dockerfile`.
- BROKER_URL=amqp://guest:guest#rabbitmq//
- DATABASE_URL=${DATABASE_URL:-mysql://root#mysql/treeherder}
- GITHUB_TOKEN=${GITHUB_TOKEN:-}
- UPSTREAM_DATABASE_URL=${UPSTREAM_DATABASE_URL:-}
- PERF_SHERIFF_BOT_CLIENT_ID=${PERF_SHERIFF_BOT_CLIENT_ID:-}
- PERF_SHERIFF_BOT_ACCESS_TOKEN=${PERF_SHERIFF_BOT_ACCESS_TOKEN:-}
- PULSE_AUTO_DELETE_QUEUES=True
- REDIS_URL=redis://redis:6379
- SITE_URL=http://backend:8000/
- TREEHERDER_DEBUG=True
- NEW_RELIC_INSIGHTS_API_KEY=${NEW_RELIC_INSIGHTS_API_KEY:-}
- HOSTNAME=${HOSTNAME:-}
entrypoint: './docker/entrypoint.sh'
# We *ONLY* initialize the data when we're running the backend
command: './initialize_data.sh ./manage.py runserver 0.0.0.0:8000'
# Django's runserver doesn't listen to the default of SIGTERM, so docker-compose
# must send SIGINT instead to avoid waiting 10 seconds for the time out.
stop_signal: SIGINT
# https://github.com/SeleniumHQ/docker-selenium#running-the-images
shm_size: 2g
volumes:
- .:/app
ports:
- '8000:8000'
depends_on:
- mysql
- redis
- rabbitmq
stdin_open: true
tty: true
I want to name this container dynamically by picking up the hostname on which it runs and rename itself on that value.
Kindly do let me know if there are any ways to do this
Current hostname of container - Random string by docker
The desired hostname of container - $(whoami)-$(hostname) of the host system on which this will run
You can name your containers when you run them (you didn't specify how you run your containers, so I will assume you use docker run).
This is achieved using: docker run -d --name YOUR_NAME YOUR_IMAGE
This way you can have YOUR_NAME as a variable containing the fields you wanted, for example, running an ubuntu:latest container:
name=$(whoami)-$(uname -n)
docker run -d --name $name ubuntu:latest
In short:
I have a hard time figuring out how to set custom IP for a Solr container from the docker-compose.yml file.
Detailed
We want to deploy local dev environments, for Drupal instances, via Docker.
The propblem is, that while from the browser I can access the Solr server via the "traditional" http://localhost:8983/solr, Drupal cannot connect to it this way. The internal 0.0.0.0, and 127.0.0.1 doesn't work either. The only way Drupal can connect to the Solr server is via lan IP, which differs for every station obviously, and since the configuration in Drupal needs to be updated anyway, I thought that specifying a custom IP on which they can communicate would be my best choice, but it's not straightforward.
I am aware that assigning static IP to the container is not the best solution, but it seems more feasible than tinkering with solr.in.sh, and if someone has a different approach to achieve this, I am opened to solutions.
Most likely I could use some command line parameter along with docker run, but we need to run the containers with docker-compose up -d, so this wouldn't be an optimal solution.
Ideal would be a Solr container section example for the compose file. Thanks.
Note:
This link shows an example how to set it, but I can't understand it well. Please keep in mind that I am by no means an expert.
Forgot to mention that the host is based on Linux, mostly Ubuntu and Debian.
Edit:
As requested, here is my compose file:
version: "2"
services:
db:
image: wodby/drupal-mariadb
environment:
MYSQL_RANDOM_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
MYSQL_DATABASE: ${MYSQL_DATABASE}
MYSQL_USER: ${MYSQL_USER}
MYSQL_PASSWORD: ${MYSQL_PASSWORD}
# command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci # The simple way to override the mariadb config.
volumes:
- ./data/mysql:/var/lib/mysql
- ./docker-runtime/mariadb-init:/docker-entrypoint-initdb.d # Place init .sql file(s) here.
php:
image: wodby/drupal-php:7.0 # Allowed: 7.0, 5.6.
environment:
DEPLOY_ENV: dev
PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
PHP_XDEBUG_ENABLED: 1 # Set 1 to enable.
# PHP_SITE_NAME: dev
# PHP_HOST_NAME: localhost:8000
# PHP_DOCROOT: public # Relative path inside the /var/www/html/ directory.
# PHP_SENDMAIL_PATH: /usr/sbin/sendmail -t -i -S mailhog:1025
# PHP_XDEBUG_ENABLED: 1
# PHP_XDEBUG_AUTOSTART: 1
# PHP_XDEBUG_REMOTE_CONNECT_BACK: 0 # This is needed to respect remote.host setting bellow
# PHP_XDEBUG_REMOTE_HOST: "10.254.254.254" # You will also need to 'sudo ifconfig lo0 alias 10.254.254.254'
links:
- db
volumes:
- ./docroot:/var/www/html
nginx:
image: wodby/drupal-nginx
hostname: testing
environment:
# NGINX_SERVER_NAME: localhost
NGINX_UPSTREAM_NAME: php
# NGINX_DOCROOT: public # Relative path inside the /var/www/html/ directory.
DRUPAL_VERSION: 7 # Allowed: 7, 8.
volumes_from:
- php
ports:
- "${PORT_WEB}:80"
pma:
image: phpmyadmin/phpmyadmin
environment:
PMA_HOST: db
PMA_USER: ${MYSQL_USER}
PMA_PASSWORD: ${MYSQL_PASSWORD}
ports:
- '${PORT_PMA}:80'
links:
- db
mailhog:
image: mailhog/mailhog
ports:
- "8002:8025"
redis:
image: redis:3.2-alpine
# memcached:
# image: memcached:1.4-alpine
# memcached-admin:
# image: phynias/phpmemcachedadmin
# ports:
# - "8006:80"
solr:
image: makuk66/docker-solr:4.10.3
volumes:
- ./docker-runtime/solr:/opt/solr/server/solr/mycores
# entrypoint:
# - docker-entrypoint.sh
# - solr-precreate
ports:
- "8983:8983"
# varnish:
# image: wodby/drupal-varnish
# depends_on:
# - nginx
# environment:
# VARNISH_SECRET: secret
# VARNISH_BACKEND_HOST: nginx
# VARNISH_BACKEND_PORT: 80
# VARNISH_MEMORY_SIZE: 256M
# VARNISH_STORAGE_SIZE: 1024M
# ports:
# - "8004:6081" # HTTP Proxy
# - "8005:6082" # Control terminal
# sshd:
# image: wodby/drupal-sshd
# environment:
# SSH_PUB_KEY: "ssh-rsa ..."
# volumes_from:
# - php
# ports:
# - "8006:22"
A docker run example would be
IP_ADDRESS=$(hostname -I)
docker run -d -p 8983:8983 solr bin/solr start -h ${IP_ADDRESS} -p 8983
Instead of assigning static IPs, you could use the following method to get the container's IP dynamically.
When you link containers together, they share there network information (IP, port) to each other. The information is stored in each container as environmental variables.
Example
docker-compose.yml
service:
build: .
links:
- redis
ports:
- "3001:3001"
redis:
build: .
ports:
- "6369:6369"
The service container will now have the following environmental variables:
Dynamic IP Address Stored Within "service" container:
REDIS_PORT_6379_TCP_ADDR
Dynamic PORT Stored Within "service" container:
REDIS_PORT_6379_TCP_PORT
You can always check this out by shelling into the container and looking yourself.
docker exec -it [ContainerID] bash
printenv
Inside your nodeJS app you can use the environmental variable in your connection function by using process.env.
let client = redis.createClient({
port: process.env.REDIS_PORT_6379_TCP_ADDR,
host: process.env.REDIS_PORT_6379_TCP_PORT
});
Edit
Here is the updated docker-compose.yml "solr" section:
solr:
image: makuk66/docker-solr:4.10.3
volumes:
- ./docker-runtime/solr:/opt/solr/server/solr/mycores
entrypoint:
- docker-entrypoint.sh
- solr-precreate
ports:
- "8983:8983"
links:
- db
In the above example the "solr" container is now linked with the "db" container. this is done using the "links" field.
You can do the same thing if you wanted to link the solr container to any other container within the docker-compose.yml file.
The db containers information will now be available to the solr container (via the enviromental variables I mentioned earlier).
Without the linking, you will not see those enviromental variables listed when you do the printenv command.