Selenium Selenoid File Server not running in Browser Container - docker

selenium is unable to download any files from the browsers due to a 502 error on my coworkers machine, none of my other coworkers are seeing the issue, just this one dude. We are using Firefox.
After looking at the Selenoid code a bit I learned that the containers the Browser runs in uses a File Server on port 8080 to allow downloading files from the container, but I discovered that this File Server is not running within these containers.
I verified this through this command:
docker exec -it <browser_container> curl 127.0.0.1:8080
On my machine I get a 200 response:
<pre>
test.xlsx
</pre>
But when I run this command on his machine I get this error:
Failed to connect to 127.0.0.1 port 8080 after 8 ms: Connection refused
This is indicative that the File Server is not running within his Browser Containers. I've been trying many different firefox arguments and I've restart selenoid and the docker containers and still can't figure out what's going on, I'm completely lost right now. If anyone knows what might be going on I would be appreciative, or even if anyone has any idea how to gain more information into what's going on.
Here is the Firefox options we are using
options = webdriver.FirefoxOptions()
options.add_argument('--width=1600')
options.add_argument('--height=900')
options.set_preference('browser.download.dir', '/home/selenium/Downloads')
And our browsers.json file
{
"chrome": {
"default": "105.0",
"versions": {
"105.0": {
"image": "selenoid/vnc_chrome:105.0",
"port": "4444",
"path": "/",
"env": ["TZ=America/Denver"]
}
},
"caps": {
"loggingPrefs": {"browser": "ALL"},
"enableVNC": true,
"browserName": "chrome",
"timeZone": "America/Denver",
"sessionTimeout": "1m30s"
}
},
"firefox": {
"default": "latest",
"versions": {
"latest": {
"image": "selenoid/firefox",
"port": "4444",
"path": "/wd/hub",
"env": ["TZ=America/Denver"]
}
},
"caps": {
"loggingPrefs": {"browser": "ALL"},
"enableVNC": true,
"browserName": "firefox",
"timeZone": "America/Denver",
"sessionTimeout": "1m30s"
}
}
}
We do have a custom docker-compose.yml file for starting the selenoid and selenoid_ui containers, here is the file just in case that setup matters, I doubt the issue lies here.
version: "3.9"
networks:
selenoid_net:
name: selenoid_net
attachable: true
ipam:
config:
- subnet: 172.198.1.0/24
services:
selenoid:
image: aerokube/selenoid
restart: always
networks:
selenoid_net:
ports:
- "4444:4444"
environment:
- OVERRIDE_VIDEO_OUTPUT_DIR=${VIDEO_OUTPUT}/video
- TZ=America/Denver
volumes:
- "/etc/selenoid:/etc/selenoid"
- "/var/run/docker.sock:/var/run/docker.sock"
- "${VIDEO_OUTPUT}/video:${VIDEO_OUTPUT}/video"
- "${VIDEO_OUTPUT}/logs:${VIDEO_OUTPUT}/logs"
- "${PWD}:/etc/browsers"
command: ["-conf", "/etc/browsers/browsers.json",
"-video-output-dir", "${VIDEO_OUTPUT}/video",
"-log-output-dir", "${VIDEO_OUTPUT}/logs",
"-limit", "6",
"-timeout", "1m30s","-container-network", 'selenoid_net']
selenoid-ui:
image: "aerokube/selenoid-ui:latest"
restart: always
networks:
selenoid_net:
links:
- "selenoid"
ports:
- "8080:8080"
command: ["--selenoid-uri", "http://selenoid:4444"]

Related

Bad Gateway when using Traefik in docker swarm

I'm currently struggling a lot to spin up a small traefik example on my docker swarm instance.
I started first with an docker-compose file for local development and everything is working as expected.
But when I define this as swarm file to bring that environment into production I always get an Bad Gateway from traefik.
After searching a lot about this it seems to be related to an networking issue from traefik since it tries to request between two different networks, but I'm not able to find the issue.
After certain iterations I tried to reproduce the Issue with "official" containers to provide an better example for other people.
So this is my traefik.yml
version: "3.7"
networks:
external:
external: true
services:
traefik:
image: "traefik:v2.8.1"
command:
- "--log.level=INFO"
- "--accesslog=true"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.swarmMode=true"
- "--providers.docker.exposedbydefault=false"
- "--providers.docker.network=external"
- "--entrypoints.web.address=:80"
- "--entrypoints.web.forwardedHeaders.insecure"
ports:
- "80:80"
- "8080:8080"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- external
deploy:
placement:
constraints: [node.role == manager]
host-app:
image: traefik/whoami
ports:
- "9000:80"
networks:
- external
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.host-app.rule=PathPrefix(`/whoami`)"
- "traefik.http.services.host-app.loadbalancer.server.port=9000"
- "traefik.http.routers.host-app.entrypoints=web"
- "traefik.http.middlewares.host-app-stripprefix.stripprefix.prefixes=/"
- "traefik.http.routers.host-app.middlewares=host-app-stripprefix#docker"
- "traefik.docker.network=external"
The network is created with: docker network create -d overlay external
and I deploy the stack with docker stack deploy -c traefik.yml server
Until here no issues and everything spins up fine.
When I curl localhost:9000 I get the correct response:
curl localhost:9000
Hostname: 7aa77bc62b44
IP: 127.0.0.1
IP: 10.0.0.8
IP: 172.25.0.4
IP: 10.0.4.6
RemoteAddr: 10.0.0.2:35068
GET / HTTP/1.1
Host: localhost:9000
User-Agent: curl/7.68.0
Accept: */*
but on
curl localhost/whoami
Bad Gateway%
I always get the bad Gateway issue.
So I checked my network with docker network inspect external to ensure that both are running in the same network and this is the case.
[
{
"Name": "external",
"Id": "iianul6ua9u1f1bb8ibsnwkyc",
"Created": "2022-08-09T19:32:01.4491323Z",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.4.0/24",
"Gateway": "10.0.4.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"7aa77bc62b440e32c7b904fcbd91aea14e7a73133af0889ad9e0c9f75f2a884a": {
"Name": "server_host-app.1.m2f5x8jvn76p2ssya692f4ydp",
"EndpointID": "5d5175b73f1aadf2da30f0855dc0697628801a31d37aa50d78a20c21858ccdae",
"MacAddress": "02:42:0a:00:04:06",
"IPv4Address": "10.0.4.6/24",
"IPv6Address": ""
},
"e23f5c2897833f800a961ab49a4f76870f0377b5467178a060ec938391da46c7": {
"Name": "server_traefik.1.v5g3af00gqpulfcac84rwmnkx",
"EndpointID": "4db5d69e1ad805954503eb31c4ece5a2461a866e10fcbf579357bf998bf3490b",
"MacAddress": "02:42:0a:00:04:03",
"IPv4Address": "10.0.4.3/24",
"IPv6Address": ""
},
"lb-external": {
"Name": "external-endpoint",
"EndpointID": "ed668b033450646629ca050e4777ae95a5a65fa12a5eb617dbe0c4a20d84be28",
"MacAddress": "02:42:0a:00:04:04",
"IPv4Address": "10.0.4.4/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4100"
},
"Labels": {},
"Peers": [
{
"Name": "3cb3e7ba42dc",
"IP": "192.168.65.3"
}
]
}
]
and by checking the traefik logs I get the following
10.0.0.2 - - [09/Aug/2022:19:42:34 +0000] "GET /whoami HTTP/1.1" 502 11 "-" "-" 4 "host-app#docker" "http://10.0.4.9:9000" 0ms
which is the correct server:port for the whoami service. And even connecting into the traefik container and ping 10.0.4.9 works fine.
PING 10.0.4.9 (10.0.4.9): 56 data bytes
64 bytes from 10.0.4.9: seq=0 ttl=64 time=0.066 ms
64 bytes from 10.0.4.9: seq=1 ttl=64 time=0.057 ms
^C
--- 10.0.4.9 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.057/0.061/0.066 ms
This logs and snippets are all on my local swarm on Docker for Windows with wsl2 Ubuntu distribution. But I tested this on an CentOS Swarm which can be requested within my company and also with https://labs.play-with-docker.com/ and leads all to the same error.
So please can anybody tell me what configuration I'm missing or what mistake I made to get this running?
After consulting a coworker and creating another example we finally found the solution by our self.
Its just my own failure that I used the published port for loadbalancing the traefik to the service which is wrong.
host-app:
image: traefik/whoami
ports:
- "9000:80"
networks:
- external
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.host-app.rule=PathPrefix(`/whoami`)"
- "traefik.http.services.host-app.loadbalancer.server.port=80" # <--- this was wrong
- "traefik.http.routers.host-app.entrypoints=web"
- "traefik.http.middlewares.host-app-stripprefix.stripprefix.prefixes=/"
- "traefik.http.routers.host-app.middlewares=host-app-stripprefix#docker"
- "traefik.docker.network=external"
and that's the reason for the Bad Gateway since traefik tries to reach the published port from the server which is not present internally.

Caddy as reverse proxy in docker refuses to connect to other containers

I wanted to try out Caddy in a docker environment but it does not seem to be able to connect to other containers. I created a network "caddy" and want to run a portainer alongside it. If I go into the volume of caddy, I can see, that there are certs generated, so that seems to work. Also portainer is running and accessible via the Server IP (http://65.21.139.246:1000/). But when I access via the url: https://smallhetzi.fading-flame.com/ I get a 502 and in the log of caddy I can see this message:
{
"level": "error",
"ts": 1629873106.715402,
"logger": "http.log.error",
"msg": "dial tcp 172.20.0.2:1000: connect: connection refused",
"request": {
"remote_addr": "89.247.255.231:15146",
"proto": "HTTP/2.0",
"method": "GET",
"host": "smallhetzi.fading-flame.com",
"uri": "/",
"headers": {
"Accept-Encoding": [
"gzip, deflate, br"
],
"Accept-Language": [
"de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
],
"Cache-Control": [
"max-age=0"
],
"User-Agent": [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.159 Safari/537.36"
],
"Sec-Fetch-Site": [
"none"
],
"Accept": [
"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
],
"Sec-Fetch-Mode": [
"navigate"
],
"Sec-Fetch-User": [
"?1"
],
"Sec-Fetch-Dest": [
"document"
],
"Sec-Ch-Ua": [
"\"Chromium\";v=\"92\", \" Not A;Brand\";v=\"99\", \"Google Chrome\";v=\"92\""
],
"Sec-Ch-Ua-Mobile": [
"?0"
],
"Upgrade-Insecure-Requests": [
"1"
]
},
"tls": {
"resumed": false,
"version": 772,
"cipher_suite": 4865,
"proto": "h2",
"proto_mutual": true,
"server_name": "smallhetzi.fading-flame.com"
}
},
"duration": 0.000580828,
"status": 502,
"err_id": "pq78d9hen",
"err_trace": "reverseproxy.statusError (reverseproxy.go:857)"
}
But two compose files:
Caddy:
version: '3.9'
services:
caddy:
image: caddy:2-alpine
container_name: caddy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- certs-volume:/data
- caddy_config:/config
volumes:
certs-volume:
caddy_config:
networks:
default:
external:
name: caddy
Caddyfile:
{
email simonheiss87#gmail.com
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
smallhetzi.fading-flame.com {
reverse_proxy portainer:1000
}
and my portainer file:
version: '3.9'
services:
portainer:
image: portainer/portainer-ce
container_name: portainer
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- portainer_data:/data portainer/portainer
entrypoint: /portainer -p :80
ports:
- "1000:80"
volumes:
portainer_data:
networks:
default:
external:
name: caddy
What I think happens is, that those two containers are somehow not in the same network, but I dont get why.
What works as a workaround right now is, when i make this change to my Caddyfile:
smallhetzi.fading-flame.com {
reverse_proxy 65.21.139.246:1000
}
Then I get a valid certificate and the portainer ui. But i would rather not spread the IPs over my Caddyfile. Do I have to configure something else for caddy to run in docker?
I just got help from the forum and it turns out, that caddy redirects to the port INSIDE the container, not the public one. In my case, portainer runs on 80 internally, so changing the Caddyfile to this:
smallhetzi.fading-flame.com {
reverse_proxy portainer:80
}
or this
smallhetzi.fading-flame.com {
reverse_proxy http://portainer
}
does the job. This also means, that I could get rid of exposing portainer directly over the port 1000. Now I can only access it via the proxy.
Hope someone gets some help from that :)

how to initial setup consul with defined key/value

I have setup docker config using docker compose.
this is part of docker compose file
version: '3'
networks:
pm:
services:
consul:
container_name: consul
image: consul:latest
restart: unless-stopped
ports:
- 8300:8300
- 8301:8301
- 8302:8302
- 8400:8400
- 8500:8500
- 8600:8600
environment:
CONSUL_LOCAL_CONFIG: >-
{
"bootstrap": true,
"server": true,
"node_name": "consul1",
"bind_addr": "0.0.0.0",
"client_addr": "0.0.0.0",
"bootstrap_expect": 1,
"ui": true,
"addresses" : {
"http" : "0.0.0.0"
},
"ports": {
"http": 8500
},
"log_level": "DEBUG",
"connect" : {
"enabled" : true
}
}
volumes:
- ./data:/consul/data
command: agent -server -bind 0.0.0.0 -client 0.0.0.0 -bootstrap-expect=1
Then set the key value via browser
I would like to add the key/value as initial at new environment, so that additional setup steps at browser could be avoided.
this is the configuration i export by using consul kv command:
# consul kv export config/
[
{
"key": "config/",
"flags": 0,
"value": ""
},
{
"key": "config/drug2/",
"flags": 0,
"value": ""
},
{
"key": "config/drug2/data",
"flags": 0,
"value": "e30="
}
]
To my knowledge Docker Compose does not have a way to run a custom command/script after the containers have started.
As a workaround you could write a shell script which executes docker-compose up and then either runs consul kv import or a curl command against Consul's Transaction API to add the data you're trying to load.

AWS ECS containers are not connecting but works perfectly in my local machine

I have an application(runs at http://localhost:8080) that talks to a backend api which runs at http://localhost:8081. I have dockerized the frontend and the backend separately and running them through docker-compose locally works perfectly without any issues. But, when I run it in ECS, the frontend couldn't find http://localhost:8081(backend).
I am using an AutoScaling group with an Elastic Load Balancer and I have my both containers defined in a single Task Definition. Also, I have the backend linked to the front end. When I ssh into my ECS instance and run docker ps -a i can see both of my containers are running at the correct ports exactly like in my local machine(Result of docker ps -a) and I can successfully ping each of them from one container to the other.
Task Definition
"CartTaskDefinition": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"ContainerDefinitions": [
{
"Name": "cs-cart",
"Image": "thishandp7/cs-cart",
"Memory": 400,
"PortMappings":[
{
"ContainerPort": "8080",
"HostPort": "8080"
}
],
"Links": [
"cs-server"
]
},
{
"Name": "cs-server",
"Image": "thishandp7/cs-server",
"Memory": 450,
"PortMappings":[
{
"ContainerPort": "8081",
"HostPort": "8081"
}
],
}
]
}
}
Listeners in my ElasticLoadBalancer,
The first listener is for the frontend and the second one is for the backend
"Listeners" : [
{
"LoadBalancerPort": 80,
"InstancePort": 8080,
"Protocol": "http"
},
{
"LoadBalancerPort": 8081,
"InstancePort": 8081,
"Protocol": "tcp"
}
],
EC2 instacne security Group Ingress rules:
"SecurityGroupIngress" : [
{
"IpProtocol" : "tcp",
"FromPort" : 8080,
"ToPort" : 8080,
"SourceSecurityGroupId" : { "Ref": "ElbSecurityGroup" }
},
{
"IpProtocol" : "tcp",
"FromPort" : 8081,
"ToPort" : 8081,
"SourceSecurityGroupId" : { "Ref": "ElbSecurityGroup" }
},
{
"IpProtocol" : "tcp",
"FromPort" : 22,
"ToPort" : 22,
"CidrIp" : "0.0.0.0/0"
}
],
Docker Compose
version: "3.5"
services:
cart:
build:
context: ..
dockerfile: docker/Dockerfile
args:
APP_LOCATION: /redux-saga-cart/
PORT: 8080
networks:
- server-cart
ports:
- 8080:8080
depends_on:
- server
server:
build:
context: ..
dockerfile: docker/Dockerfile
args:
APP_LOCATION: /redux-saga-shopping-cart-server/
PORT: 8081
ports:
- 8081:8081
networks:
- server-cart
networks:
server-cart:
Quick update: I have tried it with awsvpc network mode with application load balancer. Still not working
Thanks in advance.
What kind of Docker Network mode are you using(Brdige/Host) on ECS?. I don't think localhost will work properly on ECS containers. I had same issue so I used private IP or DNS name of EC2 host for my communication as temp testing purpose. Ex - http://10.0.1.100:8081.
Note - Please make sure to give security group rule to allow 8081 traffic from within EC2(Edit EC2 security group to allow 8081 from same sgid as source).
For Production deployments, I would recommend to use a service discovery to identify the backend service(Consul by Hashicorp) or AWS Private Service Discovery on ECS.
-- Update --
Since you are running both containers under same task def(under same ECS service), so typically ECS will bring both docker containers on same host. Do something like following.
By default ECS brings containers using Bridge mode on Linux.
You should be able to have each containers communicate using Docker Gateway IP - 172.17.0.1 on Linux. So for your case, try configuring http://172.17.0.1:8081

docker-compose referencing wrong ports

I am running multiple docker-compositions on one host (identical images for different usecases).
For that reason I use different HTTPS (+REST) ports the compositions are available remotely under. However, docker will reference the port range of the first composition in every other composition as well, but not use it. Although I cannot see any negative implication atm, I would like to get rid of it, fearing that some implication might eventually arise.
docker ps shows this
PORTS
Second container:
**8643-8644/tcp**, 0.0.0.0:8743-8744->8743-8744/tcp
0.0.0.0:27020->27020/tcp
First container:
0.0.0.0:**8643-8644->8643-8644**/tcp
0.0.0.0:27019->27019/tcp
First docker-compose file (excerpt):
version: '2'
services:
mongo:
image: *****
ports:
- "27019:27019"
tty: true
volumes:
- /data/mongodb
- /data/db
- /var/log/mongodb
entrypoint: [ "/usr/bin/mongod", "--port", "27019" ]
rom:
image: *****
links:
- mongo
ports:
- "8643:8643"
- "8644:8644"
environment:
WEB_PORT_SECURE: 8643
REST_PORT_SECURE: 8644
MONGO_PORT: 27019
MONGO_INST: mongod
entrypoint: [ "node", "/usr/src/app/app.js" ]
Second docker-compose file (excerpt):
version: '2'
services:
mongo:
image: *****
ports:
- "27020:27020"
tty: true
volumes:
- /data/mongodb
- /data/db
- /var/log/mongodb
entrypoint: [ "/usr/bin/mongod", "--port", "27020" ]
rom:
image: *****
links:
- mongo
ports:
- "8743:8743"
- "8744:8744"
environment:
WEB_PORT_SECURE: 8743
REST_PORT_SECURE: 8744
MONGO_PORT: 27020
MONGO_INST: mongod
entrypoint: [ "node", "/usr/src/app/app.js" ]
and finally, docker inspect shows this for the second container
"Config": {
"Hostname": *****,
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"8643/tcp": {},
"8644/tcp": {},
"8743/tcp": {},
"8744/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"MONGO_PORT=27020",
"MONGO_INST=mongodb",
"WEB_PORT_SECURE=8743",
"REST_PORT_SECURE=8744",
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"NPM_CONFIG_LOGLEVEL=info",
"NODE_VERSION=4.7.2",
"WORK_DIR=/usr/src/app"
],
"NetworkSettings": {
"Ports": {
"8643/tcp": null,
"8644/tcp": null,
"8743/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8743"
}
],
"8744/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8744"
}
]
},
The last block clearly shows that docker is not doing anything with the 8643 and 8644 port, but still references it there.
"8643/tcp": null,
"8644/tcp": null,
Any idea why this happens and how to avoid it?
They are there because the image exposes them (built with EXPOSE).
This is not a problem, it's totally normal. You won't have a problem unless you try to export the same port on the outside host more than once. Here, none of your exported ports are in conflict.
0.0.0.0:8743-8744->8743-8744/tcp
0.0.0.0:27020->27020/tcp
0.0.0.0:8643-8644->8643-8644/tcp
0.0.0.0:27019->27019/tcp
You are exporting 8643-8644, 8743-8744, 27019, 27020. No conflicts.
A container can expose whatever ports it wants, it is only important that exposed ports are not in conflict with one another.

Resources