Running Karate UI tests with “driverTarget” in GitLab CI - docker

Question was:
I would like to run Karate UI tests using the driverTarget options to test my Java Play app which is running locally during the same job with sbt run.
I have a simple assertion to check for a property but whenever the tests runs I keep getting "description":"TypeError: Cannot read property 'getAttribute' of null This is my karate-config.js:
if (env === 'ci') {
karate.log('using environment:', env);
karate.configure('driverTarget',
{
docker: 'justinribeiro/chrome-headless',
showDriverLog: true
});
}
This is my test scenario:
Scenario: test 1: some test
Given driver 'http://localhost:9000'
waitUntil("document.readyState == 'complete'")
match attribute('some selector', 'some attribute') == 'something'
My guess is that because justinribeiro/chrome-headless is running in its own container, localhost:9000 is different in the container compared to what's running outside of it.
Is there any workaround for this? thanks

A docker container cannot talk to localhost port as per what was posted: "My guess is that because justinribeiro/chrome-headless is running in its own container, localhost:9000 is different in the container compared to what's running outside of it."
To get around this and have docker container communicate with running app on localhost port use command host.docker.internal
Change to make:
From: Given driver 'http://localhost:9000'.
To: Given driver 'http://host.docker.internal:9000'

Additionally, I was able to use the ptrthomas/karate-chrome image in CI (GITLAB) by inserting the following inside my gitlab-ci.yml file
stages:
- uiTest
featureOne:
stage: uiTest
image: docker:latest
cache:
paths:
- .m2/repository/
services:
- docker:dind
script:
- docker run --name karate --rm --cap-add=SYS_ADMIN -v "$PWD":/karate -v
"$HOME"/.m2:/root/.m2 ptrthomas/karate-chrome &
- sleep 45
- docker exec -w /karate karate mvn test -DargLine='-Dkarate.env=docker' Dtest=testParallel
allow_failure: true
artifacts:
paths:
- reports
- ${CLOUD_APP_NAME}.log
my karate-config.js file looks like
if (karate.env == 'docker') {
karate.configure('driver', {
type: 'chrome',
showDriverLog: true,
start: false,
beforeStart: 'supervisorctl start ffmpeg',
afterStop: 'supervisorctl stop ffmpeg',
videoFile: '/tmp/karate.mp4'
});
}

Related

VSCode devcontainer: proxy: unknown scheme: http

i have some troubles starting devcontainers in VSCode.
My setup:
macOS (Monterey 12.6.2)
docker-ce on local VM (DOCKER_HOST: ssh://vagrant#127.0.0.1:2222)
corporate proxy configured via http_proxy & https_proxy & no_proxy env variables (also the uppercased variants)
VSCode (1.74.3)
no Docker-Desktop
I have created a very small example with just the devcontainer definition. Nothing else is in the project folder:
# .devcontainer/devcontainer.json
{
"name": "Example DevContainer",
"image": "mcr.microsoft.com/devcontainers/base:jammy",
"customizations": {
"vscode": {
"settings": {
"docker.environment": {
"DOCKER_HOST": "ssh://vagrant#127.0.0.1:2222"
}
}
}
}
}
when i now start the dev container using: Dev Containers: Rebuild and Reopen in Container (or any other dev container related command) i receive only the following error:
[650908 ms] Start: Run: docker version --format {{.Server.APIVersion}}
[651092 ms] Stop (184 ms): Run: docker version --format {{.Server.APIVersion}}
[651093 ms] proxy: unknown scheme: http
I have configured VSCode to use the vm as docker host:
Im not sure where to start troubleshooting. Any ideas?
EDIT: Interesting side fact: if i run Dev Containers: Try a Dev Container Sample... the sample dev container starts building, but seems to fail since the corporate proxy is not configured in the image, but at least ist starts to build and not aborts directly with the error above.
Dev Containers: Create Dev Container... immediately fails too with the error above.

Provide port to elasticsearch in gitlab ci services for integration tests

I have a code repo in which I want to support both ES 5 and ES 7 versions. So the code is written in a way to detect which ES version is being run via env variables and use the connectors accordingly. While writing integration tests for this that run in the gitlab ci, I want to run both ES5 and ES7 to test both pieces of connectors. I have defined my ES images like so
- name: docker.elastic.co/elasticsearch/elasticsearch:5.6.0
alias: elastic
command:
- /bin/env
- 'discovery.type=single-node'
- 'xpack.security.enabled=false'
- /bin/bash
- bin/es-docker
- name: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
alias: elastic7
command:
- port 9202
- /bin/env
- 'discovery.type=single-node'
- 'xpack.security.enabled=false'
- 'http.port=9202'
- /bin/bash
- bin/es-docker
But my ES7 is not able to run on the given port and i get errors like so in my coverage stage :
*** WARNING: Service runner-ssasmqme-project-5918-concurrent-0-934a00b2fdd833db-docker.elastic.co__elasticsearch__elasticsearch-4 probably didn't start properly.
Health check error:
service "runner-ssasmqme-project-5918-concurrent-0-934a00b2fdd833db-docker.elastic.co__elasticsearch__elasticsearch-4-wait-for-service" timeout
Health check container logs:
Service container logs:
2023-02-07T07:59:39.562478117Z /usr/local/bin/docker-entrypoint.sh: line 37: exec: port 9202: not found
How do I run multiple ES versions on different ports in my gitlab ci ?

Karate-Chrome docker container in azure dev ops failing to connect

I have seen many similar issues to this but none seem to resolve or describe my exact issue.
I have configured an azure devops pipeline to use a container like below:
container:
image: ptrthomas/karate-chrome
options: --cap-add=SYS_ADMIN
I have uploaded the contents of the example from the jobserver demo to a repository and then run the following:
steps:
- script: mvn clean test -DargLine='-Dkarate.env=docker' -Dtest=WebRunner
It is my understanding (and I can see from the logs) that the files are loaded into the container and the script command is being executed inside the container. So that script command is the equivalent of docker exec -it -w /src karate mvn clean test -DargLine='-Dkarate.env=docker' -Dtest=WebRunner just without having to exec into the container.
When I run the example locally it executes the tests with no issues but in azure dev ops it fails at the point the tests actually start running, throwing this error:
14:16:37.388 [main] ERROR com.intuit.karate - karate.org.apache.http.conn.HttpHostConnectException: Connect to
localhost:9222 [localhost/127.0.0.1] failed: Connection refused
(Connection refused), http call failed after 2 milliseconds for url:
http://localhost:9222/json 14:16:39.388 [main] DEBUG
com.intuit.karate.shell.Command - attempt #4 waiting for http to be
ready at: http://localhost:9222/json 14:16:39.391 [main] DEBUG
com.intuit.karate - request: 5 > GET http://localhost:9222/json 5 >
Host: localhost:9222 5 > Connection: Keep-Alive 5 > User-Agent:
Apache-HttpClient/4.5.13 (Java/1.8.0_275) 5 > Accept-Encoding:
gzip,deflate
Looking at other issues there have been suggestions to specify the driver in the feature files with this line:
* configure driver = { type: 'chrome', executable: 'chrome' }
but a) that hasn't worked for me and b) shouldn't the karate-chrome docker image render this configuration unnecessary as it should be no different than the container I run locally?
Any help appreciated!
Thanks
Only thing I can think of is that the Azure config does not call the ENTRYPOINT of the image.
Maybe you should try to create a container from scratch (that does extensive logging) and see what happens. Use the Karate one as a reference.

Github Action with selenium and docker

I am currently working on a CI pipeline for a project and just started setting up a Github Action for running integration tests but I can't get it to work.
My action looks like this:
name: Integration Tests
on:
push:
branches:
- main
workflow_dispatch:
jobs:
integration-tests:
runs-on: ubuntu-latest
services:
selenium:
image: selenium/standalone-chrome:latest
ports:
- 4444:4444
options: --shm-size="2g"
steps:
- uses: actions/checkout#v2
- name: Get IP Address
run: echo "##[set-output name=ip;]$(ifconfig eth0 | grep 'inet [0-9\.]* ' -o | sed 's/[^0-9\.]//g')"
id: ip_addr
- name: Setup Pyhon
uses: actions/setup-python#v2
with:
python-version: '3.8'
- name: Install Python dependencies
uses: py-actions/py-dependency-install#v2
with: /path/to/requirements
- name: Run Tests
run: python3 main.py --backend http://localhost:8080/ --frontend http://${{ steps.ip_addr.outputs.ip }}:4200 --selenium http://localhost:4444/wd/hub
main.py starts two docker containers (that expose their corresponding ports) and runs a suite of selenium tests. It works on my local machine with a container I run using docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-chrome:latest. It takes the url for the backend and the one for the frontend, which gets used by selenium.
I think I have to use the IP address of the runner so selenium can access the site (since it runs locally, localhost doesn't work), but it fails with the following error:
File "main.py", line 57, in <module>
testcase.run()
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_cases/t1.py", line 14, in run
self.web_driver.accept_cookies()
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_utils/parkview_webdriver.py", line 64, in accept_cookies
self.wait_and_click(By.ID, 'confirmCookies')
File "/home/runner/work/PROJECT_NAME/QualityAssurance/integration_tests/test_utils/parkview_webdriver.py", line 30, in wait_and_click
WebDriverWait(self.driver, 10).until(
File "/opt/hostedtoolcache/Python/3.8.11/x64/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
It seems to just not load the time, but I don't know to fix this. I guess it could also be that I messed up the networking somehow?
I think the service container is referenced by the service name instead of localhost. i.e. in your example:
--selenium http://localhost:4444/wd/hub
would be:
--selenium http://selenium:4444/wd/hub
as you defined it:
services:
selenium:

How to run Kafka from Jenkins Pipeline using Groovy and Kubernetes plugin?

I couldn't find such a specific command around the internet so I kindly ask for your help with this one :)
Context
I have defined a podTemplate with a few containers, by using the containerTemplate methods:
ubuntu:trusty (14.04 LTS)
postgres:9.6
and finally, wurstmeister/kafka:latest
Doing some Groovy coding in Pipeline, I install several dependencies into my ubuntu:trusty container, such as latest Git, Golang 1.9, etc., and I also checkout my project from Github.
After all that dependencies are dealt with, I manage to compile, run migrations (which means Postgres is up and running and my app is connected to it), and spin up my app just fine until it complains that Kafka is not running because it couldn't connect to any broker.
Debugging sessions
After some debug sessions I have ps aux'ed each and every container to make sure all the services I needed were running in their respective containers, such as:
container(postgres) {
sh 'ps aux' # Show Postgres, as expected
}
container(linux) {
sh 'ps aux | grep post' # Does not show Postgres, as expected
sh 'ps aux | grep kafka' # Does not show Kafka, as expected
}
container(kafka) {
sh 'ps aux' # Does NOT show any Kafka running
}
I have also exported KAFKA_ADVERTISED_HOST_NAME var to 127.0.0.1 as explained in the image docs, without success, with the following code:
containerTemplate(
name: kafka,
image: 'wurstmeister/kafka:latest',
ttyEnabled: true,
command: 'cat',
envVars: [
envVar(key: 'KAFKA_ADVERTISED_HOST_NAME', value: '127.0.0.1'),
envVar(key: 'KAFKA_AUTO_CREATE_TOPICS_ENABLE', value: 'true'),
]
)
Questions
This image documentation details https://hub.docker.com/r/wurstmeister/kafka/ is explicit about starting a Kafka cluster with docker-compose up -d
1) How do I actually do that with this Kubernetes plugin + Docker + Groovy + Pipeline combo in Jenkins?
2) Do I actually need to do that? Postgres image docs (https://hub.docker.com/_/postgres/) also mentions about running the instance with docker run, but I didn't need to do that at all, which makes me think that containerTemplate is probably doing it automatically. So why is it not doing this for the Kafka container?
Thanks!
So... problem is with this image, and way how kubernetes works with them.
Kafka does not start because you override dockers CMD with command:'cat' which causes start-kafka.sh to never run.
Because of above I suggest using different image. Below template worked for me.
containerTemplate(
name: 'kafka',
image: 'quay.io/jamftest/now-kafka-all-in-one:1.1.0.B',
resourceRequestMemory: '500Mi',
ttyEnabled: true,
ports: [
portMapping(name: 'zookeeper', containerPort: 2181, hostPort: 2181),
portMapping(name: 'kafka', containerPort: 9092, hostPort: 9092)
],
command: 'supervisord -n',
envVars: [
containerEnvVar(key: 'ADVERTISED_HOST', value: 'localhost')
]
),

Resources