GitHub Actions - Running DotNetCore Tests with Neo4j service dependency - docker

I would need some help with steps to run integration tests on GitHub. My project needs Neo4j so I have declared neo4j as a service. However while running my test project, I am seeing the following error
Connection with the server breaks due to ExtendedSocketException:
Resource temporarily unavailable Please ensure that your database is listening on the correct
host and port and that you have compatible encryption settings both on Neo4j server and driver.
Note that the default encryption setting has changed in Neo4j 4.0.
I am not sure if the issue is with how I am specifying the actions or something to do with Neo4j.
Here's my GitHub CI File: dotnetcore.yml
name: .NET Core
on:
push:
branches: [ master, GithubActions ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
env:
NEO4J_HOST: neo4j
# Service containers to run with `container-job`
services:
# Label used to access the service container
neo4j:
# Docker Hub image
image: neo4j:4.0.1
ports:
- 7474:7474 # used for http
- 7687:7687 # used for bolt
env:
NEO4J_AUTH: neo4j/password
NEO4J_dbms_connector_http_advertised__address: "NEO4J_HOST:7687"
NEO4J_dbms_connector_bolt_advertised__address: "NEO4J_HOST:7687"
steps:
- uses: actions/checkout#v2
- name: Setup .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: 3.1.101
- name: Install dependencies
run: dotnet restore ./src/BbcCorp.Neo4j.NeoGraphManager.sln
- name: Build
run: dotnet build --configuration Release --no-restore ./src/BbcCorp.Neo4j.NeoGraphManager.sln
- name: Run Integration Tests
env:
NEO4J_SERVER: NEO4J_HOST
run: |
cd ./src/BbcCorp.Neo4j.Tests
docker ps
dotnet test --no-restore --verbosity normal BbcCorp.Neo4j.Tests.csproj
I am trying to connect to: bolt://NEO4J_HOST:7687 but it just doesn't connect.
Error Details:
Using Neo4j Server NEO4J_HOST:7687 as neo4j/password
Error executing query. Connection with the server breaks due to ExtendedSocketException: Resource temporarily unavailable Please ensure that your database is listening on the correct host and port and that you have compatible encryption settings both on Neo4j server and driver. Note that the default encryption setting has changed in Neo4j 4.0.
[xUnit.net 00:00:00.83] NeoGraphManagerIntegrationTests.SimpleNodeTests [FAIL]
[xUnit.net 00:00:00.83] Neo4j.Driver.ServiceUnavailableException : Connection with the server breaks due to ExtendedSocketException: Resource temporarily unavailable Please ensure that your database is listening on the correct host and port and that you have compatible encryption settings both on Neo4j server and driver. Note that the default encryption setting has changed in Neo4j 4.0.
[xUnit.net 00:00:00.83] ---- System.Net.Internals.SocketExceptionFactory+ExtendedSocketException : Resource temporarily unavailable
The settings file for my C# test project looks like this
{
"NEO4J_SERVER": "localhost",
"NEO4J_PORT": 7687,
"NEO4J_DB_USER": "neo4j",
"NEO4J_DB_PWD": "password"
}
The tests runs fine from my local machine but fails when we execute on GitHub.

Related

Provide port to elasticsearch in gitlab ci services for integration tests

I have a code repo in which I want to support both ES 5 and ES 7 versions. So the code is written in a way to detect which ES version is being run via env variables and use the connectors accordingly. While writing integration tests for this that run in the gitlab ci, I want to run both ES5 and ES7 to test both pieces of connectors. I have defined my ES images like so
- name: docker.elastic.co/elasticsearch/elasticsearch:5.6.0
alias: elastic
command:
- /bin/env
- 'discovery.type=single-node'
- 'xpack.security.enabled=false'
- /bin/bash
- bin/es-docker
- name: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
alias: elastic7
command:
- port 9202
- /bin/env
- 'discovery.type=single-node'
- 'xpack.security.enabled=false'
- 'http.port=9202'
- /bin/bash
- bin/es-docker
But my ES7 is not able to run on the given port and i get errors like so in my coverage stage :
*** WARNING: Service runner-ssasmqme-project-5918-concurrent-0-934a00b2fdd833db-docker.elastic.co__elasticsearch__elasticsearch-4 probably didn't start properly.
Health check error:
service "runner-ssasmqme-project-5918-concurrent-0-934a00b2fdd833db-docker.elastic.co__elasticsearch__elasticsearch-4-wait-for-service" timeout
Health check container logs:
Service container logs:
2023-02-07T07:59:39.562478117Z /usr/local/bin/docker-entrypoint.sh: line 37: exec: port 9202: not found
How do I run multiple ES versions on different ports in my gitlab ci ?

Karate-Chrome docker container in azure dev ops failing to connect

I have seen many similar issues to this but none seem to resolve or describe my exact issue.
I have configured an azure devops pipeline to use a container like below:
container:
image: ptrthomas/karate-chrome
options: --cap-add=SYS_ADMIN
I have uploaded the contents of the example from the jobserver demo to a repository and then run the following:
steps:
- script: mvn clean test -DargLine='-Dkarate.env=docker' -Dtest=WebRunner
It is my understanding (and I can see from the logs) that the files are loaded into the container and the script command is being executed inside the container. So that script command is the equivalent of docker exec -it -w /src karate mvn clean test -DargLine='-Dkarate.env=docker' -Dtest=WebRunner just without having to exec into the container.
When I run the example locally it executes the tests with no issues but in azure dev ops it fails at the point the tests actually start running, throwing this error:
14:16:37.388 [main] ERROR com.intuit.karate - karate.org.apache.http.conn.HttpHostConnectException: Connect to
localhost:9222 [localhost/127.0.0.1] failed: Connection refused
(Connection refused), http call failed after 2 milliseconds for url:
http://localhost:9222/json 14:16:39.388 [main] DEBUG
com.intuit.karate.shell.Command - attempt #4 waiting for http to be
ready at: http://localhost:9222/json 14:16:39.391 [main] DEBUG
com.intuit.karate - request: 5 > GET http://localhost:9222/json 5 >
Host: localhost:9222 5 > Connection: Keep-Alive 5 > User-Agent:
Apache-HttpClient/4.5.13 (Java/1.8.0_275) 5 > Accept-Encoding:
gzip,deflate
Looking at other issues there have been suggestions to specify the driver in the feature files with this line:
* configure driver = { type: 'chrome', executable: 'chrome' }
but a) that hasn't worked for me and b) shouldn't the karate-chrome docker image render this configuration unnecessary as it should be no different than the container I run locally?
Any help appreciated!
Thanks
Only thing I can think of is that the Azure config does not call the ENTRYPOINT of the image.
Maybe you should try to create a container from scratch (that does extensive logging) and see what happens. Use the Karate one as a reference.

Openshift: any deployment resulted in Application is not available

Fist time deploying to OpenShift (actually minishift in my Windows 10 Pro). Any sample application I deploied successfully resulted in:
From Web Console I see a weird message "Build #1 is pending" although I saw it was successfully from PowerShell
I found someone fixing similiar issue changing to 0.0.0.0 (enter link description here) but I give a try and it isn't the solution in my case.
Here are the full logs and how I am deploying
PS C:\to_learn\docker-compose-to-minishift\first-try> oc new-app https://github.com/openshift/nodejs-ex warning: Cannot check if git requires authentication.
--> Found image 93de123 (16 months old) in image stream "openshift/nodejs" under tag "10" for "nodejs"
Node.js 10.12.0
---------------
Node.js available as docker container is a base platform for building and running various Node.js applications and frameworks. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.
Tags: builder, nodejs, nodejs-10.12.0
* The source repository appears to match: nodejs
* A source build using source code from https://github.com/openshift/nodejs-ex will be created
* The resulting image will be pushed to image stream tag "nodejs-ex:latest"
* Use 'start-build' to trigger a new build
* WARNING: this source repository may require credentials.
Create a secret with your git credentials and use 'set build-secret' to assign it to the build config.
* This image will be deployed in deployment config "nodejs-ex"
* Port 8080/tcp will be load balanced by service "nodejs-ex"
* Other containers can access this service through the hostname "nodejs-ex"
--> Creating resources ...
imagestream.image.openshift.io "nodejs-ex" created
buildconfig.build.openshift.io "nodejs-ex" created
deploymentconfig.apps.openshift.io "nodejs-ex" created
service "nodejs-ex" created
--> Success
Build scheduled, use 'oc logs -f bc/nodejs-ex' to track its progress.
Application is not exposed. You can expose services to the outside world by executing one or more of the commands below:
'oc expose svc/nodejs-ex'
Run 'oc status' to view your app.
PS C:\to_learn\docker-compose-to-minishift\first-try> oc get bc/nodejs-ex -o yaml apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
annotations:
openshift.io/generated-by: OpenShiftNewApp
creationTimestamp: 2020-02-20T20:10:38Z
labels:
app: nodejs-ex
name: nodejs-ex
namespace: samplepipeline
resourceVersion: "1123211"
selfLink: /apis/build.openshift.io/v1/namespaces/samplepipeline/buildconfigs/nodejs-ex
uid: 1003675e-541d-11ea-9577-080027aefe4e
spec:
failedBuildsHistoryLimit: 5
nodeSelector: null
output:
to:
kind: ImageStreamTag
name: nodejs-ex:latest
postCommit: {}
resources: {}
runPolicy: Serial
source:
git:
uri: https://github.com/openshift/nodejs-ex
type: Git
strategy:
sourceStrategy:
from:
kind: ImageStreamTag
name: nodejs:10
namespace: openshift
type: Source
successfulBuildsHistoryLimit: 5
triggers:
- github:
secret: c3FoC0RRfTy_76WEOTNg
type: GitHub
- generic:
secret: vlKqJQ3ZBxfP4HWce_Oz
type: Generic
- type: ConfigChange
- imageChange:
lastTriggeredImageID: 172.30.1.1:5000/openshift/nodejs#sha256:3cc041334eef8d5853078a0190e46a2998a70ad98320db512968f1de0561705e
type: ImageChange
status:
lastVersion: 1

Concourse pending for long time before running task

I have a Concourse Pipeline with a Task using a Docker image that is stored in our local Artifactory server. Every time I start the Pipeline it takes about 5 mins until the tasks are finally run. The log looks like this:
I assume that Concourse somehow checks for newer versions of the Docker image. Unfortunately I have no chance to debug since all the logfiles on the Concourse worker VM offer no usable information.
My Questions:
How can I possibly debug what's going on, when Concourse says "preparing build" and the status is "pending".
Is there any chance to avoid Concourse from checking for a newer version of the Docker image? I tagged the Docker image with version latest - might this be an issue?
Any further ideas how I could speed things up?
Here is the detailed configuration of my pipeline and tasks:
pipeline.yml:
---
resources:
- name: concourse-image
type: docker-image
source:
repository: OUR_DOMAIN/subpath/concourse
username: ...
password: ...
insecure_registries:
- OUR_DOMAIN
# ...
jobs:
- name: deploy
public: true
plan:
- get: concourse-image
- task: create-manifest
image: concourse-image
file: concourse/tasks/create-manifest/task.yml
params:
# ...
task.yml:
---
platform: linux
inputs:
- name: git
- name: concourse
outputs:
- name: deployment-manifest
run:
path: concourse/tasks/create-and-upload-cloud-config/task.sh
The reason for this problem was that we pulled the Docker image from an internal Docker registry, which is running on HTTP only. Concourse tried to pull the image using HTTPS and it took around 5 mins until Concourse switched to HTTP (that's what a tcpdump on the worker showed us).
Changing the resource configuration to the following config solved the problems:
resources:
- name: concourse-image
type: docker-image
source:
repository: OUR_SERVER:80/subpath/concourse
username: docker-readonly
password: docker-readonly
insecure_registries:
- OUR_SERVER:80
So basically it was adding the port explicitly to the repository and the insecure_registries.

setup drone continuous integration with github

I'm trying to setup a CI server inside a corporate network with drone (open source edition). Its author describes drone as very simple solution even for programmer (as I am), though some moments are not clear for me (may be official documentation misses them).
First, I've made up an docker image for my rails application: rails-qna.
Next, composing drone images:
docker-compose.yml:
version: '2'
services:
drone-server:
image: drone/drone:0.5
ports:
- 80:8000
volumes:
- ./drone:/var/lib/drone/
restart: always
environment:
- DRONE_OPEN=true
- DRONE_ADMIN=khataev
- DRONE_GITHUB_CLIENT=github-client-string
- DRONE_GITHUB_SECRET=github-secret-string
- DRONE_SECRET=drone-secret-string
drone-agent:
image: drone/drone:0.5
command: agent
restart: always
depends_on: [ drone-server ]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- DRONE_SERVER=ws://drone-server:8000/ws/broker
- DRONE_SECRET=drone-secret-string
Application is registered on Github and secret/client strings are provided.
I placed .drone.yml file into my project repository:
pipeline:
build:
image: rails-qna
commands:
- bundle exec rake db:drop
- bundle exec rake db:create
- bundle exec rake db:migrate
- bundle exec rspec
Unclear moments:
1) While registering OAuth application on github, we should specify Homepage URL and authorization callback URL. Where should they point to? Drone server container? Guessing that so, I specified
mycorporatedomain.com:3005
and
mycorporatedomain.com:3005/authorize
and setup port forwarding from 3005 port to 80 port of host, where drone docker is running. May be I'm wrong?
2) What should I specify in key DRONE_GITHUB_URL?
https://github.com or full path to my project repository, i.e.
https://github.com/khataev/qna?
3) What if I want to build some branch other than master? Were should I specify it? For now drone ready branch (with .drone.yml) is not a master branch - would it work?
4) Why DRONE_GITHUB_GIT_USERNAME and DRONE_GITHUB_GIT_PASSWORD are optional? How it is supposed to work if, I don't specify username and password for my github account?
5) When I start drone images with docker up, I get this errors:
→ docker-compose up
Starting drone_drone-server_1
Starting drone_drone-agent_1
Attaching to drone_drone-server_1, drone_drone-agent_1
drone-server_1 | time="2017-03-04T17:00:33Z" level=fatal msg="version control system not configured"
drone-agent_1 | 1:M 04 Mar 17:00:35.208 * connecting to server ws://drone-server:8000/ws/broker
drone-agent_1 | 1:M 04 Mar 17:00:35.229 # connection failed, retry in 15s. websocket.Dial ws://drone-server:8000/ws/broker: dial tcp: lookup drone-server on 127.0.0.11:53: no such host
drone_drone-server_1 exited with code 1
drone-server_1 | time="2017-03-04T16:53:38Z" level=fatal msg="version control system not configured"
UPD
5) this was solved - forgot to specify
DRONE_GITHUB=true
Homepage URL is the address of the server where drone is running on.
E.g. http://155.200.100.0
Authorize URL is the same address appended by /authorize
Eg. http://155.200.100.0/authorize
You dont have to specify that. DRONE_GITHUB=true says drone to use github url.
You can limit a single section to a branch or the whole drone build.
Single Section:
pipeline:
build:
image: node:latest
commands:
- npm install
- npm test
when:
branch: master
Whole build process:
pipeline:
build:
image: node:latest
commands:
- npm install
- npm test
branches: master
You don't need username and password when using OAuth.
Source:
http://readme.drone.io/admin/setup-github/
http://readme.drone.io/usage/skipping-builds/
http://readme.drone.io/usage/skipping-build-steps/
UPDATE:
Documentation is shifted to http://docs.drone.io/ due to version 0.6 of Drone

Resources