Port conflicts in starting test-network of Hyperledger Fabric using fabric-samples folder - docker

I'm a new starter and have been struggling for a while about these port errors
When I run ./network.sh in the directory: fabric-samples/test-network
the following port errors occurred:
yujindeMBP:test-network yujin$ ./network.sh up
Starting nodes with CLI timeout of '5' tries and CLI delay of '3' seconds and using database 'leveldb' with crypto from 'cryptogen'
LOCAL_VERSION=2.3.0
DOCKER_IMAGE_VERSION=2.3.0
/Users/yujin/fabric-samples-with-bis/test-network/../bin/cryptogen
Generating certificates using cryptogen tool
Creating Org1 Identities
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-org1.yaml --output=organizations
org1.example.com
+ res=0
Creating Org2 Identities
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-org2.yaml --output=organizations
org2.example.com
+ res=0
Creating Orderer Org Identities
+ cryptogen generate --config=./organizations/cryptogen/crypto-config-orderer.yaml --output=organizations
+ res=0
Generating CCP files for Org1 and Org2
Creating network "net_test" with the default driver
Creating volume "net_orderer.example.com" with default driver
Creating volume "net_peer0.org1.example.com" with default driver
Creating volume "net_peer0.org2.example.com" with default driver
Creating orderer.example.com ... error
Creating peer0.org2.example.com ...
Creating peer0.org1.example.com ...
Creating peer0.org1.example.com ... error
Creating peer0.org2.example.com ... done
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: Ports are not available: listen tcp 0.0.0.0:7051: bind: address already in use
ERROR: for orderer.example.com Cannot start service orderer.example.com: Ports are not available: listen tcp 0.0.0.0:7050: bind: address already in use
ERROR: for peer0.org1.example.com Cannot start service peer0.org1.example.com: Ports are not available: listen tcp 0.0.0.0:7051: bind: address already in use
ERROR: Encountered errors while bringing up the project.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6feb86580f43 hyperledger/fabric-orderer:latest "orderer" 1 second ago Created orderer.example.com
dbfae1aa4c11 hyperledger/fabric-peer:latest "peer node start" 1 second ago Created peer0.org1.example.com
d0367a0d6089 hyperledger/fabric-peer:latest "peer node start" 1 second ago Up Less than a second 7051/tcp, 0.0.0.0:9051->9051/tcp peer0.org2.example.com
It seems the orderer, the org1 and the org2 are using the same ports 7050 and 7051, they got conflicted with each other. I have thought I can avoid these port errors by running docker. However, it seems that I'm wrong. I checked the docker environment before I run ./network.sh and I'm sure no other processes are running in the same time.
yujindeMBP:test-network yujin$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
The docker's processes table is clear before I tried to start the test-network.
I'm really confused and need your help. Thanks a lot!

Try docker ps -a This list all the active container.
Remove containers using the command docker rm -f [container_id/container_name]
once all containers removed then bring back your network.
If you're still facing the issue then go to your crypto-config-org1.yaml, crypto-config-org2.yaml, crypto-config-orderer.yaml edit and change the ports. Also you can ask question on https://chat.hyperledger.org/

Related

SSL(curl) connection error in ElasticSearch setup

Have setup a 3-node Elasticsearch cluster using docker-compose. Followed below steps:
On one of the master nodes, es11, gets below error, however same curl command works fine on other 2 nodes i.e. es12, es13:
Error:
curl -X GET 'https://localhost:9316'
curl: (35) Encountered end of file
Below error in logs:
"stacktrace": ["org.elasticsearch.transport.RemoteTransportException: [es13][SOMEIP:9316][internal:cluster/coordination/join]",
"Caused by: org.elasticsearch.transport.ConnectTransportException: [es11][SOMEIP:9316] handshake failed. unexpected remote node {es13}{SOMEVALUE}{SOMEVALUE
"at org.elasticsearch.transport.TransportService.lambda$connectionValidator$6(TransportService.java:468) ~[elasticsearch-7.17.6.jar:7.17.6]",
"at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:95) ~[elasticsearch-7.17.6.jar:7.17.6]",
"at org.elasticsearch.transport.TransportService.lambda$handshake$9(TransportService.java:577) ~[elasticsearch-7.17.6.jar:7.17.6]",
https://localhost:9316 on browser gives site can't be reached error as well.It seems SSL certificate as created in step 4 below is having some issues in es11.
Any leads please? OR If I repeat step 4, do i need to copy the certs again to es12 & es13?
Below elasticsearch.yml
cluster.name: "docker-cluster"
network.host: 0.0.0.0
Ports as defined in all 3 nodes docker-compose.yml
environment:
- node.name=es11
- transport.port=9316
ports:
- 9216:9200
- 9316:9316
Initialize a docker swarm. On ES11 run docker swarm init. Follow the instructions to join 12 and 13 to the swarm.
Create an overlay network docker network create -d overlay --attachable elastic
If necessary, bring down the current cluster and remove all the associated volumes by running docker-compose down -v
Create SSL certificates for ES with docker-compose -f create-certs.yml run --rm create_certs
Copy the certs for es12 and 13 to the respective servers
Use this busybox to create the overlay network on 12 and 13 sudo docker run -itd --name containerX --net [network name] busybox
Configure certs on 12 and 13 with docker-compose -f config-certs.yml run --rm config_certs
Start the cluster with docker-compose up -d on each server
Set the passwords for the built-in ES accounts by logging into the cluster docker exec -it es11 sh then running bin/elasticsearch-setup-passwords interactive --url localhost:9316
(as per your https://discuss.elastic.co thread)
you cannot talk HTTP to the transport protocol port, which you have defined in transport.port. you need to talk to port 9200 in the container, which you have mapped to 9216 outside the container
the transport port runs a binary protocol that is not HTTP accessible

Deploy GitLab docker image on mac: Cannot see gitab.example.com

I am trying to run Gitlab Docker image on mac Big Sur locally following the steps from documentation: https://docs.gitlab.com/ee/install/docker.html . And always cannot see locally https://gitlab.example.com. I tried both Gitlab EE and CE, and different versions of images, including latest. I also tried to use usual Docker basic run and docker-compose. I also updated Docker Desktop to latest version 4.10.1. I refined the logs and for all cases I have same error in Gitally log below:
~/gitlab/logs/gitaly/current:
{"level":"warning","msg":"[core] [Channel #2 SubChannel #4] grpc: addrConn.createTransport failed to connect to {\n \"Addr\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0\",\n \"ServerName\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0\",\n \"Attributes\": null,\n \"BalancerAttributes\": null,\n \"Type\": 0,\n \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing dial unix /var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0: connect: no such file or directory\"","pid":344,"system":"system","time":"2022-07-26T09:57:38.226Z"}
{"level":"warning","msg":"[core] [Channel #3 SubChannel #5] grpc: addrConn.createTransport failed to connect to {\n \"Addr\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1\",\n \"ServerName\": \"/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1\",\n \"Attributes\": null,\n \"BalancerAttributes\": null,\n \"Type\": 0,\n \"Metadata\": null\n}. Err: connection error: desc = \"transport: Error while dialing dial unix /var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1: connect: no such file or directory\"","pid":344,"system":"system","time":"2022-07-26T09:57:38.228Z"}
{"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","344","/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.0"],"supervisor.name":"gitaly-ruby.0","supervisor.pid":400,"time":"2022-07-26T09:57:38.228Z"}
{"address":"/var/opt/gitlab/gitaly/gitaly.socket","level":"info","msg":"listening at unix address","time":"2022-07-26T09:57:38.234Z"}
{"level":"warning","msg":"spawned","supervisor.args":["bundle","exec","bin/ruby-cd","/var/opt/gitlab/gitaly","/opt/gitlab/embedded/service/gitaly-ruby/bin/gitaly-ruby","344","/var/opt/gitlab/gitaly/run/gitaly-344/sock.d/ruby.1"],"supervisor.name":"gitaly-ruby.1","supervisor.pid":401,"time":"2022-07-26T09:57:38.234Z"}
But I am not sure that this is the case - also checked other logs and seems to me no errors in other logs - but of course I will be happy to provide more logs if you want.
This is my 'docker ps -a' status, docker version: 4.10.1, is run with docker-compose:
web:
image: 'gitlab/gitlab-ce:latest'
container_name: 'gitlab'
restart: unless-stopped
hostname: 'gitlab.example.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://gitlab.example.com'
ports:
- '80:80'
- '443:443'
- '22:22'
volumes:
- '$GITLAB_HOME/config:/etc/gitlab'
- '$GITLAB_HOME/logs:/var/log/gitlab'
- '$GITLAB_HOME/data:/var/opt/gitlab'
list of docker active containers says container is healthy - but actually I see nothing in browser via https://gitlab.example.com.
sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b0fb6252c7f3 gitlab/gitlab-ce:latest "/assets/wrapper" 3 days ago Up 3 minutes (healthy) 0.0.0.0:22->22/tcp, 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp gitlab
I also tried to add docker host IP to my hosts file - but no success:
192.168.31.182 host.docker.internal
192.168.31.182 gateway.docker.internal
192.168.31.182 gitlab.example.com
I still see nothing in browser by https://gitlab.example.com
Adding more info to #sytech "That's just a placeholder URL... Did you replace gitlab.example.com with your actual GitLab instance URL?"
You either need to add an /etc/hosts, setting to route gitlab.example.com to localhost or modify that to be localhost
hostname: 'localhost'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'localhost'
These settings are telling the nginx proxy what url should be forwarded to gitlab. That's useful when you're running multiple sites from one server. If you are just running it locally, you should just use localhost.
I also tried to add docker host IP to my hosts file - but no success:
You are forwarding the ports from the docker ip. Just use localhost. It probably didn't work because the docker IP you're looking at is for a different docker network.

Problem in ./startFabric.sh: Error response from daemon: Container is not running

I have been using hyperledger composer for a long time. I have installed hyperledger composer on a new machine but on the command ./startFabric.sh it is ending with the following error
Development only script for Hyperledger Fabric control
Running 'startFabric.sh'
FABRIC_VERSION is unset, assuming hlfv12
FABRIC_START_TIMEOUT is unset, assuming 15 (seconds)
Stopping orderer.example.com ... done
Stopping couchdb ... done
Stopping ca.org1.example.com ... done
Removing peer0.org1.example.com ... done
Removing orderer.example.com ... done
Removing couchdb ... done
Removing ca.org1.example.com ... done
Removing network composer_default
Creating network "composer_default" with the default driver
Creating ca.org1.example.com ...
Creating couchdb ...
Creating orderer.example.com ...
Creating orderer.example.com
Creating ca.org1.example.com
Creating orderer.example.com ... done
Creating peer0.org1.example.com ...
Creating peer0.org1.example.com ... done
sleeping for 15 seconds to wait for fabric to complete start up
Error response from daemon: Container abf272793595d47750a9b3ab8f381f183f64bcdf3b9c21fba00bef6de6517060 is not running
This command is running fine on older machine but on new machine it is giving this error I have installed it by following the link https://hyperledger.github.io/composer/latest/installing/installing-index
Please help... I am seriously tired of this. I have also tried solutions on web nothing helped...
I faced the same issue too. after searching all different answers in stackoverflow , i finally managed to find one that works for me .. all thanks to the contributor .
below is the step :
to solve this issue. Needed to add below line under environment: in docker-compose.yaml file for orderer and peer containers in your hlfv12 script subdir.
- GODEBUG=netdns=go
Hope this work for you too .

Error starting userland proxy: listen tcp 0.0.0.0:7050: bind: address already in use

I'm setting up a Hyperledger Fabric private network on Linux and got the message while running ./byfn.sh up
as I'm a newbie in Ubuntu and docker I think that the port needs to be changed in order to fix the problem, however, I have no clue in doing so. Any help would be appreciated.
alaa#ubuntu:~/fabric-samples/first-network$ sudo netstat -pna | grep 7050
tcp6 0 0 :::7050 :::* LISTEN 3682/docker-proxy
did a netstat to check the port and its docker-proxy
alaa#ubuntu:~/fabric-samples/first-network$ sudo ./byfn.sh up
Starting with channel 'mychannel' and CLI timeout of '10' seconds and CLI delay of '3' seconds
Continue? [Y/n] y
proceeding ...
2019-05-19 14:07:22.820 UTC [main] main -> INFO 001 Exiting.....
LOCAL_VERSION=1.1.0
DOCKER_IMAGE_VERSION=1.1.0
Creating network "net_byfn" with the default driver
Creating volume "net_orderer.example.com" with default driver
Creating volume "net_peer0.org1.example.com" with default driver
Creating volume "net_peer1.org1.example.com" with default driver
Creating volume "net_peer0.org2.example.com" with default driver
Creating volume "net_peer1.org2.example.com" with default driver
Creating orderer.example.com ... error
Creating peer1.org2.example.com ...
Creating peer1.org1.example.com ...
Creating peer0.org1.example.com ...
Creating peer1.org2.example.com ... done
Creating peer1.org1.example.com ... done
Creating peer0.org1.example.com ... done
Creating peer0.org2.example.com ... done
ERROR: for orderer.example.com Cannot start service orderer.example.com: b'driver failed programming external connectivity on endpoint orderer.example.com (60d170dbc933d3c2de9eacd1bb6c7842cf79a52b3a938c9e0e69d1bd55f5e1a9): Error starting userland proxy: listen tcp 0.0.0.0:7050: bind: address already in use'
ERROR: Encountered errors while bringing up the project.
ERROR !!!! Unable to start network
alaa#ubuntu:~/fabric-samples/first-network$ sudo netstat -pna | grep 7050
tcp6 0 0 :::7050 :::* LISTEN 3682/docker-proxy
Well, first of all for any kind of hyperledger tutorial, u better follow the official link, cos most of other sources were also taken from that one: https://hyperledger-fabric.readthedocs.io/en/release-1.4/
Secondly,bring down the network, stop&remove all running&previous containers, restart docker, re-run the network properly, should work fine:
$./byfn.sh down
$docker ps -qa|xargs docker rm
$sudo systemctl daemon-reload
$sudo systemctl restart docker
$cd....fabric-samples/first-network
$./byfn.sh -m generate
$./byfn.sh -m up

Connection to docker container failing because of postgis port issue

My docker container is able to successfully build but when I enter the command docker-compose build, the following error is returned:
Starting docker_etl_1 ...
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ...
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1
Starting 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 ... error
ERROR: for 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland prStarting docker_etl_1 ... done
ERROR: for postgis Cannot start service postgis: driver failed programming external connectivity on endpoint 1e5f56853e10_1e5f56853e10_1e5f56853e10_docker_postgis_1 (91464afbee8bf7212061797ec0f4c017a56cc3c30c9bdaf513127a6e6a4a5a52): Error starting userland proxy: Bind for 0.0.0.0:5432 failed: port is already allocated
Here is my docker-compose.yaml
version: '2'
services:
postgis:
build: ./postgis
volumes:
- ../src/main/sql:/sql
ports:
- "5432:5432"
etl:
build: ./etl
volumes:
- ..:/national-voter-file
entrypoint:
- python3
- /national-voter-file/load/loader.py
and here is the Dockerfile:
FROM mdillon/postgis:9.5
ENV POSTGRES_DB VOTER
RUN mkdir /sql
COPY ./dockerResources/z-init-db.sh /docker-entrypoint-initdb.d/
EXPOSE 5432
Docker ps -a returns:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
da74ad97b95c docker_postgis "docker-entrypoint..." About a minute ago Created docker_postgis_1
5872c6e55fe2 docker_etl "python3 /national..." About a minute ago Exited (2) About a minute ago docker_etl_1
However, when I try rm $(docker ps -qa) I get the following error:
rm: da74ad97b95c: No such file or directory
rm: 5872c6e55fe2: No such file or directory
I don't believe I have another container running so I'm confused by the message Bind for 0.0.0.0:5432 failed: port is already allocated
Is it possible that you ran the same docker-compose earlier, which probably failed or at least failed to clean up the services?
Try running docker ps -a to check if any stopped containers exist. It is possible that the stopped containers are hogging the port. If so, just clear them out using docker rm $(docker ps -qa)

Resources