Hyperledger Composer - Docker Swarm - docker

Question

I've been experimenting with Hyperledger Composer and with the official multi org tutorial. I was successful in modifying the given demo, adding a third organisation and finally installing my own bna.
The next step was to fully understand how to deploy the Fabric network and Composer on multiple physical machines. And I went through all the available info about deploying such a process but without much luck.
Let suppose:
PC1: 1 Orderer, 1 Organisation, One cli container;
PC2: 1 Organisation;
PC3: 1 Organization;
I'm able to put the 3 machines in a swarm.
I know that I need to generate the certificates for all the machines and that they should be identical.
But from there I don't fully understand how to continue, or how to add references to the swarm network inside the compose files...
#docker-compose-cas-template-0 - This is for PC 1
version: '2'
networks:
example:
services:
ca0:
image: hyperledger/fabric-ca
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca-manager
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem
- FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY
ports:
- "7054:7054"
command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY -b admin:adminpw -d'
volumes:
- ./crypto-config/peerOrganizations/manager.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca_peerManager
networks:
- example
# docker-compose-base-1.yaml - This is for PC1
version: '2'
services:
orderer.proa.com:
container_name: orderer.proa.com
image: hyperledger/fabric-orderer:$IMAGE_TAG
environment:
- ORDERER_GENERAL_LOGLEVEL=INFO
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
# enabled TLS
- ORDERER_GENERAL_TLS_ENABLED=true
- ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
- ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
- ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: orderer
volumes:
- ../channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
- ../crypto-config/ordererOrganizations/proa.com/orderers/orderer.proa.com/msp:/var/hyperledger/orderer/msp
- ../crypto-config/ordererOrganizations/proa.com/orderers/orderer.proa.com/tls/:/var/hyperledger/orderer/tls
- orderer.proa.com:/var/hyperledger/production/orderer
ports:
- 7050:7050
peer0.manager.proa.com:
container_name: peer0.manager.proa.com
extends:
file: peer-base.yaml
service: peer-base
environment:
- CORE_PEER_ID=peer0.manager.proa.com
- CORE_PEER_ADDRESS=peer0.manager.proa.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer1.manager.proa.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.manager.proa.com:7051
- CORE_PEER_LOCALMSPID=ManagerMSP
volumes:
- /var/run/:/host/var/run/
- ../crypto-config/peerOrganizations/manager.proa.com/peers/peer0.manager.proa.com/msp:/etc/hyperledger/fabric/msp
- ../crypto-config/peerOrganizations/manager.proa.com/peers/peer0.manager.proa.com/tls:/etc/hyperledger/fabric/tls
- peer0.manager.proa.com:/var/hyperledger/production
ports:
- 7051:7051
- 7053:7053
peer1.manager.proa.com:
container_name: peer1.manager.proa.com
extends:
file: peer-base.yaml
service: peer-base
environment:
- CORE_PEER_ID=peer1.manager.proa.com
- CORE_PEER_ADDRESS=peer1.manager.proa.com:7051
- CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.manager.proa.com:7051
- CORE_PEER_GOSSIP_BOOTSTRAP=peer0.manager.proa.com:7051
- CORE_PEER_LOCALMSPID=ManagerMSP
volumes:
- /var/run/:/host/var/run/
- ../crypto-config/peerOrganizations/manager.proa.com/peers/peer1.manager.proa.com/msp:/etc/hyperledger/fabric/msp
- ../crypto-config/peerOrganizations/manager.proa.com/peers/peer1.manager.proa.com/tls:/etc/hyperledger/fabric/tls
- peer1.manager.proa.com:/var/hyperledger/production
ports:
- 8051:7051
- 8053:7053
UPDATED with docker-compose-cli.yaml - for ORG2 and PC2
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.sponsor.example.com:
peer1.sponsor.example.com:
networks:
example:
external:
name: example
services:
peer0.sponsor.example.com:
container_name: peer0.sponsor.example.com
extends:
file: base/docker-compose-base-2.yaml
service: peer0.sponsor.example.com
networks:
- example
peer1.sponsor.example.com:
container_name: peer1.sponsor.example.com
extends:
file: base/docker-compose-base-2.yaml
service: peer1.sponsor.example.com
networks:
- example
cli2:
container_name: cli2
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- CORE_LOGGING_LEVEL=DEBUG
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.sponsor.example.com:7051
- CORE_PEER_LOCALMSPID=SponsorMSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/users/Admin#sponsor.example.com/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
depends_on:
- peer0.sponsor.example.com
- peer1.sponsor.example.com
networks:
- example

Answer 1

You're on right track. I'll list down the steps below:
We create a docker swarm and connect these hosts. Since you've already created a swarm, I assume your hosts are connected.
We create an overlay network from one of the hosts. In our case, it is "example" network.
docker network create --attachable --driver overlay example
Now, this overlay network will be available in all the hosts. You can run the following command in each of the hosts:
docker network ls
Here, you'll be able to see the network with the name "example" that is an overlay network.
Also, you can inspect the network to see which all hosts (peers) are connected to this network using:
docker network inspect example
Spin up the containers. However, in this step, we need to make these containers join the existing overlay network i.e "example". So your compose files will be like:
version: '2'
networks:
example:
external:
name: example
services:
ca0:
image: hyperledger/fabric-ca
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca-manager
- FABRIC_CA_SERVER_TLS_ENABLED=true
- FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem
- FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY
ports:
- "7054:7054"
command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY -b admin:adminpw -d'
volumes:
- ./crypto-config/peerOrganizations/manager.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca_peerManager
networks:
- example
This configuration will remain similar for all your docker containers, be it, peer, orderer, ca or cli. Also, this configuration will make sure that your container will join the existing network instead of creating a new one.
Note: Running the docker in swarm mode requires few ports to be opened up. You can find those references in this article.