I'm running Fabric v2.3 test network, and I'm trying to take a snapshot of the ledger.
When I run:
# peer snapshot submitrequest -c mychannel -b 0 --peerAddress peer0.org1.example.com:7051 --tlsRootCertFile /opt/gopath/src/github.com/hyperledger/fabric/peer/o
rganizations/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
I obtain:
Error: failed to retrieve snapshot client: snapshot client failed to connect to peer0.org1.example.com:7051: failed to create new connection: context deadline exceeded
The cli container can connect to peer1. Inspecting the orderer logs, it seems that a connection to 172.19.0.1 (on variable ports) is not successful: https://gist.github.com/RafaelAPB/962477f9bab5f34076add1a3b9f63588
Part of it:
2021-10-04 00:31:54.519 UTC [common.deliver] Handle -> WARN 066 Error reading from 172.19.0.8:53714: rpc error: code = Canceled desc = context canceled
By inspecting the logs of peer0.org1, it really seems that the problem is the connection with the orderer (in particular the TLS handshake): https://gist.github.com/RafaelAPB/6adf0c6e702387e4e0edd8de0dcf0a76
Part of it:
2021-10-04 00:52:03.595 UTC [comm.grpc.server] 1 -> INFO e6d unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=172.19.0.1:52374 grpc.code=OK grpc.call_duration=4.028085ms
2021-10-04 00:52:04.293 UTC [core.comm] ServerHandshake -> ERRO e6e Server TLS handshake failed in 33.508µs with error tls: first record does not look like a TLS handshake server=PeerServer remoteaddress=172.19.0.8:58532
2021-10-04 00:52:04.419 UTC [gossip.privdata] StoreBlock -> INFO e6f Received block [505] from buffer channel=mychannel
No configurations were added to the configtx.yaml regarding the snapshotting feature.
Any ideas on how to run the snapshot command successfully?
Related
We are trying to establish a grpcs (TLS) connection between a docker container running API server (based on Node.js) and another docker container running peer0 from Fabric network.
All containers are orchestated by docker swarm, and both containers happen to be running on the same Linux host.
The error log thrown by API container is the following:
2021-01-07T18:27:38.110Z - error: [Remote.js]: Error: Failed to
connect before the deadline URL:grpcs://10.0.1.2:9051 Query has
completed, checking results error from query = { Error: Failed to
connect before the deadline URL:grpcs://10.0.1.2:9051
at checkState (/usr/src/app/node_modules/grpc/src/client.js:833:16) connectFailed:
true } sampleEvent ERROR : Error: 14 UNAVAILABLE: Connect Failed E0107
18:27:53.602719124 16 ssl_transport_security.cc:1229] Handshake
failed with fatal error SSL_ERROR_SSL: error:14090086:SSL
routines:ssl3_get_server_certificate:certificate verify failed.
And the error log thrown from peer0 is:
2021-01-07 18:50:22.224 UTC [core.comm] ServerHandshake -> ERRO 043 TLS handshake failed with error EOF server=PeerServer remoteaddress=10.0.1.4:46212
IP addresses layout
IP address for API container is 10.0.1.94
IP address for peer0 container is 10.0.1.3
virtual IP address for docker service peer0 is 10.0.1.2
IP address for docker swarm load balancer endpoint is 10.0.1.4
Any suggestion of where to further troubleshoot? At this point is not clear if the problem is with the docker swarm internal networking, or an issue with ssl certificates in either side of the network.
UPDATE Feb 2 2021
The original TLS handshake error was fixed by upgrading the javascript used in NodeSDK. Among other things we started using the addToWallet.js script contained in the commercial-paper example
After being able to stablish TLS succesfully between Node.js API and peer0, we get a new access denied error when making a simple query to chaincode_example02
Facts:
We are running the query with 2 Admin users
One Admin is first-network original Admin#org1.example.com, with credentials generated by cryptogen tool
The other Admin is Admin#buyer.dlt.com whose credentials were created with openssl and a self signed in-company CA
From CLI, both Admin are good and are allowed to run peer commands interchangeably
From Node.js app, only Admin#org1.example.com is allowed to run queries. The message printed to console.log is:
Transaction has been evaluated, result is: 100
When running queries with Admin#buyer.dlt.com we get the following error logs:
Error logs from peer0#buyer.dlt.com
2021-02-02T04:08:45.291086617Z ^[[36m2021-02-02 04:08:45.290 UTC [protoutils] checkSignatureFromCreator -> DEBU 6e637^[[0m creator is &{BuyerMSP 8b7cc2ee996be4f7e5dbb1a4f64db67afd2ff8a2f41276c9bd7f33a2447dd9df}
2021-02-02T04:08:45.291094817Z ^[[36m2021-02-02 04:08:45.290 UTC [protoutils] checkSignatureFromCreator -> DEBU 6e638^[[0m creator is valid
2021-02-02T04:08:45.291100418Z ^[[36m2021-02-02 04:08:45.290 UTC [msp.identity] 2021-02-02T04:08:45.303821799Z ^[[33m2021-02-02 04:08:45.303 UTC [protoutils] ValidateProposalMessage -> WARN 6e63b^[[0m channel [mychannel]: creator's signature over the proposal is not valid: The signature is invalid
2021-02-02T04:08:45.303891604Z ^[[36m2021-02-02 04:08:45.303 UTC [endorser] func1 -> DEBU 6e63c^[[0m Exit: request from 10.0.1.84:52696
2021-02-02T04:08:45.303902005Z ^[[34m2021-02-02 04:08:45.303 UTC [comm.grpc.server] 1 -> INFO 6e63d^[[0m unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=10.0.1.84:52696 error="access denied: channel [mychannel] creator org [BuyerMSP]" grpc.code=Unknown grpc.call_duration=13.783655ms
Error log on console.log from script query.js:
2021-02-02T04:08:45.305Z - error: [Channel.js]: Error: 2 UNKNOWN: access denied: channel [mychannel] creator org [BuyerMSP]
2021-02-02T04:08:45.307Z - error: [Network]: _initializeInternalChannel: Unable to initialize channel. Attempted to contact 1 Peers. Last error was Error: 2 UNKNOWN: access denied: channel [mychannel] creator org [BuyerMSP]
Failed to evaluate transaction: Error: Unable to initialize channel. Attempted to contact 1 Peers. Last error was Error: 2 UNKNOWN: access denied: channel [mychannel] creator org [BuyerMSP]
In the end, this issue turned out to be two issues, in a 'russian doll like' style.
1. First issue: TLS Handshake error
This was fixed by upgrading the SDK library to the latest release
2. Second issue: Node SDK query triggers error "The signature is invalid".
The reason turned out to be that the CLI (written on Go) is using the Go crypto support which allows it to generate a signature from a hash without any knowledge of the curve used for the key. Instead, the SDK libraries used by the Node implementation require a specific curve to be specified by the code generating the signature, separately from the private key itself.
Bottom line, private keys used within Node SDK should be P-256.
As an alternative, as suggested by hyperledger dev team:
If you really must use a curve other than P-256 then you might be able
to use one of the following approaches:
-Use the off-line signing approach included in the documentation but specify an alternative curve instead of 'p256'. The supported curves
for the elliptic package documented here:
https://github.com/indutny/elliptic
-Set your own CryptoSuite implementation on the Client that underpins the Gateway object, with your own CryptoSuite.sign() implementation:
https://hyperledger.github.io/fabric-sdk-node/release-2.2/CryptoSuite.html#sign
I bring up 4 peers using byfn.sh.
Everything comes up fine and I can see the orderer, peers, CA nodes, etc. running in docker containers
I then use a custom service layer application that uses the Fabric Node SDK to connect with this network.
I have no issues making the connection using byfn.sh generated TLS CA certificate (/usr/local/src/test_env/hlf_scripts/first-network/crypto-config/peerOrganizations/org1.example.com/tlsca/tlsca.org1.example.com-cert.pem)
Here are the docker log messages for the call Gateway.getNetwork(“mychannel”):
2020-01-14 22:12:20.981 UTC [comm.grpc.server] 1 -> INFO 083 unary call completed grpc.service=discovery.Discovery grpc.method=Discover grpc.peer_address=192.168.80.1:38858 grpc.code=OK grpc.call_duration=515.21µs
2020-01-14 22:12:21.012 UTC [comm.grpc.server] 1 -> INFO 084 unary call completed grpc.service=discovery.Discovery grpc.method=Discover grpc.peer_address=192.168.80.1:38858 grpc.code=OK grpc.call_duration=552.801µs
I then bring up another client application, that uses pretty much the same code, for manipulating the network. The first step here is to ensure that the user’s (admin in this case) identity exists and can be imported into a wallet. The same piece of code is used to generate this identity, and using the same connection profile (connection_org1.json). The identity can be created and imported into the filesystem wallet.
I can connect to the network using the Gateway.connect(…) method and using the connection profile connection_org1.json. However, the moment I invoke Gateway.getNetwork(“mychannel”), my issues start. Here is the error returned by the Fabric SDK:
[nodemon] starting `node ./dist/server.js`
Express server listening on port 3050
Wallet path: /usr/local/src/fabric-network-code/wallet
E0114 17:16:47.872273165 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0114 17:16:48.873788447 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0114 17:16:50.368082218 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
2020-01-14T22:16:50.869Z - error: [Remote.js]: Error: Failed to connect before the deadline URL:grpcs://localhost:7051
2020-01-14T22:16:50.870Z - error: [Channel.js]: Error: Failed to connect before the deadline URL:grpcs://localhost:7051
E0114 17:16:50.877630619 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0114 17:16:51.876732084 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0114 17:16:53.263732590 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
E0114 17:16:53.303070959 1998 ssl_transport_security.cc:1238] Handshake failed with fatal error SSL_ERROR_SSL: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed.
2020-01-14T22:16:53.875Z - error: [Remote.js]: Error: Failed to connect before the deadline URL:grpcs://localhost:8051
2020-01-14T22:16:53.875Z - error: [Channel.js]: Error: Failed to connect before the deadline URL:grpcs://localhost:8051
2020-01-14T22:16:53.875Z - error: [Network]: _initializeInternalChannel: Unable to initialize channel. Attempted to contact 2 Peers. Last error was Error: Failed to connect before the deadline URL:grpcs://localhost:8051
Unable to initialize channel. Attempted to contact 2 Peers. Last error was Error: Failed to connect before the deadline URL:grpcs://localhost:8051
[nodemon] app crashed - waiting for file changes before starting...
And here is the log from the docker container log for peer0.org1.example.com
2020-01-14 22:16:47.872 UTC [core.comm] ServerHandshake -> ERRO 085 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.80.1:38886
2020-01-14 22:16:48.874 UTC [core.comm] ServerHandshake -> ERRO 086 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.80.1:38890
2020-01-14 22:16:50.368 UTC [core.comm] ServerHandshake -> ERRO 087 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.80.1:38894
2020-01-14 22:16:53.303 UTC [core.comm] ServerHandshake -> ERRO 088 TLS handshake failed with error EOF server=PeerServer remoteaddress=192.168.80.1:38910
I have tried various permutations and combinations of using different certs but not getting anywhere. Online forums indicate this is an issue of communication between orderer and the peer. However, there are no error messages in the orderer.
I have asked this question in chat.hyperledger.org but no response yet.
So extremely foolish of me. I had a crypto-config directory sitting in my node project folder (I don't know when I may have copied it over....) and that was throwing off the tlscacert variable.
I have removed the offending directory and now it is reading the correct one under .../first-network/crypto-config/.../<tlsca file>.pem
Sometimes (and quite often :-) ) the answer is right there but one is too stupid to see it!!!
I am trying create a channel according to documentation
Hyperledger Fabric v1.0 docs
Have an issue with certificate. On the docker "hyperledger/fabric-tools" node I can find certificate with current name - tlsca.example.com. But the channel cannot be created. I have certificate hand shake issue. Should I check/mount certificate to the peer node ?
root#4b6423da537b:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com# peer channel create -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/channel.tx --tls $CORE_PEER_TLS_ENABLED --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem
2017-07-27 16:49:58.949 UTC [msp] GetLocalMSP -> DEBU 001 Returning existing local MSP
2017-07-27 16:49:58.949 UTC [msp] GetDefaultSigningIdentity -> DEBU 002 Obtaining default signing identity
2017-07-27 16:49:58.954 UTC [grpc] Printf -> DEBU 003 Failed to dial orderer.example.com:7050: connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"x509: ECDSA verification failure\" while trying to verify candidate authority certificate \"tlsca.example.com\")"; please retry.
Error: Error connecting due to rpc error: code = Internal desc = connection error: desc = "transport: authentication handshake failed: x509: certificate signed by unknown authority (possibly because of \"x509: ECDSA verification failure\" while trying to verify candidate authority certificate \"tlsca.example.com\")"
Usage:
Thanks.
i meet the same problem. And run this command to close the network.
./network_setup.sh down mychannel
The reason that cause my problem is that the source code exists a error. So i modify this code error and reopen the network. This problem work out.
It would seem that you are in the incorrect working directory. When running the sample manually, you start the cli container and it places you in the /opt/gopath/src/github.com/hyperledger/fabric/peer directory. That is where you should be running the peer command. It would seem from your post that you were running the peer command in the /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com directory in the container, and it is not finding the configuration files that were mounted for the example.
I tried to build the development environment of fabric using the guide at https://github.com/hyperledger/fabric/blob/master/docs/dev-setup/devenv.md
After vagrant ssh into the VM, I wanted to start a peer node to test the example chaincode to verify if I have set the environment correctly, so with the guide at https://github.com/hyperledger/fabric/blob/master/docs/Setup/Chaincode-setup.md
I make peer, and started the peer node by peer node start --peer-chaincodedev, but error appeared:
[crypto] Errorf -> ERRO 008 [validator.vp] Failed requesting read certificate [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure].
[crypto] Errorf -> ERRO 009 [validator.vp] Failed requesting ECA certificate [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure].
[crypto] Errorf -> ERRO 00a [validator.vp] Failed getting ECA certificate [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure].
[crypto] Errorf -> ERRO 00b [validator.vp] Failed retrieving ECA certs chain [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure].
[crypto] Errorf -> ERRO 00c [validator.vp] Failed registering node crypto engine [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure].
[crypto] Errorf -> ERRO 00d [validator.vp] Failed registering peer [vp]: [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure]
[crypto] Errorf -> ERRO 00e [validator.vp] Failed registering [vp]: [rpc error: code = 14 desc = grpc: RPC failed fast due to transport failure]
grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp [::1]:7054: getsockopt: connection refused"; Reconnecting to {"localhost:7054" <nil>}
Failed to dial localhost:7054: grpc: the connection is closing; please retry.
I've also tried the optional step of security setup, make membersrvc and run it, but when running membersrvc, there is no output and no stop but just staying at there for several hours.
I have no idea why this appears and there seems no one else meeting the same problem with me after two-day searching. Please help me, thanks!
I was trying the simple chain code, with membersvc and validation peer running in Docker for Mac. I got this error when running the chain code example 2:
CORE_CHAINCODE_ID_NAME=mycc CORE_PEER_ADDRESS=localhost:30303 ./chaincode_example02
16:58:41.242 [shim] DEBU : Peer address: localhost:30303
16:58:41.244 [shim] DEBU : os.Args returns: [./chaincode_example02]
16:58:41.244 [shim] DEBU : Registering.. sending REGISTER
2016/08/22 16:58:41 transport: http2Client.notifyError got notified that the client transport was broken EOF.
16:58:41.245 [shim] ERRO : Received error from server: rpc error: code = 13 desc = "transport is closing", ending chaincode stream
Error starting Simple chaincode: rpc error: code = 13 desc = "transport is closing"
This issue is resolved with the latest docker build with the latest docker-compose file.