Im trying to monitor my blockchain with hyperledger caliper. I created my own benchmark file by using one from the templates given in the repo. but I am getting this error
Error: Network configuration file "F:Blockchain\hyperledger-caliper\networks\fabric\docker-compose\3org1peercouchdb\docker-compose.yaml" is missing its "caliper.blockchain" string attribute
at Function.assertConfigurationFilePaths (F:\SE\3rd_Year\FYP\Blockchain\hyperledger-caliper\node_modules\#hyperledger\caliper-core\lib\common\utils\caliper-utils.js:75:19)
any ideas how to fix this?
Problem
The problem is with the network configuration file that you are giving to Caliper. According to the official documentation of Hyperledger Caliper, the networking configuration consists of an YAML network configuration file. This file needs to have an object caliper, since it's Required and an Non-empty object, with an attribute blockchain within, that is Required and Non-empty string. Your log says that this object, and specific the attribute, are missing in the file you are specifying.
SOLUTION
The file you specify is a docker-composer file. The network configuration file for your Fabric blockhain should be similar to, for example, the one at 'networks/fabric/v1/v1.4.1/2org1peercouchdb/fabric-go.yaml' that contains the network configuration parameters required by the Hyperledger Caliper and that manages the docker containers at the attribute commmand within the caliper object.
You can get those files at caliper github repository and also check the Hyperledger Caliper official documentation for a better understanding.
Example
You should have a network configuration file that specifies the blockhain that you are using, on the attribute caliper.blockchain, and how are the docker containers managed, on the attribute caliper.command (just like the remaining and required parameters). For example:
...
caliper:
blockchain: fabric
command:
start: export FABRIC_VERSION=1.4.0;export FABRIC_CA_VERSION=1.4.0;docker-compose -f networks/fabric/docker-compose/2org1peercouchdb/docker-compose.yaml up -d;sleep 3s
end: docker-compose -f networks/fabric/docker-compose/2org1peercouchdb/docker-compose.yaml down;(test -z \"$(docker ps -aq)\") || docker rm $(docker ps -aq);(test -z \"$(docker images dev* -q)\") || docker rmi $(docker images dev* -q);rm -rf /tmp/hfc-*
...
...
Notes
I'm using JSON files as network configuration files to Hyperledger Sawtooth and Ethereum, with Hyperledger Caliper. However, I'm not sure if a JSON file works on Fabric benchmarking.
Related
I just downloaded this docker image to set up a spark cluster with two worker nodes. Cluster is up and running however I want to submit my scala file to this cluster. I am not able to start spark-shell in this.
When I was using another docker image, I was able to start it using spark-shell.
Can someone please explain if I need to install scala separately in the image or there is a different way to start
UPDATE
Here is the error bash: spark-shell: command not found
bash: spark-shell: command not found
root#a7b0682ff17d:/opt/spark# ls /home/shangupta/Scripts/
ProfileData.json demo.scala queries.scala
TestDataGeneration.sql input.scala
root#a7b0682ff17d:/opt/spark# spark-shell /home/shangupta/Scripts/input.scala
bash: spark-shell: command not found
root#a7b0682ff17d:/opt/spark#
You're getting command not found because PATH isn't correctly established
Use the absolute path /opt/spark/bin/spark-shell
Also, I'd suggest packaging your Scala project as an uber jar to submit unless you have no external dependencies or like to add --packages/--jars manually
I'm trying to run docker in a partially locked-down environment, with /etc on a read-only mount point and a "/data" folder in a read/write mount point. I've added an /etc/docker/daemon.json file:
{
"data-root": "/data/docker"
}
but dockerd is failing on startup with this error:
failed to start daemon: Error saving key file: open /etc/docker/.tmp-key.json128868007: read-only file system
Can I stop dockerd from trying to write into /etc? Are there best practices for running docker on a host with read-only mounts?
EDIT: Turns out there was only one file being written: /etc/docker/key.json which is talked about in detail here. The .tmp-key.json bit is likely a part of some atomic file write code.
Looks like only the "key.json" file is written to /etc. After some digging, I found this PR which talks about making it configurable. As of docker 19.03.6, the option is still available for use in the daemon.json file as "deprecated-key-path": "/path/to/file".
While setting up and configure some docker containers I asked myself how I could automatically edit some config files inside the container after the containerized service finished installing (since the config files are created at the installation).
I have tried that using a shell file and adding it as the entrypoint in the Dockerfile. However, as I have said the config file does not exist right at the beginning and hence the sed commands in the script fail.
Linking an config files with - ./myConfig.conf:/xy/myConfig.conf is also not an option because the config contains some installation dependent options.
The most reasonable solution I have found was running a script, which edits the config, manually after the installation has finished with docker exec -i mycontainer sh < editconfig.sh
EDIT
My question is formulated in general terms. However, the question arose while working with Nextcloud in a docker-compose setup similar to the official example. That container contains a config.php file which is the general config file of Nextcloud and is generated during the installation. Certain properties of that files have to be changed (there are only a very limited number of environmental variables to specify). Since I am conducting some tests with this container I have to repeatedly reinstall it and thus reedit the config file.
Maybe you can try another approach and have your config file/application pick its settings from the environmental variables. That would be consistent with the 12factor app methodology see here
How I understand your case you need to start your container from creating config by some template.
I see a number of options to do it:
Use some script that generates a config from template and arguments from a command line or environment variables. (Jinja2 and python for example or Mustache and node.js ). In this case, your entrypoint generate the template and after this start application. For change config, you will be forced restart service (container).
Run some service can save the configuration and render you configuration in run time. Personally, I like consul template, we active use this engine in our environment, and have no problems for while. In this case, config is more dynamic and able to be changed "on the fly". In your container, you will have two processes, application, and consul-template daemon. Obviously, you will need to run and maintain consul. For reloading config restart of an application process is enough.
Run a custom script to create the config. :)
I am trying to customize tuna-app chaincode of the tuna-app example. I want to use cid package inside my chaincode to make ABAC decisions about who is allowed to run the chaincode. When I try to install chaincode, I get the following error:
Error: Error getting chaincode code chaincode:
Error getting chaincode package bytes: Error obtaining dependencies for github.com/hyperledger/fabric/core/chaincode/lib/cid:
<go, [list -f {{ join .Deps "\n"}} github.com/hyperledger/fabric/core/chaincode/lib/cid]>: failed with error: "exit status 1"
cannot load package: package github.com/hyperledger/fabric/core/chaincode/lib/cid: cannot find package "github.com/hyperledger/fabric/core/chaincode/lib/cid" in any of:
/opt/go/src/github.com/hyperledger/fabric/core/chaincode/lib/cid (from $GOROOT)
/opt/gopath/src/github.com/hyperledger/fabric/core/chaincode/lib/cid (from $GOPATH)
I am usind Docker to run peer, orderer, ca, and cli containers. The Docker image which is used to build chaincode is hyperledger/fabric-ccenv. This image is created using Dockerfile; the interesting line I found was:
ADD payload/goshim.tar.bz2 $GOPATH/src/
which adds the tar.bz2 inside the $GOPATH/src folder (I believe). The .tar.bz2 file contains all Go packages used by chaincode. I tried to insert the cid package and to create a new .tar.bz2 file with the package inside. Then I rebuilt the image. The image now contains the cid package, but I still get the same error.
Why is it still missing the package?
In the startFabric.sh from your tuna-app, you launch the cli container using:
docker-compose -f ./docker-compose.yml up -d cli
Have a look at the mounting declaration of the persistent volumes in your compose yaml file. You should see something like this because the tuna-app is based on fabcar from the fabric-samples:
./../chaincode/:/opt/gopath/src/github.com/
If you see this declaration, copy in your local machine the folder /hyperledger/fabric/core/chaincode/lib/cid into your chaincode folder. You should find it in chaincode/abac if you are using the last version of fabric samples (https://github.com/hyperledger/fabric-samples).
I think you should not create a new goshim.tar.bz2. If you think it is easier make sure cid is in the correct path within the archive, e.g. github.com/hyperledger/fabric/core/chaincode/lib/cid
To test this you can make a debug output:
ADD payload/goshim.tar.bz2 $GOPATH/src/
RUN ls $GOPATH/src/github.com/hyperledger/fabric/core/chaincode/lib/cid
I would recommend to download cid within the Dockerfile:
RUN go get -d github.com/hyperledger/fabric/core/chaincode/lib/cid
I am trying to use dokku-persistent-storage so my uploads for my rails app stay on the server, but I don't quite understand how to build the path since I am new to Dokku and Docker.
(I am running this on an Ubuntu droplet on Digital Ocean)
I'm not sure if it should be something like this:
[SERVER IP ADDRESS]/home/dokku/myapp/public_folder
or
/home/dokku/myapp/public_folder
or if i'm way off and it should be something completely different.
This is what the github section says about it:
In your applications folder (/home/dokku/app_name) create a file called PERSISTENT_STORAGE.
Inside this file list one volume-map/volume per line to mount. For example:
/host/path:/container/path
/another/container/path
The above example will result in the following arguments being passed to docker during deploy and docker run:
-v /host/path:/container/path -v /another/container/path
Move information on docker volumes can be found here: http://docs.docker.io/en/latest/use/working_with_volumes/
I am not into Ruby or dokku, but if I understood correctly, you want your docker to have a persistent storage on the host machine.
PERSISTENT_STORAGE file, as to the documentation that you've quoted, contains mappings from host file-system directories to your container file-system directories (translated to -v arguments of the CLI).
Therefore, you should map the directory of your uploads in the container, to the desired directory on the host.
For example, if your app's uploads are saved to this dir (inside the docker container):
/home/dokku/myapp/public_folder
and you'd like them to be kept in your host at:
/home/some/dir
then, as I understand, the content of PERSISTENT_STORAGE file should be:
/home/some/dir:/home/dokku/myapp/public_folder
I hope I got you right.
Use Dokku's storage:mount option.
You'll need to SSH into your dokku host:
ssh dokku#host
Run the following command to link the storage directory for that app to the app/public/uploads folder, for example:
storage:mount <app> /var/lib/dokku/data/storage:/app/public/uploads
The Dokku docs cover this well at: at http://dokku.viewdocs.io/dokku/advanced-usage/persistent-storage/