I have created a PoC application using hyperledger fabric for three organizations.
Most of the content on internet is instructing to use cloud for the deployment of the solution.
https://juarezjunior.medium.com/how-to-deploy-your-first-hyperledger-fabric-components-with-azure-kubernetes-service-aks-760563d9d543
https://medium.com/google-cloud/fabric-on-google-cloud-platform-97525323457c
https://www.youtube.com/watch?v=ubrA3W1JMk0
If peers, orderers,ca server of all organizations are deployed in a
cloud then how come hyperledger fabric is distribued?
Can this setup can be made in distributed , "in premise"
infrastructure?
Is there any source for reference/links to do this sort of setup?
any suggestions/references will be very helpful.
If peers, orderers,ca server of all organizations are deployed in a cloud then how come hyperledger fabric is distribued?
Each service (orderer, peer, etc) would be run on a different (virtual) machine within the same environment/cloud provider. Those machines could be distributed across various data-centers globally, or across different cloud providers, or even on many varying organisation's hardware. It makes relatively little difference, so long as they're not all on the same box, under the control of one organisation/person. Then it will be distributed.
Can this setup can be made in distributed , "in premise" infrastructure?
Yes - it can be set up to run however you like, although you may run into access issues, from firewalls, etc. Perhaps you'd have different departments run their own peers within a single organisation across many offices.
Is there any source for reference/links to do this sort of setup?
I believe most people start using the scripts from the hyperledger fabric codebase. Theres documentation here https://hyperledger-fabric.readthedocs.io/en/release-1.4/build_network.html about spinning up the "build your first network" script - byfn.sh (looks like this may have been removed)
https://github.com/hyperledger/fabric-samples has a ci folder. Within it there's some steps to build/run/test the codebase examples. Eg. https://github.com/hyperledger/fabric-samples/blob/main/ci/scripts/run-test-network-basic.sh
calls network.sh which is here: https://github.com/hyperledger/fabric-samples/blob/main/test-network/network.sh
If you really want to understand the necessary steps you'll need to work through it all. There's also some good documentation on what the various parts of the hyperledger-fabric system do here: https://github.com/hyperledger/fabric/tree/345a08b5cd30ef937a8270d1722462e4cad1caa9/docs/source - you'll need to navigate through the directories to the parts you're interested in and locate the .md files which contain the documentation : eg.
Peer docs/source/peers/peers.md
Orderer docs/source/orderer/ordering_service.md
Smart Contracts (aka Chaincode) - docs/source/smartcontract/smartcontract.md
Channels - docs/source/create_channel/channel_policies.md
You may also find some Dockerfiles in various repositories with different setups.
Related
My organization manages systems where each client is provisioned a VPS and then their tech stack is spun up on that system via Docker Compose.
Data is stored on-system, using Docker Compose volumes. None of the fancy named storage - just good old direct path volumes.
While this solution is workable, the problem is that this method does not scale. We can always give the VPS more CPU/Memory but that does not fix the underlying issues.
Staging / development environments must be brought up manually - and there is no service redundancy. Hot swapping is impossible with our current system.
Kubernetes has been pitched to me to solve our problems, but honestly I have no idea where to begin - most of the documentation is obtuse and I have failed to find somebody with our particular predicament.
The end goal would be to have just a few high-spec machines running Kubernetes - with redundancy, staging, and the ability to spin up new clients as necessary (without having to provision additional machines or external IPs).
What specific tools would my organization need to use to achieve this goal?
Are there any tools that would allow us to bring over our existing Docker Compose stacks into Kubernetes?
Where to begin: given what you're telling us, I would first look into my options to implement some SDS.
You're currently using local volumes, which you probably won't be able to do with Kubernetes - or at least shouldn't, if you don't want to bind your containers to a unique node.
The most easy way - while not necessarily the one I would recommend - would be to use some NFS servers. Even better: with some DRBD, pacemaker / corosync, using a VIP for failover -- or the FreeBSD way: hastd, carp, ifstated, maybe some zfs. You would probably have to deploy distinct systems scaling your Kubernetes cluster, distributing IOs, ... a single NFS server doesn't last long without its load going over 50 and iowaits spiking ...
A better way would be to look into actual SDS solutions. One I could recommend is Ceph, though there's a lot of new solutions I'm less familiar with ... and there's GlusterFS I would definitely avoid. An easy way to deploy Ceph would be to use ceph-ansible.
Given what corporate hardware you have at your disposal, maybe you would have some NetApp or equivalent, something that can implement NFS shares, and/or some iSCSI gateways.
Now, those are all solutions you could run on the side, although note that you would also find "CNS" solutions (container native), which are meant to be deployed on top of Kubernetes. Ceph clusters can be managed using Rook. These can be interesting, though in terms of maintenance and operations, it requires good knowledge of both the solution you operate and kubernetes/containers in general: troubleshooting issues and fixing outages may not be as easy as a good-old bare-meta/VM setup. For a first Kubernetes experience: I would refrain myself. When you'll feel comfortable enough, go ahead.
In any cases, another critical consideration before deploying your cluster would be the network that would host your installation. Consider that Kubernetes should not be directly deployed on public instances: you would probably want to have some private VLAN, maybe an internal DNS, a local resitry (could be Kubernetes-hosted), or other tools such as an LDAP server, some SMTP relay, HTTP cache/proxies, loadbalancers to put in front of your API, ...
Once you've made up your mind regarding those issues, you can look into deploying a Kubernetes cluster using tools such as Kubespray (ansible) or Kops (uses Terraform, and thus requires some cloud API, eg: aws). Both projects are part of the Kubernetes project and maintained by its community. Kubespray would cover all scenarios (IAAS & bare-metal), integrate with popular SDS out of the box, can ship with various ingress controllers, ... overall offers good defaults, and lots of variables to customize your installation.
Start with a 3-master 2-workers cluster, make sure the resulting cluster matches what you would expect.
Before going to prod, take your time to properly translate your existing configurations. Sometime, refactoring code or images could be worth it.
Going to prod, consider adding a group of "infra" nodes: if you want to host some logging solution or other internal services that are somewhat critical to users and shouldn't suffer outages caused by end-users workloads (eg: ingress routers, monitoring, logging, integrated registry, ...).
Kubespray: https://github.com/kubernetes-sigs/kubespray/
Kops: https://github.com/kubernetes/kops
Ceph: https://ceph.com/en/discover/
Ceph Ansible: https://github.com/ceph/ceph-ansible
Rook (Ceph CNS): https://github.com/rook/rook
I recently finished a project where I created an App consisting of several docker containers. The purpose of the app was to collect some data and safe it to an databank and also allow user interactions over an simple web gui. The app was hosted on four different Raspberry Pi's and it was possible to collect data from all physicial maschines through an api. Further you could do some simple machine learning tasks like calculating anomalies in the sensor data of the Pi's.
Now I'm trying to take the next step and using kubernetes for some load balancing and remote updates. My main goal is to remote update all raspberries from my master node. Which, in theory, would be a very handy feature. Also I want to share the ressources of the Pi's within the cluster for calculations.
I read a lot about Kubernets, Minikube, K3's, Kind and all the different approaches to set up an Kubernetes cluster, but feel like I am missing "a last puzzle piece".
So from what I understood I need an approach which allows me to set up an local (because all machines are laying on my desk/ no cloud needed) multi node cluster. My master node would be (idealy) my laptop, running Ubuntu in a virtual machine. My rasberry's would be my slave/worker nodes. If I would want to update my cluster I can use the kubernetes remote update functionality.
So my question out of this would be: Does it makes sense to use several rasberries as nodes in a kubernetes cluster and to manage them from one master node (laptop) and do you have any suggestions about the way to achieve this setup.
I usally dont like those question not containing any specific code or questions by myself, but feel like an simple hint could accelerate my project noteable. If it's the wrong place please feel free to delete this question.
Best regards
You didn't mention which rpi models you are using, but I assume you are not using rpi zeros.
My main goal is to remote update all raspberries from my master node.
Assuming that by that you mean updating your applications running in kubernetes that is installed on rpi then keep reading. Otherwise ignore all I wrote, and what you probably need is ansible or other simmilar provisioning/configuration-management/application-deployment tool.
Now answering to your question:
Does it makes sense to use several rasberries as nodes in a kubernetes cluster
yes, this is why people created k3s, so such setup is possible using less resources.
and to manage them from one master node (laptop)
assuming you will be using it for learning purpouses then why not. It is possible, but just be aware that when master node goes down (e.g. when you turn off your laptop), all cluster goes down (or at least api-server communication so you wont be able to change cluster's state). Also make sure you are using bridge networking interface for your VM so it is visible in your local network as a standalone instance.
and do you have any suggestions about the way to achieve this setup.
installing k3s on all nodes would be the easiest in your case. There are plenty of resources on the internet explaining how to achieve it.
One last thing I would like to explain is the thing with updates.
Speaking of kubernetes updates you need to know that kubernetes doesn't update itself automatically. You need to explicitly update it. New k8s version is beeing released every 3 months that sometimes "breaks" things and backward compatibility is not possible (so always read changelog before updating stuff because rollbacks may not be possible unless you backed up an etcd cluster earlier).
Speaking of updating applications - To run your app all you do is send yaml files describing your application to k8s and it handles the rest. So if you want to update your app just update the tag on container image to newer version and k8s will handle the updates. Read here more about update strategies in k8s.
disclosure: I work with NEAR and am currently onboarding
based on the docs (docs.nearprotocol.com) and diving into the nearcore/scripts folder, looks like there are currently 5 ways to start a local node for development and testing (ie. developing DApps, integrating with or contributing to the platform) as well as validating (as part of NEAR Stakewars)
they all have the option of starting a Docker container or compiling and running the code natively
(see repo on GitHub #nearprotocol/nearcore/scripts/start_*)
the two that seem most useful to a DApp developer are
start_localnet and
start_testnet
the first launches a node that is totally isolated to local development while the second connects to the NEAR TestNet (via a common list of bootnodes and a telemetry URL)
which of the above does NEAR recommend for local development?
for completeness, here all 5 startup scripts:
start_localnet
totally isolated, not related to NEAR TestNet at all (no bootnodes nor telemetry)
start_testnet
connected to NEAR TestNet via bootnodes and telemetry. apparently can also reuse existing genesis.json file if found
start_stakewars
if you're participating in NEAR Stakewars, this is your startup script
start_unittest
used by nearcore, near-bindgen and near-evm
start_staging_testnet
used by near-bindgen examples "cross-contract-high-level"
PSA1: near-bindgen has some well documented examples -- if you don't already, just decide now that you want to learn Rust
PSA2: near-evm seems like a proof of concept that lets you run Ethereum Smart Contracts on the NEAR Platform
Normally start_testnet should be enough as it lets developers to deploy and test their contract on testnet, which I assume is what most developers want. However, there are certain cases where start_localnet is preferable. For example, if you want to use a different genesis for whatever reason (One reaons I can see is that if you want to see how much things cost with different config parameters), or if testnet is unstable, or if you want to run customized nearcore code that maybe incompatible with testnet code.
Our company is developing an application which runs in 3 seperate kubernetes-clusters in different versions (production, staging, testing).
We need to monitor our clusters and the applications over time (metrics and logs). We also need to run a mailserver.
So basically we have 3 different environments with different versions of our application. And we have some shared services that just need to run and we do not care much about them:
Monitoring: We need to install influxdb and grafana. In every cluster there's a pre-installed heapster, that needs to send data to our tools.
Logging: We didn't decide yet.
Mailserver (https://github.com/tomav/docker-mailserver)
independant services: Sentry, Gitlab
I am not sure where to run these external shared services. I found these options:
1. Inside each cluster
We need to install the tools 3 times for the 3 environments.
Con:
We don't have one central point to analyze our systems.
If the whole cluster is down, we cannot look at anything.
Installing the same tools multiple times does not feel right.
2. Create an additional cluster
We install the shared tools in an additional kubernetes-cluster.
Con:
Cost for an additional cluster
It's probably harder to send ongoing data to external cluster (networking, security, firewall etc.).
3) Use an additional root-server
We run docker-containers on an oldschool-root-server.
Con:
Feels contradictory to use root-server instead of cutting-edge-k8s.
Single point of failure.
We need to control the docker-containers manually (or attach the machine to rancher).
I tried to google for the problem but I cannot find anything about the topic. Can anyone give me a hint or some links on this topic?
Or is it just no relevant problem that a cluster might go down?
To me, the second option sound less evil but I cannot estimate yet if it's hard to transfer data from one cluster to another.
The important questions are:
Is it a problem to have monitoring-data in a cluster because one cannot see the monitoring-data if the cluster is offline?
Is it common practice to have an additional cluster for shared services that should not have an impact on other parts of the application?
Is it (easily) possible to send metrics and logs from one kubernetes-cluster to another (we are running kubernetes in OpenTelekomCloud which is basically OpenStack)?
Thanks for your hints,
Marius
That is a very complex and philosophic topic, but I will give you my view on it and some facts to support it.
I think the best way is the second one - Create an additional cluster, and that's why:
You need a point which should be accessible from any of your environments. With a separate cluster, you can set the same firewall rules, routes, etc. in all your environments and it doesn't affect your current workload.
Yes, you need to pay a bit more. However, you need resources to run your shared applications, and overhead for a Kubernetes infrastructure is not high in comparison with applications.
With a separate cluster, you can setup a real HA solution, which you might not need for staging and development clusters, so you will not pay for that multiple times.
Technically, it is also OK. You can use Heapster to collect data from multiple clusters; almost any logging solution can also work with multiple clusters. All other applications can be just run on the separate cluster, and that's all you need to do with them.
Now, about your questions:
Is it a problem to have monitoring-data in a cluster because one cannot see the monitoring-data if the cluster is offline?
No, it is not a problem with a separate cluster.
Is it common practice to have an additional cluster for shared services that should not have an impact on other parts of the application?
I think, yes. At least I did it several times, and I know some other projects with similar architecture.
Is it (easily) possible to send metrics and logs from one kubernetes-cluster to another (we are running kubernetes in OpenTelekomCloud which is basically OpenStack)?
Yes, nothing complex there. Usually, it does not depend on the platform.
Given that you can upload docker images to both the app engine and the Compute Engine, what's the real difference for a person that always contains their apps in docker?
According to a similar question the difference boils down to PaaS vs IaaS, but with a docker file you always specify the OS and runtime environment. So the only difference I see is you might end up over specifying on App Engine by giving a container.
Fundamentally, if you just want your app to scale seamlessly and quickly without much input, use App Engine Flex. If you want more control that you can configure in different ways using other Google products, consider an unmanaged instance group with Compute Engine.
Some history
It's worth noting that the ability to add Docker images to Compute Engine instances was an extremely recent development. And before that, App Engine Flexible was the new kid in town because we used to only have App Engine standard which definitely didn't allow you to use Docker as a base.
Key Differences
Here are the key differences in my experience:
App Engine is designed as a PaaS product and so you can customize scaling parameters in your app.yaml and App Engine reads those and takes over from you. It's technically true that you can do this using Compute Engine but it involves more configuration - you need to set up an instance group, a backend and a frontend. With App Engine, all of that is taken care of for you
You can't setup any load balancers or any peripheral services or products on top of App Engine. App Engine Flexible went quite a way further to give the user more control and more (sorry) flexibility. However, it doesn't allow unfettered integration with other services. Whereas with Compute Engine, you can setup an HTTPs load balancer, add your machines to different networks and subnets, set custom tags etc...
On an additional note, a more detailed explanation on the difference between App Engine and Compute Engine can be found here.
App Engine is PaaS service and managed service from google. It has options as dynamic instance and resident instance to achieve scaling. It has predefined runtime setup for the supported languages, we just need to execute the code. It supports execution of multiple versions of the applications simultaneously, so we can release the code changes for separate group of users. It's inherently support container environment, so the basic details like number of pods or nodes need no to provide. Even for the App Engine Standard if there is no load than the node count reduced to zero means no cost. App Engine Flexible requires at-least one up instance. Here the deployment can be done by a single command i.e. gcloud app deploy app.yaml
Compute engine is IaaS so developer needs to create a machine, setup the desired software's, setup the docker container environment. There will be no scaling, version management, traffic control,security, firewall, health monitoring and repair etc. So with compute engine it's really tough to achieve the capabilities provided by App Engine. The batter alternative is Kubernetes Engine.