I am developing a distributed application with several small zeromq applications. is there a recommended way to monitor and configure all the zeromq applications? I think of configuring ports and network addresses, starting of applications, monitoring, etc. I know that there exist systems like Nagios, but I wonder if there is an easy and recommended way for zeromq.
This is far from easy, because it depends on your system, its architecture and the requirements e.g. on availability that you have. If I assume that you provide a service that should be restarted when it becomes unavailable, there are a bunch of solutions:
monit: http://mmonit.com/monit/
runit: http://smarden.org/runit/
daemontools
some more alternatives: http://packages.debian.org/jessie/daemontools
None of these have as goal the configuration of services or their deployment. Also, independently monitoring instead of fully controlling your services is still a good idea, so keep your nagios install.
Related
I'm currently rethinking an architecture I was planning.
So suppose I have a system where there are about 8 different services interacting with a single database. Some services listen and react to database events and do stuff like sending SMS.
Then there's an API layer sitting on top of the database and a frontend connected to this API. So in my understanding this is rather monolithic.
In fact I don't see any advantage of using containers in this scenario. Their real advantage is that they can be swapped out, right? My intuition tells me that there is often no purpose in doing that except maybe some load balancing on API level. Instead many companies just seem to blindly jump on the hype train of containerizing everything.
Now the question arises, is docker the right tool for this context? In each forum people refrain from using docker for the sole purpose of a more resource efficient "VM" aggregating all services within a single container. However this is the only real scenario I'd see any advantages in using docker (the environment, e.g. alpine-linux, is the same on all customer's computers when rolling out the system).
Even docker-compose is not "grouping" containers together as a complete system only exposing port 443 but instead starts an infrastructure of multiple interacting containers. Oftentimes services like Kubernetes are then used for deploying these infrastructures on "nodes", i.e. VMs.
However, in my opinion it would be great to have a single self-contained container without putting them into a VM. This container would include every necessary service only exposing one port, e.g. 443.
Since I'm rather confused now, I'd really appreciate your help here.
Thanks in advance!
Kubernetes does many things and has many useful features. But Kubernetes also require that you architect your apps to follow The Twelve-Factor App principles. An important thing here is that your apps are stateless.
When the app is stateless, it is easy to scale out horizontally - this can also be done automatically when the load increases.
When the app is stateless, it is easy to do Rolling Deployments that upgrade the app to a new version without downtime.
You can run containers on bare metal Linux servers, but this is mostly very big servers. If you use a cloud, you probably want more VM instances, but distributed to 3 Availability Zones - for increased availability.
"Self-contained container - exposing one port". With Kubernetes, you typically use a private network and you only expose services via a single load balancer - typically on a port, but different URLs send traffic to different services.
Some services listen and react to database events and do stuff like sending SMS.
As I said, many things is easier when it is horizontal scalable, but this kind of app - that listen for events and react - is one of few examples where you can not scale horizontally. But it is a good fit for a serverless architecture instead, possibly on Kubernetes using Knative.
Now the question arises, is docker the right tool for this context?
My opinion is that most workload will run in containers. It is more a question about how it should be run in Kubernetes - one or multiple replicas. As stateless Deployments or stateful StatefulSet or some other way.
Some background:
I'm building a pretty involved website (as far as used stack concerned). Components among some other smaller stuff include:
Elasticsearch
Redis
ZeroMQ
Couchbase
RethinkDB
traffic through Nginx -> Node
The intention is to have a high available website running but be pretty lean (and low cost) at the same time.
Current topology I'm considering:
2 webservers in active/active config with DNS-loadbalancing. (Nginx, static asset serving, etc. + loadbalancing to the second tier:
2 appservers in active/active. Most of the components like Elasticsearch can do sharding/replication themselves so this should not be as hard to set-up (fingers crossed)
session handling in replicated Redis
Naturally I want monitoring and alerting when something is wrong, and ideally the system should be able to handle failures automatically. Stuff like: promote Redis from Slave to Master, or even initialize a new ec2-instance, if I were to be on Ec2 that is.
However, I want to be free from a particular hosting provider. Which I believe (please correct if wrong) is where Openstack comes in.
Is it correct that:
- openstack allows me to control the entire lifecycle of my website-stack (covering multiple boxes / virtual machines? )
- Does it allow me to (with work on config of course) to spin-up instances, monitor, alert when something goes wrong, take appropriate actions in those scenario's, etc.?
Or is Openstack just entirely the wrong tool for the job? Anything else that would fit better as a sort of "management layer" on top of my entire website?
Thanks
OpenStack isn't VMWare ESX. It's not a very good straight up simple virtual machine hosting environment. If what you want is a way to easily manage virtual machines I might suggest Ganeti. It even has HA failover of virtual machines. In a two physical host environment, this is probably the way to go.
What OpenStack gives you that Ganeti won't is RESTful APIs. It has AWS Compatible APIs, but it has OpenStack APIs that are even better. If you want to automate elasticity or healability this is huge. Being able to link up in python using existing client APIs and just write scripts that spin up instances as needed is something joe DevOps is all about.
So I guess it comes down to what your level of commitment is and what you need. For 2 physical machines OpenStack probably isn't the best solution. But, down the line when you've got more apps and more vms than you can manage manually, openstack will be there to help you write code that makes your datacenter dance to your melodic tunes.
We got these two clusters with eight nodes each, and we are looking for a good cluster framework that would allow us to launch jobs, has an inbuilt scheduler with different scheduling policies and a monitoring system with web frontend. Each of the nodes are running on Ubuntu 11.04. Both commercial and opensource are OK.
Some of them i saw were,
TORQUE and MAUI.(Not sure if it has a web frontend for monitoring)
SLURM and MAUI.
GEXEC and GANGLIA.(Doesn't have a scheduler)
Which product(s) would you recommend? Also is there any advantage using cluster operating systems like MOSIX instead of tools?
The paid version of Maui is called Moab (it normally uses TORQUE as the RM). It also can be sold with monitoring tools as well. I think Moab is a really good product, but I am strongly biased towards it (I work for the company that develops TORQUE/Moab).
I have a CouchDB instance running on one machine, and thus with its own Erlang VM process. If I have another separate Erlang application running on that machine too, is it be better to share the same VM between CouchDB and my application, or is it recommended to start a new Erlang node?
While many would recommend decoupling these subsystems I would take the opposite approach. Erlang has a built in strategy to run many applications on the same release. If your applications talk to each other directly it might make sense for you to bundle them together into a release. This will make calls between the applications faster. Some will argue that all you applications now share the same fate should you need to take the system down for an upgrade that only one of the applications needs. This is a moot point with Erlang where you are distributing your applications across many nodes. Also most upgrades can be done with hot code loading.
There is no problem with running several VMs on the same machnie (at least recent OTP releases), however it is quite handy if you have all your applications on one Erlang node. Easier communication, dependency management, supervision, fault-tolerance - you get it for free in this case, not mentioning maintaining only one 'node' in source control system.
The problem starts with CouchDB. It does not have decent build system which let you use it as one of independent Erlang node applications. So in this case you need to have at least 2 VMs (one acts as Couch daemon, the other one hosts your application)
I'm really impressed with the power of cloud computing when it comes to the possibility to scale up and down your facilities depending on your load.
How can I shift my paradigm and learn to write my applications in that way? Write it once and forget(no matter of the future load) would be the best solution.
How can I practice my skills in that area?
Setup virtualization environment when I can add another VMs into the private cloud(via command line?) on some smart algorithms to foresee the load for some period of time?
Ideally I want to practice it without buying actual Cloud computing services and just on my hardware.
The only thing I want to practice here is app/web role and/or message queue systems scaling when current workers have too many jobs in queue. So let's rule out database scaling from the question's goal as too big topic.
One option I will throw out is to use a native Cloud execution framework. You might look at CloudIQ Platform. One component is CloudIQ Engine. It allows you to develop cloud native apps in C/C++, Java and .NET. You get the capabilities of scale up by simply adding workers to your cloud. The framework automatically distributes your applications to the new machine(s), and once installed, will begin sending work to them as requests come in. So in effect the cloud handles your queueing issue for you.
Check out the Download and Community links for more information.
You should try AWS- Amazon's offering a free tier that gives you storage, messaging and micro instances (only linux). you can start developing small try-outs without paying. writing an application that scales isn't that hard- try to break your flow into small, concurrent tasks. client-server applications are even easier- use a load balancer to raise\terminate servers by demand.