I’m somewhat new to Kubernetes and not sure the standard way to do this. I’d like to have many instances of a single microservice, but with each of the containers parameterized slightly differently. (Perhaps an environment variable passed to the container that’s different for each instance, as specified in the container spec of the .yaml file?)
It seems like a single deployment with multiple replicas wouldn’t work. Yet, having n different deployments with very slightly different .yaml files seems a bit redundant. Is there some sort of templating solution perhaps?
Or should each microservice be identical and seek out its parameters from a central service?
I realize this could be interpreted as an “opinion question” but I am looking for typical solutions.
There are definitely several ways of doing it. One popular option is to use Helm. Helm lets you define kubernetes manifests using Go templates, and package them on a single unit called a Helm Chart. Later on you can install this Chart (install is what Helm calls to save these manifests in the Kubernetes API). When installing the Helm Chart, you can pass arguments that will be used when rendering the templates. That way you can re-use pretty much everything, and just replace the significant bits of your manifests: Deployments, Services, etc.
There are plenty of Helm charts available as open sources projects, that you can use as an example on how to create your own Chart.
And many useful guides on how to create your first Helm Chart.
Here you can find the official docs on developing your own Charts.
As an option you can use StatefulSet with InitContainers plus ConfigMap.
Statefulset will guarantee you proper naming and order.
ConfigMap will let you store fine-grained information like individual properties or coarse-grained information like entire config files.
Configuration data can be consumed in pods in a variety of ways. ConfigMaps can be used to:
1) Populate the values of environment variables
2) Set command-line arguments in a container
3) Populate config files in a volume
For the begging you can review Kubernetes – StatefulSets article where you can find a good explanation on how this pieces work together and inspect prepared example on how to deploy containers from the same image but with different properties.
Related
I'm in the process of designing a web-service hosted with Google App Engine comprised of three parts, a client website (or more), a simple CMS I designed to edit and view the content of that website, and lastly a server component to communicate between these two services and the database. I am new to docker and currently doing research to figure out how exactly to set up my containers along with the structure of my project.
I would like each of these to be a separate service, and therefor put them in different containers. From my research it seems perfectly possible to put them in separate containers and still have them communicate, but is this the optimal solution? Also given that in the future I might want to scale up so that my backed can supply multiple different frontends all managed from the same CMS.
tldr:
How should I best structure my web-service with docker, as well as assuming my back-end supplies more than one front end managed from a CMS.
Any suggestion for tools, or design patterns that make my life easier are welcome!
Personally, I don't like to think of designing whatever in terms of containers. Containers should be good for deployment process, for their main goal.
If you keep your logic in separate components/services you'll be able to combine them within containers in many different ways.
Once you have criteria what suits your product requirements (performance, price, security etc) you'll configure your docker images in the way you prefer.
So my advise is focus on design of your application first. Start from the number of solutions you have, provide a dockerfile for each one and then see what you will have to change.
In aws_cdk there are three different classes for creating lambdas:
aws_cdk.aws_lambda.CfnFunction
aws_cdk.aws_lambda.Function
aws_cdk.aws_sam.CfnFunction
What are the differences and reasons for choosing one over the other?
For every AWS Resource that is supported by CloudFormation there are always two classes. The Cfn version maps directly to what you would normally do in YAML. The other version, in this case aws_lambda.Function, is a higher level class that already sets useful default values to get up and running faster. e.g. memorySize and timeout.
I'm not completely sure about the aws_sam thing, but I wouldn't recommend using it as the libary is not stable and you can achieve the same thing without this libary.
e.g. https://github.com/aws-samples/aws-cdk-examples/tree/master/typescript/api-cors-lambda-crud-dynamodb
Our documentation deals with this subject: https://docs.aws.amazon.com/cdk/latest/guide/constructs.html#constructs_lib
I'm trying to implement a couple of services using terraform and I am not quite sure how to efficiently handle variables (ideally the proper terraform way).
Let's say I want to spin up a couple of vms in a couple of datacenters, one each and every datacenter differs slightly (think aws-regions, VPC-IDs, Securitygroup-IDs etc.)
Currently (in ansible) I have a dict that contains a dict per region containing the configuration specific to the region.
I would like to be able to deploy each datacenter on its own.
I have read through a lot of documentation and I came up with a couple of ways I could use to realise this.
1. use vars-files
have one vars-file per datacenter containing exactly the config per DC and call terraform -var-file ${file}
That somehow seems not that cool, but I'd rethink that if there was a way to dynamically load the vars-file according to the datacenter-name I set.
2. use maps
have loads of maps in an auto-loaded vars-file and reference them by data-center-name.
I've looked at this and that does not look like it's really readable in the future. It could work out if I create separate workspaces per datacenter, but since maps are string -> string only I can't use lists.
3. use an external source
Somehow that sounds good, but since the docs already label the external data source as an 'escape hatch for exceptional situations' it's probably not what I'm looking for.
4. use modules and vars in .tf-file
Set up a module that does the work, set up one directory per datacenter, set up one .tf-file per datacenter-directory that contains the appropriate variables and uses the module
Seems the most elegant, but then I don't have one central config but lots of them to keep track of.
Which way is the 'proper' way to tackle this?
To at least provide an answer to anyone else that's got the same problem:
I went ahead with option 4.
That means I've set up modules that take care of orchestrating the services, defaulting all variables I use to reflect the testing-environment (as in: If you don't specify anything extra you're setting up testing, if you want anything else you've got to override the defaults).
Then I set up three 'branches' in my directory tree, testing, staging and production, and added subdirectories for every datacenter/region.
Every region-directory contains main.tf that sources the modules, all but testing contain terraform.tfvars that define the overrides. I also have backend.tf in all of those directories that defines the backends for state-storage and locking.
I initially thought that doing it this way is a bit too complex and that I may be overengineering the problem, but it turned out that this solution is easier to understand and maintain.
I want to use multiple individual docker files/containers in one combined application. Suppose I have three docker containers. Two containers, Framework A and Framework B, that process data in a two different ways, but give the same output. The other container, main, will process that output. Now in my application I would like to use either Framework A or Framework B, or perhaps both.
I looked at a solution to combine multiple docker containers, but this does not seem to be supported by docker and is also not exactly what I want. So I looked into Docker volumes, but I don't really want to 'write' or store the data from the two individual frameworks, just pass it to the third container. I also read about docker-compose, but again this does not seem to be exactly what I want.
Suppose that in the main container I have some python code, that will call some other code from either Framework A or Framework B, and data then gets passed from either of those two to main (a python object, array or something similar). What would be the best way to achieve this? And is this easily interchangeable, given that the outputs of framework A and framework B are the same?
I can't seem to find a definition of exactly what is a Docker repository. The general approach to labelling seems to be username/imagename.
Of course, it contains Docker images BUT do they need to be different versions of the same image or can they be different Docker images?
For example, could I keep App1, App2, ... in the same Docker repository and just use labels to distinguish them.
By convention the Docker images are named as <owner>/<application>:<tag>. There is not any technical restriction if you want to keep different applications under the same repository (i.e.: there doesn't has to be related different tags unless you force that relation), and you can have, for example, mycompany/myuser:app1 and mycompany/myuser:app2. But this is not the way you will found most of the public images which are tagged as I indicated before.
I believe the answer is that a Docker Repository is a labelled set of versions of an image.
Although it seems technically possible for them to be semantically different images, the difference perhaps being denoted by the label, this will be 1) confusing because they all have the same image name (confusing for humans and software that uses the images), and 2) not in line with the intended use of the repositories as far as I can tell, and 3) probably in opposition to the business model for the Docker Hub hosted registry for public and private repositories.
It was never my intention to attempt to break that business model, I was just confused somewhat since the term repository often means something more general than just a single conceptual entity. Docker provides the ability to privately host your own registries, that contain many repositories.
According to Docker Doc,
A registry is a collection of repositories, and a repository is a collection of images – sort of like a GitHub repository, except the code is already built.
The notation for associating a local image with a repository on a registry, is
username/repository:tag.
The :tag is optional, but recommended; it’s the mechanism that registries use to give Docker images a version. So, putting all that together, enter your username, and repo and tag names, so your existing image will upload to your desired destination:
docker tag friendlyhello username/repository:tag
I also find using the word repository a bit confusing in that context. In the end, the repository is meant to contain all the versions of an image, with the tag latest used to designate the image which will be pulled in case no tag is specified when using the docker pull command.