Quick question:
I have two servers, and the initial idea was to use one as application-tier and data-tier, and the other one as build machine.
But it's a relatively small project, so it seems like total overkill to use one server only for the build services (I was assuming a weaker machine would come, then I got surprised).
If I do split app tier and data tier between the two servers, where should I put the Build Service?
On the app tier side or on the data tier side? Which one would be better ?
In our environment, we have the AT and DT separated on their own machines, and host a build service on the AT. What it comes down to is where you have the most "spare" resources available for the build machine. Take a look at perfmon counters for memory and CPU over time, and see which one looks like it is the most lightly used, and put your build agent there.
From experience, that's likely to be your application tier, particularly if you've got the reporting services and the cube installed on the data tier along with the source code repository and work item store.
Related
Our company is developing an application which runs in 3 seperate kubernetes-clusters in different versions (production, staging, testing).
We need to monitor our clusters and the applications over time (metrics and logs). We also need to run a mailserver.
So basically we have 3 different environments with different versions of our application. And we have some shared services that just need to run and we do not care much about them:
Monitoring: We need to install influxdb and grafana. In every cluster there's a pre-installed heapster, that needs to send data to our tools.
Logging: We didn't decide yet.
Mailserver (https://github.com/tomav/docker-mailserver)
independant services: Sentry, Gitlab
I am not sure where to run these external shared services. I found these options:
1. Inside each cluster
We need to install the tools 3 times for the 3 environments.
Con:
We don't have one central point to analyze our systems.
If the whole cluster is down, we cannot look at anything.
Installing the same tools multiple times does not feel right.
2. Create an additional cluster
We install the shared tools in an additional kubernetes-cluster.
Con:
Cost for an additional cluster
It's probably harder to send ongoing data to external cluster (networking, security, firewall etc.).
3) Use an additional root-server
We run docker-containers on an oldschool-root-server.
Con:
Feels contradictory to use root-server instead of cutting-edge-k8s.
Single point of failure.
We need to control the docker-containers manually (or attach the machine to rancher).
I tried to google for the problem but I cannot find anything about the topic. Can anyone give me a hint or some links on this topic?
Or is it just no relevant problem that a cluster might go down?
To me, the second option sound less evil but I cannot estimate yet if it's hard to transfer data from one cluster to another.
The important questions are:
Is it a problem to have monitoring-data in a cluster because one cannot see the monitoring-data if the cluster is offline?
Is it common practice to have an additional cluster for shared services that should not have an impact on other parts of the application?
Is it (easily) possible to send metrics and logs from one kubernetes-cluster to another (we are running kubernetes in OpenTelekomCloud which is basically OpenStack)?
Thanks for your hints,
Marius
That is a very complex and philosophic topic, but I will give you my view on it and some facts to support it.
I think the best way is the second one - Create an additional cluster, and that's why:
You need a point which should be accessible from any of your environments. With a separate cluster, you can set the same firewall rules, routes, etc. in all your environments and it doesn't affect your current workload.
Yes, you need to pay a bit more. However, you need resources to run your shared applications, and overhead for a Kubernetes infrastructure is not high in comparison with applications.
With a separate cluster, you can setup a real HA solution, which you might not need for staging and development clusters, so you will not pay for that multiple times.
Technically, it is also OK. You can use Heapster to collect data from multiple clusters; almost any logging solution can also work with multiple clusters. All other applications can be just run on the separate cluster, and that's all you need to do with them.
Now, about your questions:
Is it a problem to have monitoring-data in a cluster because one cannot see the monitoring-data if the cluster is offline?
No, it is not a problem with a separate cluster.
Is it common practice to have an additional cluster for shared services that should not have an impact on other parts of the application?
I think, yes. At least I did it several times, and I know some other projects with similar architecture.
Is it (easily) possible to send metrics and logs from one kubernetes-cluster to another (we are running kubernetes in OpenTelekomCloud which is basically OpenStack)?
Yes, nothing complex there. Usually, it does not depend on the platform.
I'm working on a big project, which is based on micro service architecture , so consider I have 10 service which some of them have their own database,
these databases are in different technologies (mysql, mongodb , elastic, ... )
so what is the best practice for backup and restore collection of services?
the real problem is these databases are related to each other, for example in my logic backend server I keep oauhId of each user which comes from oauth server,
now consider restore these two databases separately and now my users db in logic server contains some users which there aren't any related records to them on oauth server,
just for your information, I'm using docker , docker-compose, docker swarm for my service orchestration.
As an idea: check how your services depend on each other. If your dependencies are acyclic, you might be able to backup all your data outside-in or inside-out, without running into consistency issues.
Doing so would guarantee you to have no elements in services depending on an inner one after your restore.
If your services show cyclic dependencies, you might be better serviced to have each service redundantly (e.g. master slave replication). Then you can take down the slave instances, taking a backup from the whole lot of slaves while they are offline. That would allow you to create an atomic backup accross all services. However your quality of the backup is then based on the quality of your master slave replication at each service.
Lastly you could keep record of change per service, plus a full backup. Thus you can write your rollback and the start applying the record of change until you reach a consistent state accross the service instances. I think that requires you to have logical dependencies (request identifier) that allows you to correlate the record of change elements (i.e. apply them across the services without the risk to apply them in a way that defies the logical dependencies that occured when clients actually interacted with your services).
I hope these ideas can help you solve your problem :)
I have an data processing application which is updated on a regular basis. This application has a bunch of dependencies which are also updated every now and then. However, different versions of the software (+dependencies) might produce different results (this is expected). The application is run on a remote computer and it can be accessed through a Web page. Every time the user uses the Web page to do some processing she/he also chooses which version of the software he/she wants to use.
Now I am trying to decide which is the best way of keeping track different software (+dependencies) versions. The simplest way of course is to just compile and install each version of my software and its dependencies in a different folder, and then based on the request the user sends, the appropriate folder is selected. However, this sounds very clunky to me. So I thought I could use Docker to keep track of the different software versions. Do you think that it is a good idea? If yes, what is most appropriate to do every time I have a new version of the software (and/or dependencies): 1) Create a new container from scratch with the new version (and end up having multiple containers), or 2) Update the existing container and commit the changes? (I suppose I can access the older commits of the container, right?)
PS: Keep in mind that the reason I looked into Docker and not a simple virtual machine solution is that the application I am running is a high-performance GPU-based software.
Docker is a reasonable choice. Your repository would contain all of the app versions you wish to publish. Note, you will only realize savings if you organize the resulting app filesystem into layers, of which the lower layers are the least likely to change between versions. This will keep the storage requirements at a minimum.
Then you have to decide how you will process each job. A robust (but complex) solution would be to have one or more API containers which take in processing jobs from your user and "dole" them out to worker containers (one or more from each release version). This would provide the lowest response latency and be non-blocking. You can look at different service discovery models to see how your "worker" containers can register with your "manager" containers. This is probably more than you'd like to bite off, but consider using a good key-value database (another container!) like etcd or a 3rd party service discovery tool like zookeeper/eureka/consul.
A much simpler model would have a single API container with one each of the release containers created, but not started. The API container would start, direct, and then stop the appropriate release container. You would incur the startup latency, but this is the least resource intensive... and easiest to manage. But this is a blocking operation.
Somewhere in the middle, but less user friendly is to have each release container running but listening on different host ports (the app always sees the same port). The user would would connect to the port which is servicing the desired release of the app. You'd have to provide some sort of index to make this useful.
I need the VPS services for hosting my ASP.NET project.
However, it's not just asp.net hosting, I also need SQL Server, RabbitMq and either my running conrole app or my windows service.
So I read the suggestions to use Amazon Web Services as they provide first year for free.
However when I registered I found that I don't have a clue of where I am:
I don't see the option of creating a virtual machine with Windows
I don't see the option of setting up SQL Server on such the machine
and so on.
So I was wondering whether I'm in the right place?
Please advise if AWS can provide me with what I need or I came to the wrong place?
AWS can provide all that you listed, but you'll need to do some learning on your end.
Basically you create an EC2 instance, and then use RDP to remote into it, and you can install software and configure it to your hearts content - just like it was any other physical server.
If you want to use SQL Server, you'll have the choice of installing it directly on the instance using your own license, or using their 'hosted' version of SQL Server call RDS. You'll need to read about it and decide which option is better for your project - there is no single right way.
Lastly, I will point out that although the 'free-tier' is nice, except for a really small application (i.e. small db on a low traffic website), you may find out the 'free-tier' does not quite give you all the power you need to run a busy application. I would not base your decision on wether or not you should use AWS on how much 'free' stuff you can get. The free-tier is nice for learning, but plan on spending some money for a truly robust solution.
We have Tfs as single server installation with several TeamCollections. If i configure the build i have to assign 1 BuildController (only one is allowed!) to one TeamProjectCollection.
It's possible (and how) to have more than one BuildControler, because we need several BuildConrtoller - one for each TeamProjectCollection.
Is the only solution to have a own BuildServer for each TeamProjectCollection!?
I'm going to make some assumptions about your environment, if you update the question then I'll update my answer. Also some information about the size of your team would be helpful
Assumption number 1: you have a single server which is acting as your TFS application server and as a build controller and build agent.
Assumption number 2: this single server has 4 processor cores.
If that's the case I would suggest that you get at least 1 other machine to act as the controller / agent (it can be a VM). Firstly for performance and secondly for security. The build service runs as administrator and if someone creates a build that formats a drive or deletes the registry then you've just lost your TFS server!
A build controller can only be associated with 1 Team Project Collection. You can have as many controllers / build agents as you want associated with a collection but they cannot be shared across collections.
You can have as many build agents as you have cores so a 4 core machine can host up to 4 build agents (you should see this option when you install team build on the server) Of course this will have an impact on performance as each controller will be sharing the same RAM and Disk.
You can also set up multiple build controllers on the same machine, but this isn't supported by Microsoft and is considered a hack. Jim Lamb blogged about this and Mark Nichols wrote a nice guide
So to summarise. If you're using a single 4 core server you could potentially host up to 4 build agents, and using the hack a similar number of controllers. This might suit your needs. It's a bit of a hack but it should solve your problem.
The correct (but more expensive) way to do this is to have a VM for each controller and then separate machines for the build agents themselves. For example, we have 2 Team Project Collections. 2 VM's for build controllers (one per collection) these machines don't need to be that powerful as the controller doesn't do much work. We have 16 quad core machines for the build agents themselves. At the moment we only host 2 agents per machine so in effect each Controller / Team Project Collection has 16 build agents. Although we do occasionally move the agents between Controllers if there is a need.