I have a working Pod for a Deployment in Openshift 3 Starter. This is based off an Image stream from a Docker image. However, I cannot get it to build in Openshift with the inbuilt S2I.
The Docker option is not good as I cannot find setting anywhere to make a Image stream update and cause a redeployment.
I tried setting it up so that a webhook would trigger an Openshift Build, but the server needs python 3 with numpy and scipy, which makes the build get stuck. The best I could do is inelegantly get a Python 3 cartridge install numpy based on requirements.txt and the rest via setup.py, but this still got stuck. I have a working webhook going for a different app that runs on basically the same layout bar for requirements (Python3 Pyramid with waitress).
Github: https://github.com/matteoferla/pedel2
Docker: https://hub.docker.com/r/matteoferla/pedel2/
Openshift: http://pedel2-git-matteo-ferla.a3c1.starter-us-west-1.openshiftapps.com
UPDATE I have made a Openshift pyramid starter template.
I would first suggest going back to using the builtin Python S2I builder. If you are doing anything with numpy/pandas, you will need to increase the amount of memory available during the build phase of your application as the compiler runs out of memory when building those packages. See:
Pandas on OpenShift v3
See if that helps and if need be can look at what your other options are around using an externally built container image.
Related
I'm new to docker and have been dabbling with it for the past few days. I've managed to successfully use docker-compose for a multi-container deployment involving an app server (flask + gunicorn) and web server (nginx).
Now, I'd like to recreate the deployment on an offline machine. After doing research, it seems that most have mentioned use docker save and docker load to transfer over the base images. However, I'm wondering whether its possible to recreate the deployment from the image created by docker-compose build? Reason being I would like to skip the entire process of wheeling my python package dependencies for offline use, which I would have to do for the method starting from the base images.
I've tried to save that particular image (output of docker-compose build) and load it on the offline machine, and then tried docker run and docker-compose up but both don't seem to work. Would like to check with the community whether this method is even possible, and if so what's the right way to go about it?
Thanks!
To solve my issue, I ended up making an image of each individual container post pip install, then using docker-compose.yml simply to spin them up. As David mentioned, it doesn't seem possible to spin up the container from the single image output by docker-compose build.
Currently I am developing a software which uses Camunda and interacts with a certain Camunda process.
All of this is intended to be delivered as a single 'package'. To do this, I plan on simply using a Camunda Docker container and another container which runs my software. To put it all together I intend to use docker-compose.
For the Camunda container I currently use the 'official' image which you can find here: https://hub.docker.com/r/camunda/camunda-bpm-platform/
I know that it is not for production use yet, but currently it works fine.
To clean it up a little, I derive from the official image and delete certain files and folders. This is my Dockerfile:
FROM camunda/camunda-bpm-platform:7.11.0
RUN rm -r /camunda/webapps/camunda-invoice
RUN rm -r /camunda/webapps/examples
The problem I am encountering right now is that, as far as I know, you have to use the REST Api of Camunda to deploy a process. I did not find any information on deploying it by putting it in a certain directory or something.
For my use case it would be perfect if the Camunda process was deployed while building the Docker image. But I cannot think of a solution of how to do that.
Theoretically, I have to start the Camunda engine during building, deploy the process and stop it afterwards. Did anyone try that already with the official Camunda docker image? Or do you have a better solution for my problem?
An alternative would be to deploy the process via the software I am developing. But I think that is a rather ugly solution and I would like to avoid that.
Thanks in advance,
Timo
I just came across docker, and was looking through its docs to figure out how to use this to distribute a java project across multiple nodes, while making this distribution platform independent i.e the nodes can be running any platform. Currently i'm sending classes to different nodes and running it on them with the assumption that these nodes have the same environment as the client. I couldn't quite figure out how to do this, any suggestions wouldbe greatly appreciated.
I do something similar. In my humble opinion Docker or not is not your biggest problem. However, using Docker images for this purpose can and will save you a lot of headaches.
We have a build pipeline where a very large Java project is built using Maven. The outcome of this is a single large JAR file that contains the software we need to run on our nodes.
But some of our nods also need to run some 3rd party software such as Zookeeper and Cassandra. So after the Maven build we use packer.io to create a Docker image that contains all needed components which ends up on a web server that can be reached only from within our private cloud infrastructure.
If we want to roll out our system we use a combination of Python scripts that talk with the OpenStack API and create virtual machines on our cloud, and Puppet which performs the actual software provisioning inside of the VMs. Our VMs are CentOS 7 images, so what Puppet actually does is to add the Docker yum repos. Then installs Docker through yum, pulls in the Docker image from our repository server and finally uses a custom bash script to launch our Docker image.
For each of these steps there are certainly even more elegant ways of doing it.
Recently I started to practice Dockers. Basically, I am running a C application on Docker container. Now, I want to try cloud foundry, therefore, trying to understand the difference between the two.
I'll describe the application as a novice because I am.
The application I start as a service(from /etc/init.d) and it reads a config file during startup, which specifies what all modules to load and IP of other services and it's own (0.0.0.0 does not work, so I have to give actual IP).
I had to manually update the IP and some details in the config file when the container starts. So, I wrote a startup script which did all the changes when the container starts and then the service start command.
Now, moving on to Cloud Foundry, the first thing I was not able to find is 'How to deploy C application' then I found a C build pack and a binary build pack option. I still have to try those but what I am not able to understand how I can provide a startup script to a cloud foundry container or in brief how to achieve what I was doing with Dockers.
The last option I have is to use docker containers in Cloud foundry, but I want to understand if I can achieve what I described above.
I hope I was clear enough to explain my doubt.
Help appreciated.
An old question, but a lot has changed since this was posted:
Recently I started to practice Dockers. Basically, I am running a C application on Docker container. Now, I want to try cloud foundry, therefore, trying to understand the difference between the two.
...
The last option I have is to use docker containers in Cloud foundry, but I want to understand if I can achieve what I described above.
There's nothing wrong with using Docker containers on CF. If you've already got everything set up to run inside a Docker container, being able to run that on CF give you yet another place you can easily deploy your workload.
While these are pretty minor, there are a couple requirements for your Docker container, so it's worth checking those to make sure it's possible to run on CF.
https://docs.cloudfoundry.org/devguide/deploy-apps/push-docker.html#requirements
Anyways, I am not working on this now as CF is not suitable for the project. It's an SIP application and CF only accepts HTTP/S requests.
OK, the elephant in the room. This is no longer true. CF has support for TCP routes. These allow you to receive TCP traffic directly to your application. This means, it's no longer just HTTP/S apps that are suitable for running on CF.
Instructions to set up your CF environment with TCP routing: https://docs.cloudfoundry.org/adminguide/enabling-tcp-routing.html
Instructions to use TCP routes as a developer: https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#create-route-with-port
Now, moving on to Cloud Foundry, the first thing I was not able to find is 'How to deploy C application' then I found a C build pack and a binary build pack option.
Picking a buildpack is an important step. The buildpack takes your app and prepares it to run on CF. A C buildpack might sound nice as it would take your source code, build and run it, but it's going to get tricky because your C app likely depends on libraries. Libraries that may or may not be installed.
If you're going to go this route, you'll probably need to use CF's multi-buildpack support. This lets you run multiple buildpacks. If you pair this with the Apt buildpack, you can install the packages that you need so that any required libraries are available for your app as it's compiled.
https://docs.cloudfoundry.org/buildpacks/use-multiple-buildpacks.html
https://github.com/cloudfoundry/apt-buildpack
Using the binary buildpack is another option. In this case, you'd build your app locally. Perhaps in a docker container or on an Ubuntu VM (it needs to match the stack being used by your CF provider, i.e. cf stacks, currently Ubuntu Trusty or Ubuntu Bionic). Once you have a binary or binary + set of libraries, you can simply cf push the compiled artifacts. The binary buildpack will "run" (it actually does nothing) and then your app will be started with the command you specify.
My $0.02 only, but the binary buildpack is probably the easier of the two options.
what I am not able to understand how I can provide a startup script to a cloud foundry container or in brief how to achieve what I was doing with Dockers.
There's a few ways you can do this. The first is to specify a custom start command. You do this with cf push -c 'command'. This would normally be used to just start your app, like './my-app', but you could also use this to do other things.
Ex: cf push -c './prep-my-app.sh && ./my-app'
Or even just call your start script:
Ex: cf push -c './start-my-app.sh'.
CF also has support for a .profile script. This can be pushed with your app (at the root of the files you push), and it will be executed by the platform prior to your application starting up.
https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html#profile
Normally, you'd want to use a .profile script as you'd want to let the buildpack decide how to start your app (setting -c will override the buildpack), but in your case with the C or binary buildpack's, it's unlikely the buildpack will be able to do that, so you'll end up having to set a custom start command anyway.
For this specific case, I'd suggest using cf push -c as it's slightly easier, but for all other cases and apps deployed with other buildpacks, I'd suggest a .profile script.
Hope that helps!
I have GitLab, GitLab-CI and gitlab-ci-multi-runner running on different machines. I've successfully added a runner using docker and the ruby:2.1 image as detailed on https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/install/linux-repository.md
What I'd like to do next is have runners for a minimal Ubuntu 12.04, 14.04 configured. Not knowing docker, I thought I'd try to use the ubuntu:14.04 and ubuntu:12.04 images. However when the start up and try to clone my project repo, they complain about git not being found. I then assumed that with these images, git wasn't installed. So my questions are:
What tools need to be available in a docker image to be used "out-of-the-box" by the gitlab-ci-multi-runner
Are there a set of images already available for various OS's with these already included
Should I really be looking to create my own docker images for this purpose?
Popular base images that contain commonly used software for building software are (based on) buildpack-deps https://hub.docker.com/r/library/buildpack-deps/ (e. g. openjdk is based on that)
In your case you could specify FROM buildpack-deps:stretch-scm which is based on Debian Stretch and includes source code management (SCM) tools like git.
I suppose you could create a Gitlab Runner that uses these images as default. But I think you should always specify the needed image in your .gitlab-ci.yml-file.
I would always hesitate in creating my own Docker Images, because there are a lot available. Still many specific ones are maintained (or better not maintained) by one person for a specific use case that often does not fit your application. The best choice would be to pick a base-image and only add minor RUN commands for the customization.