I want to use Docker as a development environment. I am familiar with the basic Docker concepts such as containers, images, volumes, etc. I am also reading this article.
I think that there are already images specifically created for RoR development. Could someone recommend me a couple of images to start with?
Suppose that I create a container, mount my
working folder (RoR projects). Besides code writing, there are also command line jobs such as Linux tasks (update, install), Rails specific commands (Rake, migrations....). I may need to install new binaries or new gems, change Ruby version using rbenv. How can I accomplish these tasks under Docker? May I type a command in a console or ssh the container?
I managed to create an ubuntu container and run it as following:
docker run -it -v /Users/me/Documents/Projects:/var/source_files ubuntu
It creates a console for my container. Next I guess I can run commands like gem install, apt-get update and etc. Is this how we should configure our environment?
I cannot find information on how to run, how to maintain, how add/remove gems, etc.
It's really up to you and what you're comfortable the most with. I'm assuming solo development on some kind of libraries rather than full-fledged apps[1].
I, for example, tend to use Makefiles when developing on specific Golang projects and have some separate images I tend to use for different occasions. For example, if I have to test a Python / Node scripts, I simply type play and I get into a silly container with a few dependencies pre-installed:
https://github.com/odino/dev#play
https://github.com/odino/dev/blob/master/play/Dockerfile
In my personal experience, though, I found that shell scripts / aliases work very well across projects, so I tend to have simple aliases that work on most projects. If I were you, for example, I would use a minimalistic approach and alias dev to docker run -ti -v $(pwd):/src $RUBY_IMAGE so that you can then run dev rake test, dev rails server etc etc from any project. Your $RUBY_IMAGE should have a few utilities installed (htop, curl and so on) and you should be good to go.
Again, I must stress on the fact that it really depends what you're comfortable with -- most of the times I'm extremely productive with just a Makefile.
[1] if working on full-fledged apps docker-compose works well for a lot of people and has a very good DX. minikube is a tool I'd recommend you to pick up only if you know how to work with kubernetes. We used docker-compose for a long time but have switched to minikube since a few months as it closely mirrors our production environment, and minikube works better (imho) when you have quite a few services talking to each other.
Related
New to Emacs and recently been trying to get used to it. loving it so far!
One thing I cannot seem to figure out by myself nor find any proper examples of how to figure out to following workflow:
Since I work on multiple projects with different languages and like to keep my work and private projects separated as much as possible in my OS, ive been working with development containers using docker and VScode for the past years.
This allowed me to keep both my project dependencies and the development tools in one container, where i just attached my VScode instance to a project and used extensions such as Language servers / linting / debugging from within that container.
Currently I can open my projects in emacs as the code is local and mounted to the containers, but im looking for a way to either:
Allow my local emacs to use the language/linting/debugging services installed in the container.
Install emacs in the dev containers and mount my configs to keep this synchronized.
Or better alternatives?
Most valuable would be to get language servers working again.
In case it matters, i'm working in DOOM Emacs on Arch. Mostly Python, PHP and NodeJS projects.
... use the language/linting/debugging services installed in the container
By design this is difficult to do with Docker: by design the host system can't directly access files or binaries installed in a container. Without a lot of tricks around bind mounts and user IDs and paths and permissions it's very difficult to run a program in a container in a way that looks like it's on the host system. A couple of tools have those tricks built in, but it's not at all universal. (Jenkins for example generates about 5 lines' worth of docker run command options if you ask it to run a step inside a container.)
My Emacs experience has generally been much better using a host-based per-language version manager and per-project packaging tool (a per-project node_modules directory, rbenv plus Ruby gem sets, pipenv for Python programs, ...).
In short: Emacs can't use language servers, language interpreters, or other tools from Docker images instead of the host system (without writing a lot of Lisp (and if you do consider publishing it to MELPA (and also to GitHub))).
Most valuable would be to get language servers working again.
M-x lsp-install-server will download one of the language servers lsp-mode knows about and save it in your $HOME/.emacs directory. If you activate lsp-mode and it doesn't already have a language server for the current major mode, it will offer to download it for you. There's not much to "get working" usually.
I'm new to using docker and so far I'm unable to find many ruby/rails images that contain RVM or rbenv.
The most common thing I see is that each container has multiple tags and each tagged image version has only one version of Ruby installed. See this image for example.
The only way to use another version is to use another tag for the image you are using as you can not install a new version with RVM nor with rbenv.
Is this done on purpose?
Is it a bad practice to use version managers for programming languages inside docker containers?
Why?
This would be considered a bad practice or anti-pattern in docker. RVM is trying to solve a similar problem that docker is solving, but with a very different approach. RVM is designed for a host or VM with all the tools installed in one place. Docker creates an isolated environment where only the tools you need to run your single application are included.
Containers are ideally minimalistic, only containing the prerequisites needed for your application, making them more portable. Docker also uses layers and a union filesystem to reuse common base images for each image, so any copy of something like Ruby version X is only downloaded and written to disk once, ever (ignoring updates to that image).
It depends on how you are going to use it. If you need to install about any version of ruby on the custom docker image without messing around with downloading tarballs and applying patches RVM can be just perfect. RVM is basically a bash script so using it inside of docker container is as bad as using any other bash script inside of docker container.
I'm trying to develop a Rails project without having to install Ruby and all server tools in my Windows local machine. I've created my Docker containers (Ruby and MySQL) and installed the Docker plugin on RubyMine 2016.1, however it seems not very practical for the development daily use, I mean the cycle develop, run, debug, just before deployment to test server.
Am I missing something to make this workflow possible? Or isn't Docker suggested for this step in the development process?
I don't develop under Windows, but here's how I handle this problem under Mac OS X. First off, my Rails project has a Guardfile set up that launches rails (guard-rails gem) and also manages running my tests whenever I make changes (guard-minitest gem). That's important to get fast turnaround time in development.
I launch docker daemonized, mounting a local directory into the docker image, with an exposed port 3000, running a never-ending command.
docker run -d -v {local Rails root}:/home/{railsapp} -p 3000:3000 {image id} tail -f /dev/null
I do this so I can connect to it with an arbitrary number of shells, to do any activities I can only do locally.
Ruby 2.2.5, Rails 5, and a bunch of Unix developer tools (heroku toolbelt, gcc et al.) are installed in the container. I don't set up a separate database container, as I use SQLite3 for development and pg for production (heroku). Eventually when my database use gets more complicated, I'll need to set that up, but until then it works very well to get off the ground.
I point RubyMine to the local rails root. With this, any changes are immediately reflected in the container. In another command line, I spin up ($ is host, # is container):
$ docker exec -it {container id} /bin/bash
# cd /home/{railsapp}
# bundle install
# bundle exec rake db:migrate
# bundle exec guard
bundle install is only when I've made Gemfile changes or the first time.
bundle exec rake db:migrate is only when I've made DB changes or the first time.
At this point I typically have a Rails instance that I can browse to at localhost:3000, and the RubyMine project is 'synchronized' to the Docker image. I then mostly make my changes in RubyMine, ignoring messages about not having various gems installed, etc., and focus on keeping my tests running cleanly as I develop.
For handling a console when I get exceptions, I need to add:
config.web_console.whitelisted_ips = ['172.16.0.0/12', '192.168.0.0/16']
to config/environments/development.rb in order for it to allow a web debug console when exceptions happen in development. (The 192.168/* might not be necessary in all cases, but some folks have run into problems that require it.)
I still can't debug using RubyMine, but I don't miss it anywhere near as much as I thought I would, especially with web consoles being available. Plus it allows me to run all the cool tools completely in the development environment, and not pollute my host system at all.
I spent a day or so trying to get the remote debugger to work, but the core problem appears to be that (the way ruby-debug works) you need to allow the debugging process (in the docker container) to 'reach out' to the host's port to connect and send debugging information. Unfortunately binding ports puts them 'in use', and so you can't create a 'listen only' connection from the host/RubyMine to a specific container port. I believe it's just a limitation of Docker at present, and a change in either the way Docker handles networking, or in the way the ruby-debug-ide command handles transmitting debugging information would help fix it.
The upshot of this approach is that it allows me very fast turnaround time for testing, and equally fast turnaround time for in-browser development. This is optimal for new app development, but might not work as well if you have a large, old, crufty, and less-tested codebase.
Most/all of these capabilities should be present in the Windows version of Docker, as well.
I currently have a LAMP stack installed on my mac running through Homebrew, which, to be honest hardly ever get's used.
Lately I have been working a lot with AngularJS and service based apps, so generally run the sites through a gulp / nodeJS based webserver.
I am totally frontend orientated, so very rarely do I play with backend related technologies other than the odd Drupal site and mysql.
I am interested to learn more NodeJS, perhaps even some Ruby, purely to understand programming more - not really for it to become my new job description.
So reading up on NodeJS a bit last night I read a lot about Docker, and installed it the toolkit and gui this morning. It looks pretty neat!
My question is: Would it work better for me to just run everything I need through Docker? For example, I can just install the mysql container, and turn it on when I need a db, and just spin up a drupal instance when I need one and connect it to my db instance?
I understand that running Docker on Mac is slower as it doesn't have the native Linux kernel and runs through a VM - but considering my needs from it, this should be okay?
I love the idea of just deploying containers, so will probably want to install Docker on my hosting environment too (VM in the cloud).
Follow up question: 90% of the sites I work on are AngularJS based frontends that speak to APIs that our backend guys build separately. Would it be overkill to have a Docker for each of those sites, or would I rather just run them all in one, or just bypass docker entirely for that (as I mentioned, I normally just load them up from within my Gulp's webserver)
Thanks a lot. I realise this is a n00b asking questions about big technology, but I'm trying to wrap my head around it and hopefully grow a bit in the process.
The interest in deploying Docker container is reproducibility.
You can easily reproduce:
either a complex development environment requiring the installation of numerous libraries (that you don't want to pollute directly your host)
or an execution environment, for a given tool to run (like a web server)
If you are not likely to repeat a setup (for dev or exec), a docker container would bring little value.
But if you want to keep track of the exact specification of an environment (through its Dockerfile) and will deploy it not just on your workstation, but in other places as well, then docker is certainly a good option to consider.
I have searched the history a little bit but failed to find a good answer. So I just asked my question here. If there is a good answer already, please redirect it for me. Thanks.
The question is, I found my company's new hire doc lists a bunch of software to install to setup the development environment. Usually it took 1 or 2 days for a new hire to setup everything ready for a new mac. We want to shorten that process. The first thing I thought is Docker.
I read through the user guide of Docker and followed some blogs regarding to how to setup dev environment using Docker but still a little confused if Docker applies to our setting. So here's the detail of requirements:
We need to install a bunch of software (many of them are customized binaries). Right now, we distribute the source code, a new hire need to build from the source code, install it and set environment to include the binary into path. I am wondering if Docker allows us to install customized binaries into it's container?
The source code should not stay in the container. The source code is still checked out in one's local machine using git. Then, how can I rely on the Docker container's environment to build my software? I have searched a little bit is that, you need to mount your folder into the container, and then shell into your container to build? Is that how it works?
We usually develop in mac, does Docker also support mac container or it just allows you to run Linux container using boot2Docker?
Thank you so much in advance for your help.
Some answers :)
First, I think it's a really good idea to use Docker to standardise the development configuration (softwares, custom packages, env variables, ...).
With Docker, you can get your customised binaries from the host, it's not a problem. With the CMD command, you can use bash to install them and add them into your PATH. You can also write a shell script to install all your stuff and launch this script when you build your container
Your code will be on the host and you can "mount" a host folder in your docker image with the -v command. Ex: docker run -v /home/user/code:/tmp/code your_image. I'll detail below how the developer will use your Docker image.
Yep, you have to use Boot2Docker, it works well
Once your development image will be ready, you have to publish it on the official Docker registry (or to host a local registry on your network).
Next, the developer will launch the following Docker command:
docker run -rm -ti your_build_image /bin/bash
This will launch a bash terminal in your Docker image and the developer will be able to compile the code. Ex: cd /tmp/code + mvn clean install
Please have a look to this article to learn about volumes: http://jam.sg/blog/mongodb-docker-part-2/
And this one about Dockerfile: https://www.digitalocean.com/community/tutorials/docker-explained-using-dockerfiles-to-automate-building-of-images
You can also find a lot of Dockerfiles on github (search Dockerfile).
If the goal is to speed up the time it takes to get a Mac setup and usable in your environment, you might want to look at Boxen.
From the "About" section:
"Boxen is your team's IT robot. It's a dangerously opinionated framework that automates every piece of your development environment. GitHub, Inc. wrote the first version of Boxen (imaginatively called “The Setup”) to help employees start shipping on day one."