I have code-server installed with docker.
The docker compose file and config folder are under
/home/al3xis/containers/code-server/
├── compose.yaml
└── config/
├── workspace
├── data
└── ...
In my compose.yaml I have linked the volumes:
- ./config:/config
- /home/al3xis/containers:/config/workspace/containers
- /home/al3xis/projects:/config/workspace/projects
I also have the environment input:
- DEFAULT_WORKSPACE=/config/workspace
All is working great in code-server where I see in the workspace my two folders and I can work as expected in them.
But when I went to one of them and cloned my github repository then vscode wouldn't show me any git information.
For example, the .git folder is inside /home/al3xis/projects/sites/mysite and everything works as expected from my terminal but not from inside vscode.
I tried starting git from vscode interface but that created a .git inside the /config/workspace folder which meant that all files from all folders where added in git.
I only want git to be present inside that one folder that I have my website files.
Have I made a simple mistake with linking the volumes?
Related
.
├── A
└── B
├── .dockerignore
├── Dockerfile
└── run_script.sh
My folder structure looks similar to the above tree. I have a Dockerfile, and a .dockerignore file. I use the run-script to build my docker image. In the build, i send the entire context from the root. So, effectively, the context contains both the folders A, and B.
The run script contains the following command:
docker build docker build -f Dockerfile ..
In the .dockerignore file, I would like to ignore certain files which are inside the folder A.
In the dockerignore documents, I can use */ or **/ basically to ignore subdirectories, but using them here does not seem to be effective in ignoring the files (As they are located in the parent/root directory)
These below commands do not help
**/A
*/A
I tried having the .dockerignore file in the root (i.e root contains the folders A, B and .dockerignore -This works). The ignore file contains:
A/
B/
So my question is if there is a solution of ignoring files and folders which are present in the build context, but are technically a level above the stored .dockerignore?
The .dockerignore file has to be placed in the context directory (in your case the folder that contains both A and B).
This is specified in the docs:
To increase the build’s performance, exclude files and directories by adding a .dockerignore file to the context directory.
Also here
Before the docker CLI sends the context to the docker daemon, it looks for a file named .dockerignore in the root directory of the context.
Is it possible to open files in a Docker container in my local operating system outside of the container using the Remote Development extension for VSCode
(such as using right-click 'Open in Explorer' as in the Remote - WSL extension for VSCode)
I tried to:
Build Docker Image Without Context
Mount filesystem Volume
so that I could access files in my operating system from my container without copying.
To build without context I can use docker build - < Dockerfile (see docker docs).
I can mount files via docker run -v <path-to-file-in-host>:<path-to-file-in-container> IMAGE_NAME (as discussed here, and in docker docs here)
From the devcontainer.json reference it's possible to pass build args to Docker build in your devcontainer.json but this doesn't work for the - < as vscode-dev-containers defaults to including build context...
The extension also also automatically runs a container after build so I'm not sure how to override this.
Any tips would be much appreciated!
TL;DR By default dev-containers extension (as of 22/10/2020) bind mounts files from local OS to container so can can open them on local OS.
For faster bind mounting (by skipping Sending build context to docker daemon... during docker build . step) ignore all in .dockerignore & specify bind mount in .devcontainer/devcontainer.json to mount file system to container.
.
├── .devcontainer/devcontainer.json
├── .dockerignore
├── .git
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── data
├── src
└── tests
.dockerignore
# Ignore everything
**
.devcontainer/devcontainer.json
From Changing or removing the default source code mount
{
...
"workspaceMount": "type=bind,source=${localWorkspaceFolder},target=/workspace,consistency=delegated",
"workspaceFolder": "/workspace"
...
}
(backup-plan) docker cp
Can also use docker cp to copy files in and out of a container; if I have csv or txt files in my container's data folder that I want to view in Excel on my local OS I can run:
docker cp 12890c3a2602:/workspaces/drem/data/commercial_building_benchmarks comm_bldg_bmarks
Where 12890c3a2602 is my container id, /workspaces/drem/data/commercial_building_benchmarks is the path to my data folder within my container and comm_bldg_bmarks is the name of the destination folder on my local OS. See Copying Files To And From Docker Containers for more info...
Currently I have a project (repo) in Gitlab which is an angular app. I'm using Gitlab CI/CD to build, test, release and deploy. Releasing will build a new docker image pushing it to the Gitlab registry and after deploying it on NGinx in a docker container on my Digital Ocean droplet. This works fine.
Let's say I want to add a backend to it like the MEAN stack so I would have 2 containers running using a docker-compose file.
container 1 - Angular
container 2 - Node.js, Express.js and MongoDB
The 2 gitlab projects (repo's) will have to be build separately when a change occurs (own Dockerfile and gitlab-ci.yml file) but deployed together using the docker-compose file.
Where do I manage/put the docker-compose file?
I hope my explanation is clear and if I'm assuming correctly.
Thanks in advance.
According to your comment I understand you'd be interested in adopting a monorepo configuration.
In this case, for the question
Where do I manage/put the docker-compose file?
you could just put the docker-compose.yml file at the root of your GitLab CI project, which would lead to a directory structure like this:
monorepo-project/
├── backend/
│ ├── Dockerfile
│ ├── .dockerignore
│ └── src/
├── frontend/
│ ├── Dockerfile
│ ├── .dockerignore
│ └── src/
├── docker-compose.yml
├── .git/
├── .gitignore
└── .gitlab-ci.yml
As pointed out in https://docs.gitlab.com/ee/user/packages/workflows/monorepo.html (the original version of this page, deleted by this commit, is still available at this URL), you can tweak your configuration using the changes: key, so that if just one part of the project changes (e.g., the frontend), then the CI behaves accordingly.
Further pointers
For more examples, see e.g. this article in Medium which specifically relies on Docker, or that blog article which takes advantage of the needs: key.
Finally, the semantics of the GitLab CI YAML conf file is well-documented in https://docs.gitlab.com/ee/ci/yaml/ (to be bookmarked!).
My directory structure:
├── src/
├───── backend/
├─────── Dockerfile
├─────── ...
├───── frontend/
├─────── Dockerfile
├─────── node_modules/
├─────── ...
├───── commons/
├─────── ...
├── .dockerignore
.dockerignore includes the line **/node_modules and my build context is the root directory due to usage of commons.
I'm running both
docker build ... -f src/backend/Dockerfile .
docker build ... -f src/frontend/Dockerfile .
in parallel, and sometimes the backend build will fail on the following specific error:
error checking context: 'file
('/workspace/src/frontend/node_modules/.staging/wrap-ansi-2a6f888f')
not found or excluded by .dockerignore'.
From my understanding (see Why do Node modules go into .staging folder?) the .staging folder is temporary and there might be a race condition when walking it (see Docker cli source).
However, why is this file walked in the first place if node_modules is ignored? Am I misusing the ignore context functionality or is it an actual race condition?
The issue is reproduced in Google Cloud Build environment, where I'm using docker cli via the image gcr.io/cloud-builders/docker, which uses docker client version 19.03.5 and is run with docker server version 18.09.3, API version 1.39.
Clean your system.
docker system prune -a
Note this will remove also all images not used in a container.
My current project consists of a mongo server, a rabbitmq server and a dotnet core service. It is structured as follows:
.
├── project1.docker-compose.yml #multiple docker-compose files for all projects
├── .dockerignore
├── Util/
| └── some common code across all projects
└── Project1/ #there are multiple projects at the same level with the same structure
├── .docker/
| ├── mongodb
| | └──Dockerfile
| └── rabbitmq
| └──Dockerfile
├── BusinessLogicClasses/
| └── some classes that contain my business logic
└── DotNetCoreService/
├── my service code
└── .docker
└──Dockerfile
Right now I am able to use docker-compose command to build the images for mongodb, rabbitmq and the dot net core succesfully. The docker-compose.yml sits at the home directory level because my different projects (in this case Project1) references code found under the Util directory. Therefore I need to be able to provide a context that is above both directories so that I can use COPY operations on the Dockerfile.
My basic project1.docker-compose.yml is as follows (I excluded not important parts)
version: '3'
services:
rabbitmq:
build:
context: Project1/.docker/rabbitmq/
mongodb:
build:
context: Project1/.docker/mongodb/
dotnetcoreservice:
build:
context: ./
dockerfile: Project1/DotNetCoreService/.docker/Dockerfile
As can be seen, the context for the dotnetcoreservice is at the home directory level. Therefore my Dockerfile for that specific image needs to target the full paths from the context as follows:
#escape=`
FROM microsoft/dotnet:2.0-sdk AS build
WORKDIR /app
COPY Project1/ ./Project1/
COPY Util/ ./Util/
RUN dotnet build Project1/DotNetCoreService/
This dockerfile works succesfully when invoked via the docker-compose command at the home directory level, however when invoked via the docker build .\Project1\DotNetCoreService\.docker\ command it fails with the following message:
COPY failed: stat
/var/lib/docker/tmp/docker-builder241915396/Project1: no
such file or directory
I think this is a matter of the actual context because the docker build instruction automatically sets the context to where the Dockerfile is. I would like to be able to use this same directory structure to create images both with the docker-compose build as well as with the docker build instructions.
Is this somehow possible?
Use flag -f to set custom path
Example: docker build --rm -t my-app -f path/to/dockerfile .
May 2022: The new releases of Dockerfile 1.4 and Buildx v0.8+ come with the ability to define multiple build contexts.
This means you can use files from different local directories as part of your build.
Dockerfiles now Support Multiple Build Contexts
Tõnis Tiigi
Multiple Projects
Probably the most requested use case for named contexts capability is the possibility to use multiple local source directories.
If your project contains multiple components that need to be built together, it’s sometimes tricky to load them with a single build context where everything needs to be contained in one directory.
There’s a variety of issues:
every component needs to be accessed by their full path,
you can only have one .dockerignore file,
or maybe you’d like each component to have its own Dockerfile.
If your project has the following layout:
project
├── app1
│ ├── .dockerignore
│ ├── src
├── app2
│ ├── .dockerignore
│ ├── src
├── Dockerfile
…with this Dockerfile:
#syntax=docker/dockerfile:1.4
FROM … AS build1
COPY –from=app1 . /src
FROM … AS build2
COPY –from=app2 . /src
FROM …
COPY –from=build1 /out/app1 /bin/
COPY –from=build2 /out/app2 /bin/
…you can invoke your build with docker buildx build –build-context app1=app1/src –build-context app2=app2/src .. Both of the source directories are exposed separately to the Dockerfile and can be accessed by their respective names.
This also allows you to access files that are outside of your main project’s source code.
Normally when you’re inside the Dockerfile, you’re not allowed to access files outside of your build context by using the ../ parent selector for security reasons.
But as all build contexts are passed directly from the client, you’re now able to use --build-context othersource=../../path/to/other/project to avoid this limitation.