I am working on a cxx project using docker and cmake to build and I'm now tasked to integrate a third party library that I have locally.
To get started I added a project containing only a src folder and a single cpp file with a main function as well as includes that I will need from the library mentioned above. At this point, I'm already stuck as my included files are not found when I build in the docker environment. When I call cmake without docker on the project then I do not get the include error.
My directory tree:
my_new_project
CMakeLists.txt
src
my_new_project.cpp
In the CMakeLists.txt I've the following content:
CMAKE_MINIMUM_REQUIRED (VERSION 3.6)
project(my_new_project CXX)
file(GLOB SRC_FILES src/*.cpp)
add_executable(${PROJECT_NAME} ${SRC_FILES})
include_directories(/home/me/third_party_lib/include)
What is needed to make this build in the Docker environment? Would I need to convert the third party library into another project and add it as dependency (similar to what I do with projects from GitHub)?
I would be glad for any pointers into the right direction!
Edit:
I've copied the entire third party project root and can now get add include directories with include_directories(/work/third_party_lib/include), but would that be the way to go?
When you are building a new dockerized app, you need to COPY/ADD all your src, build and cmake files and define RUN instructions in your Dockerfile. This will be used to build your docker image that captures all the necessary binaries, resources, dependencies, etc.. Once the image is built, you can run the container from that image on docker, which can expose ports, bind volumes, devices, etc for your application.
So essentially, create your Dockerfile:
# Get the GCC preinstalled image from Docker Hub
FROM gcc:4.9
# Copy the source files under /usr/src
COPY ./src/my_new_project /usr/src/my_new_project
# Copy any other extra libraries or dependencies from your machine into the image
COPY /home/me/third_party_lib/include /src/third_party_lib/include
# Specify the working directory in the image
WORKDIR /usr/src/
# Run your cmake instruction you would run
RUN cmake -DKRISLIBRARY_INCLUDE_DIR=/usr/src/third_party_lib/include -DKRISLIBRARY_LIBRARY=/usr/src/third_party_lib/include ./ && \
make && \
make install
# OR Use GCC to compile the my_new_project source file
# RUN g++ -o my_new_project my_new_project.cpp
# Run the program output from the previous step
CMD ["./my_new_project"]
You can then do a docker build . -t my_new_project and then docker run my_new_project to try it out.
Also there are few great examples on building C** apps as docker containers:
VS Code tutorials: https://blogs.msdn.microsoft.com/vcblog/2018/08/14/c-development-with-docker-containers-in-visual-studio-code/
GCC image and sample: https://hub.docker.com/_/gcc/
For more info on the this, please refer to the docker docs:
https://docs.docker.com/engine/reference/builder/
Related
[EDIT - added clarity]
Here is my current env setup :
$GOPATH = /home/fzd/go
projectDir = /home/fzd/go/src/github.com/fzd/amazingo
amazingo has a go.mod file that lists several (let's say thousands) dependencies.
So far, I used to go build -t bin/amazingo cmd/main.go, but I want to share this with other people and have a build command that is environment-independent. Using go build has the advantage of downloading each dependency once -- and then using those in ${GOPATH}/pkg/mod, which saves time and bandwidth.
I want to build in a multistage docker image, so I go with
> cat /home/fzd/go/src/github.com/fzd/amazingo/Dockerfile
FROM golang:1.17 as builder
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o /bin/amazingo cmd/main.go
FROM alpine:latest
COPY --from=builder /bin/amazingo /amazingo
ENTRYPOINT ["/amazingo"]
As you can expect it, the builder is "naked" when I start it, so it has to download all my dependencies when I docker build -t amazingo:0.0.1 . . But it will do so everytime I call it, which can be several times a day.
Fortunately, I already have most of these dependencies on my disk. I would be happy to share these files (that are located in my $GOPATH/pkg/mod) with the builder, and help it build faster on my machine.
So the question is: how can I share my ${GOPATH} (or ${GOPATH}/mod/pkg) with the builder ?
I tried adding the following to the builder
ARG SRC_GOPATH
COPY ${SRC_GOPATH} /go
and call docker build --build-arg SRC_GOPATH=${GOPATH} -o amazingo:0.0.1 ., but it wasn't good enough - I got an error (COPY failed: file not found in build context or excluded by .dockerignore: stat home/fzd/go: file does not exist)
I hope this update brings a bit more clarity to the problem.
=======
I have a project with a go.mod file.
I want to build that project using a multistage docker image.
(this article is a perfect example)
The issue is that I have "lots" of dependencies, and each of them will be downloaded inside my Docker builder stage.
Is there a way to "share" my GOPATH/pkg/mod with the docker build... command (in some ways, having a local cache) ?
Your end goal isn't completely clear, but the way that I use a multistage build would look something like this for a (dirt-simple) go app, assuming that you ultimately want the docker container to run your go app. You will need to get your source into the build container somehow as well - that is not shown here:
FROM golang:1.17.2-alpine3.14 as builder
WORKDIR /my/app/source/dir
RUN go get && go build -o /path/to/my/app/binary
FROM alpine3.14 AS release
# install runtime deps, if any
# create necessary files and folders, if any
COPY --from=builder /path/to/my/app/binary /usr/local/bin
ENTRYPOINT /usr/local/bin/binary --options
In this way, the source of your application and all dependencies will not be present in the released image, only the compiled binary.
Of course you don't have to specify an output path for that, I think it just makes it a little clearer in this example. And of course you can use whatever base image/images you want to - I'm treating this as though you don't need the go runtime on your release image.
I'm pretty new to development Golang & Docker. I'm following the instructions in the official Golang DockerHub image. Here's the part I'm a bit confused:
The part I really don't get is the last line of the Dockerfile:
CMD ["app"]
My question is, how is the "app" executable created in the first place? I created a standard hello-world.go file and added this Docker file to a directory. I don't get how building the Docker image would generate an executable called "app". Can someone explain?
Excerpt of the go command https://golang.org/cmd/go/#hdr-Compile_and_install_packages_and_dependencies
Compile and install packages and dependencies
Usage:
go install [-i] [build flags] [packages]
Install compiles and installs the packages named by the import paths.
Executables are installed in the directory named by the GOBIN
environment variable, which defaults to $GOPATH/bin or $HOME/go/bin if
the GOPATH environment variable is not set. Executables in $GOROOT are
installed in $GOROOT/bin or $GOTOOLDIR instead of $GOBIN.
When module-aware mode is disabled, other packages are installed in
the directory $GOPATH/pkg/$GOOS_$GOARCH. When module-aware mode is
enabled, other packages are built and cached but not installed.
The -i flag installs the dependencies of the named packages as well.
For more about the build flags, see 'go help build'. For more about
specifying packages, see 'go help packages'.
See also: go build, go get, go clean.
This makes an executable out of your go code.
If I build a cmake file, create an executeble with make and delete everything except the executable, the executable is still functional. Can I,
build the file but the only output is the file that can be executed with ./project
or
have all of the files build, create the executable with make, then delete everything except the executable afterwards
and if so, how do I?
If I am getting this correctly, you want to create a stand-alone binary that cannot be executed even if the docker image does not has any dependencies then you need to use static option during the build - i am not expert in this - maybe as described in the following answer of Compiling a static executable with CMake.
Next you might use a multi-stage builds in docker which will makes you able to have a final minimal image with your executable file only without any build dependencies, just the needed packages for your run-time environment. I have an example not with make, it was created using g++ but achieving the similar concept as below:
FROM gcc:5 as builder
COPY ./hello_world_example.cc /hello_world_example.cc
RUN g++ -o hello_world_binary -static hello_world_example.cc && chmod +x hello_world_binary
FROM debian:jessie
COPY --from=builder /hello_world_binary /hello_world_binary
CMD ["/hello_world_binary"]
And the final result when you run the container:
$ docker run --rm -it helloworldimage:latest
Hello from Dockerized image
Why do you need that?
You can add install() command to your CMakeLists.txt and then call make install to copy your executable into CMAKE_INSTALL_PREFIX directory. If you set CMAKE_INSTALL_PREFIX to an empty dir, you'd end with a directory containing only your executable file.
I'm new to docker. What I want to do is run an openwrt bin file inside a docker container and compile socketman source inside that docker image.
this is the image file
http://download.gl-inet.com.s3.amazonaws.com/firmware/b1300/v1/qsdk-b1300-2.272.bin
I wanted to compile some source(socketman) to openwrt. Here is my work around.
I downloaded the sdk for appropriate firmware. (There is the bin file and also the SDK.)
If you have sdk then you dont have to build the toolchain. tools are already there. (If you get SDK then the build process faster other than compiling whole firmware)
Then cd into the sdk directory. place your source code inside the package directory.
then in the terminal (inside the appropriate SDK folder) type make menuconfig
Then star the package you want to build
save and exit
then type make if you want to log out the debug info type make -j4 V=s
-j4 means the available # of cors, you can put the # of cores that you have on your computer.nproc
If you want to build inside a docker container.
install docker
then clone ubuntu docker image
run the docker image with interactive shell
git clone or wget SDK folder into the docker container
then proceed all the above steps.
I have a big tar/executable (over 30GB) I COPY/ADD it but this is used only for the installation. Once the application is installed I don't need it anymore.
How can I do? I am trying to use it but:
Everytime I run a build, it takes minutes to define the build context.
I'd like to share this image, if I create a tar with docker save, Is the final version or each layer included in it?
I found some solutions that said I can use RUN wget tar ... && rm tar but I don't want to create webserver for that.
Why isn't possible to mount a volume during build process?! It would be very useful.
Use Docker's multi-stage builds. This mechanism allows you to drop intermediate artifacts and therefore achieve a lightweight image.
Example:
FROM alpine:latest as build
# copy large file
# build
FROM alpine:latest as output
# copy necessary files built in the previous stage
COPY --from=build app /app
Anything built in the build stage will not be included in the final image, unless you explicitly COPY them.
Docs: https://docs.docker.com/develop/develop-images/multistage-build/
This is solvable using 2 different context.
Please follow these steps as mentioned below.
Objective is to create a
docker image that will have you large-build file.
docker image that will have you real codebase/executables.
For this you have to create 2 folders (Build & CodeBase) as follow.
Application<br/>
|---> BUILD <br/>
|======|--->Large-File<br/>
|======|--->Dockerfile<br/>
|--->CodeBase<br/>
|======|--->SRC+Other stuff<br/>
|======|--->Dockerfile<br/>
Build & Codebase both folders will have individual Dockerfile and arrange files accordingly.
Dockerfile(Build)
FROM **Base-Image**
COPY Large-File /tmp/Large-File
Build this and tag it with some name like (base-build-app-image)
#>cd Application <==Application root folder as mentioned above==>
#>docker build -t base-build-app-image BUILD <==path of your build-folder==>
Dockerfile(Codebase)
FROM base-build-app-image
RUN *****
CMD *****
RUN rm -f **/tmp/Large-File**
RUN rm -f **Remove installation files that is not required**
ENTRYPOINT *****
Build this-code-base and base-build-app-image is already in your local docker-repository and your large iso file is not in the current-buid-context
#>cd Application <==Application root folder as mentioned above==>
#>docker build CodeBase <==path of your code-base==>
This time since the context size is only your code base and since this doesn't include that Large file - it will definitely reduce your build time.
You can also take an advance of using docker-compose to do both operations together so you will not have to execute 2 separate commands.
If you need help on preparing this docker-compose file then do let me know in comments.
If anything is not clear then leave a comment or come over chat to fix this issue.