How to build docker-ce from source on macOS - docker

Anyone knows if there's a guide to build from source and replace docker binary on Mac with it?
The readme doesn't say so I try some make target but got https://github.com/docker/for-mac/issues/3353
Edited
What I was trying to do, to be exact, is to debug docker cli to see why the Auth doesn't work for a single developer in my former company, regardless all factor checked and verified to be correct.

To do this, checkout the repo of docker cli (it was confusing at first which part of docker live where). But the cli is at:
git#github.com:docker/cli.git
Build it( this build dist for all platform), assuming you have make already:
make -f docker.Makefile binary cross
Then either use this binary(this is for Mac), for example:
build/docker-darwin-amd64 pull mysql
Or backup and replace your original /usr/local/bin/docker with the binary above.

Since on macOS you're not going to run the engine, you may try a different approach. Building the docker client using the Makefile requires docker engine, and a docker client, which you may not have.
I'm building docker (the client) from the docker/cli repository as a plain Go project:
Clone the repo:
$ git clone https://github.com/docker/cli.git
Build master, or checkout a specific tag:
$ git checkout v19.03.6
cd into the repo, create a build directory, and create the require Go project structure:
$ cd cli
$ mkdir -p build/src/github.com/
$ cd build/src/github.com/
$ ln -s ../../.. cli
cd into the build directory and set the GOPATH:
$ cd ../..
$ export GOPATH=$(pwd)
Build the docker client:
$ go build github.com/docker/cli/cmd/docker
Copy the binary from the build directory, e.g.:
$ cp docker /usr/local/bin
You'll notice that some build-related information is not set:
./docker version
Client:
Version: unknown-version
API version: 1.40
Go version: go1.13.8
Git commit: unknown-commit
Built: unknown-buildtime
OS/Arch: darwin/amd64
Experimental: false
You can pass a suitable -ldflags argument to set those variables as in:
$ go build \
-ldflags \
"-X github.com/docker/cli/cli/version.GitCommit=${docker_gitcommit} \
-X github.com/docker/cli/cli/version.Version=${version} \
-X \"github.com/docker/cli/cli/version.BuildTime=${build_time}\""
provided you have set the docker_gitcommit, version and build_time variables. The escaped quotes in the third flag are required if build_time contain spaces (as the upstream docker binaries do).
Hope this helps.

Related

How do I boot into Mendel OS on the sdcard with the Coral Dev board?

From this instruction it looks like Google allows booting to the sdcard. However, the image they provided (recovery.img) is only an image to boot the board automatically to U-boot, after which you'll still have to flash the MendelOS to eMMC with USB.
I'd like to have the entire OS on the sdcard, that way it's easier to make copy of it and save the state of the OS.
In order to create a MendelOS image that can be flashed in to an sd-card, you'll need to build the image your self. The steps for doing this:
Cloning the Mendel repo's source from here
# Get the repo binary that's necessary to clone Mendel
$ mkdir -p bin
$ export PATH=$PATH:$HOME/bin
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
# Setup your git env
$ git config --global user.name "Your Name"
$ git config --global user.email "you#example.com"
# clone the repo
$ mkdir mendel && cd mendel
$ repo init -u https://coral.googlesource.com/manifest -b release-day
$ repo sync -j$(nproc)
Make sure that the build system on your host machine meets these requirements:
# System
A 64-bit CPU
Kernel 4.15 or newer
binfmt-support 2.1.7 or newer
# OS
The suggested OSes are: Ubuntu 18.10+ or Debian Buster or newer.
# Install qemu
$ sudo apt-get install qemu-user-static
# Install docker
$ sudo apt-get install docker.io
$ sudo adduser $USER docker
Build:
$ sudo su
$ source build/setup.sh
$ FETCH_PACKAGES=true m docker-sdcard
After the build succeeds, you'll see your sdcard image in the out directory. You can install that to your sdcard using the dd command or some opensrouce tools like Balena Etcher.
Enable boot from the sd-card by switching the boot switch to as described ON OFF ON ON here.
Cheers!

Error while running make in Hyperledger Fabric on MacOS

I have installed all the prerequisites for Fabric.
OS: Mac OS X EI Capitan
Docker version 18.03.0-ce, build 0520e24
docker-compose version 1.20.1, build 5d8c71b
go version go1.10.3 darwin/amd64
GOPATH="/usr/local/go"
GOROOT="/usr/local/go"
I have placed the Fabric in /usr/local/go/src/github.com/hyperledger/fabric and added GOPATH in the PATH variable of the system.
While running the make docker command, I get the following error:
Building .build/docker/bin/peer
can't load package: package github.com/hyperledger/fabric/peer: cannot find package "github.com/hyperledger/fabric/peer" in any of:
/opt/go/src/github.com/hyperledger/fabric/peer (from $GOROOT)
/opt/gopath/src/github.com/hyperledger/fabric/peer (from $GOPATH)
make: *** [.build/docker/bin/peer] Error 1
I also went through the Makefile rule for the same:
# We (re)build a package within a docker context but persist the $GOPATH/pkg
# directory so that subsequent builds are faster
$(BUILD_DIR)/docker/bin/%: $(PROJECT_FILES)
$(eval TARGET = ${patsubst $(BUILD_DIR)/docker/bin/%,%,${#}})
#echo "Building $#"
#mkdir -p $(BUILD_DIR)/docker/bin $(BUILD_DIR)/docker/$(TARGET)/pkg
#$(DRUN) \
-v $(abspath $(BUILD_DIR)/docker/bin):/opt/gopath/bin \
-v $(abspath $(BUILD_DIR)/docker/$(TARGET)/pkg):/opt/gopath/pkg \
$(BASE_DOCKER_NS)/fabric-baseimage:$(BASE_DOCKER_TAG) \
go install -tags "$(GO_TAGS)" -ldflags "$(DOCKER_GO_LDFLAGS)" $(pkgmap.$(#F))
#touch $#
The error seems to be the with the line $(BASE_DOCKER_NS)/fabric-baseimage:$(BASE_DOCKER_TAG) \. I tried to replace $(BASE_DOCKER_NS) with absolute path, i.e. /usr/local/go/src/github.com/hyperledger. It again gives the error:
docker: invalid reference format.
The issue is likely that Is /usr/local/go is not shared with Docker. Assuming you are using Docker for Mac, you can check this by right-clicking on the Docker icon in the status bar and selecting Preferences and then the File Sharing tab. You will need to add /usr/local/go if it's not in the list. If you are using Docker Toolbox, you'll need to add shared folders via the VirtualBox GUI.
If you run make docker for Fabric 1.4 and earlier, it there's multiple stages to the build. The first stage involves building the binaries in a Docker container and mounts the current directory as a volume. This is the error you are getting since the host path is not shared with Docker.
If you run make docker on the master branch, you won't run into this issue as master uses multi-stage Docker builds instead.

error while loadingDocker: shared libraries: libsystemd-journal.so.0: cannot open shared object file: No such file or directory

I'm using Docker version:
Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:25:01 UTC 2015
OS/Arch: linux/amd64
Server:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:25:01 UTC 2015
OS/Arch: linux/amd64
I'm on Centos 7
I have a Jenkins-container running in my Docker Environment.
When I'm accessing the Jenkins-container and try to perform a Docker-command I got this error:
libsystemd-journal.so.0: cannot open shared object file: No such file or directory
I tried:[root#localhost lib64]# sudo ln -s /usr/lib64/libsystemd.so.0 libsystemd.so.0
ln: failed to create symbolic link ‘libsystemd.so.0’: File exists
I saw this issue after solving this: question
Here is the same issue: https://botbot.me/freenode/docker/2015-12-01/?page=4
After multiple comments on the previous question, the OP Jenson confirms making it work with:
I will have to make a dockerfile because the run command is too much.
But it works at the moment:
docker run -d --name jenkins --volumes-from jenkins-dv --privileged=true \
-t -i \
-v /var/run/docker.sock:/var/run/docker.sock
-v $(which docker):/bin/docker \
-v /lib64/libsystemd-journal.so.0:/usr/lib/libsystemd-journal.so.0 \
-v /lib64/libsystemd-id128.so.0:/usr/lib/libsystemd-id128.so.0 \
-v /lib64/libdevmapper.so.1.02:/usr/lib/libdevmapper.so.1.02 \
-v /lib64/libgcrypt.so.11:/usr/lib/libgcrypt.so.11 \
-v /lib64/libdw.so.1:/usr/lib/libdw.so.1 \
-p 8080:8080 jenkins
I mentioned that running docker from a container ("cic": "container-in-container") means mounting the docker executable and /var/run/docker.sock.
Apparently, that particular image needs a bit more to run from within a container.
For my developer environment, I'm runnining docker-compose and I connect to the ubuntu image container (14.04 LTS) (I mount the /var/run/docker.sock as well).
Since an update of my host ubuntu system yesterday evening, I had the same error when I wanted to run a docker command inside the dev container :
[dev#docker_dev]:~$ docker ps
docker: error while loading shared libraries: libsystemd-journal.so.0: cannot open shared object file: No such file or directory
So I did an update and I installed libsystemd-journal0 :
[dev#docker_dev]:~$ sudo apt-get update
[dev#docker_dev]:~$ sudo apt-get install libsystemd-journal0
And now my dev container is working fine with docker commands
From the error, it appears that the shared library required by your executable is missing. One way to resolve this issue is:
Use "COPY" command inside Dockerfile to copy the shared libraries/dependencies inside the container. Example: COPY {local_path} {docker_path}
Then, set the environment variable where shared libraries are searched for first before the standard set of directories. For instance for Linux based OS, LD_LIBRARY_PATH is used. Environment variables can be set via Docker's Environment replacement (ENV). Example: ENV LD_LIBRARY_PATH={docker_path}:$LD_LIBRARY_PATH
Other is to statically link your binary with the dependencies (language dependent).

docker suspend and resume using criu

I am building docker from this version of this source code:
https://github.com/boucher/docker/tree/cr-combined
after cloning the code :
git clone -b cr-combined --single-branch https://github.com/boucher/docker.git
cd docker
#make build
#make binary
And then copied the resulting file #./bundles/../docker to the usr/bin directory
After reopening the terminal and starting the docker engine again.
its shows that i am using my own built version but
This version should have two main docker commands that won't show up in my built one
1- checkpoint
2- restore
could you please help me and tell me where it went wrong
Here is what I do:
$ git clone https://github.com/boucher/docker
$ cd docker
$ git checkout cr-combined
$ env AUTO_GOPATH=1 DOCKER_EXPERIMENTAL=1 \
DOCKER_BUILDTAGS='exclude_graphdriver_btrfs \
exclude_graphdriver_devicemapper' ./hack/make.sh binary
$ ./bundles/1.10.0-dev/binary/docker-1.10.0-dev --help | grep checkpoint
checkpoint Checkpoint one or more running containers
restore Restore one or more checkpointed containers
Hope this helps.

How to reproduce a travis-ci build environment for debugging

I am seeing a build failure on travis-ci, which I cannot reproduce on my local machine. Are there instructions somewhere for setting up a VM that is identical to the travis-ci linux build environment? I'm glad to have travis-ci already reveal a new bug, but less excited to debug it by sending in commits that add debug code.
For container-based builds, there are now instructions on how to setup a docker image locally.
Unfortunately, quite a few steps are still manual. Here are the commands you need to get it up and running:
# change the image according to the language chosen in .travis.yml
$ docker run -it -u travis quay.io/travisci/travis-jvm /bin/bash
# now that you are in the docker image, switch to the travis user
sudo su - travis
# Install a recent ruby (default is 1.9.3)
rvm install 2.3.0
rvm use 2.3.0
# Install travis-build to generate a .sh out of .travis.yml
cd builds
git clone https://github.com/travis-ci/travis-build.git
cd travis-build
gem install travis
travis # to create ~/.travis
ln -s `pwd` ~/.travis/travis-build
bundle install
# Create project dir, assuming your project is `me/project` on GitHub
cd ~/builds
mkdir me
cd me
git clone https://github.com/me/project.git
cd project
# change to the branch or commit you want to investigate
travis compile > ci.sh
# You most likely will need to edit ci.sh as it ignores matrix and env
bash ci.sh
You can use Travis Build which is a library (which means you've to place it in ~/.travis/) to generate a shell based build script (travis compile) which can be then uploaded to the VMs using SSH and executed.
Below steps are just guidance in order to get you into the right track (if anything is missing, let me know).
Docker
Example command to run container (which can be found at Docker Hub):
docker run -it travisci/ubuntu-ruby:18.04 /bin/bash
Run your container, clone your repository then test it manually.
See: Running a Container Based Docker Image Locally
SSH access
Check out this answer. Basically you need to setup bounce host, then configure your build to run SSH tunnel.
Here is the example .travis.yml:
sudo: required
dist: trusty
language: python
python: "2.7"
script:
- echo travis:$sshpassword | sudo chpasswd
- sudo sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/' /etc/ssh/sshd_config
- sudo service ssh restart
- sudo apt-get install sshpass
- sshpass -p $sshpassword ssh -R 9999:localhost:22 -o StrictHostKeyChecking=no travisci#$bouncehostip
Local setup
Here are the steps to test it on your local environment:
cd ~
git clone https://github.com/travis-ci/travis-build.git
ln -s ~/travis-build/ ~/.travis/travis-build
sudo gem install bundler
bundle install --gemfile ~/.travis/travis-build/Gemfile
cd repo-dir/
travis login -g <github_token>
vim .travis.yaml
travis lint # to validate script
travis compile # to transform into shell script
Vagrant/VM
After you did travis compile which would produce the bash script as result of your .travis.yml, you can use use vagrant to run this script into virtualized environment using provided Vagrantfile and the following steps:
vagrant up
vagrant ssh
cd /vagrant
bundle exec rspec spec
You probably need to install more tools in order to test it.
Here is some git hint which avoids you to generates unnecessary commits when doing trial & errors commits for Travis CI testing:
Fork the repo (or use separate branch).
After initial commit, keep adding --amend to replace your previous commit:
git commit --amend -m 'Same message.' -a
Push the amended commit by force (e.g. into already opened PR):
git push fork -f
Now Travis CI would re-check the same commit over and over again.
See also: How to run travis-ci locally.
I'm facing the same issue right now. I used to use CircleCI before, where you could just login to VM via ssh, but this doesn't work with Travis-CI VMs.
I was able to debug it (to a certain point) by setting up Travis-ci VM clone via Travis-Cookbooks. You would need to install VirtualBox and Vagrant on your computer first before cloning this repository.
Once you have Travis-Cookbooks cloned, open the folder, launch command prompt|terminal and type vagrant up. Once Vagrant finishes setting up VM (may take a long time) on your machine, you can connect to it via ssh by running vagrant ssh.
From there, you would need to clone your own repository (or just copy the code to VM) and apply the steps from your .travis.yml file.
Eregon's answer failed for me at travis compile, there error looks like:
/home/travis/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- travis/support (LoadError)
I got it working with the following adjustments: (Adjustments marked with # CHANGED. I'm using the node environment)
# change the image according to the language chosen in .travis.yml
# Find images at https://quay.io/organization/travisci
docker run -it quay.io/travisci/travis-node-js /bin/bash
# now that you are in the docker image, switch to the travis user
su travis
# Install a recent ruby (default is 1.9.3) to make bundle install work
rvm install 2.3.0
rvm use 2.3.0
# Install travis-build to generate a .sh out of .travis.yml
sudo mkdir builds # CHANGED
cd builds
sudo git clone https://github.com/travis-ci/travis-build.git
cd travis-build
gem install travis
travis # to create ~/.travis
ln -s `pwd` ~/.travis/travis-build
bundle install
bundler add travis # CHANGED
sudo mkdir bin # CHANGED
sudo chmod a+w bin/ # CHANGED
bundler binstubs travis # CHANGED
# Create project dir, assuming your project is `me/project` on GitHub
cd ~/builds
mkdir me
cd me
git clone https://github.com/me/project.git
cd project
# change to the branch or commit you want to investigate
~/.travis/travis-build/bin/travis compile > ci.sh # CHANGED
# You most likely will need to edit ci.sh as it ignores matrix and env
# In particular I needed to edit --branch=’’ to the branch name
bash ci.sh

Resources