According to the docs, a git url can be passed to a build command:
But what happens if the git url needs to be a branch name? In other words, how do I do the equivalent of this:
git clone -b my-firefox-branch
git#github.com:creack/docker-firefox.git
Start your URL with git:// (or https://) and simply append the branch name after #.
I just forked the OP's repo and created a branch to confirm it works (docker version 1.11.1):
root#box:~# docker build git://github.com/michielbdejong/docker-firefox#michielbdejong-patch-1
Sending build context to Docker daemon 52.22 kB
Step 1 : FROM ubuntu:12.04
12.04: Pulling from library/ubuntu
4edf76921243: Downloading
[==========> ] 9.633 MB/44.3 MB
^Croot#box:~#
See https://docs.docker.com/engine/reference/commandline/build/ for full docs.
So far, No. it can't.
Here's what I got:
$ docker build git#github.com:shawnzhu/docker-ruby.git#branch1
2014/12/04 08:19:04 Error trying to use git: exit status 128 (Cloning into '/var/folders/9q/bthxttfj2lq7jtz0b_f938gr0000gn/T/docker-build-git859493111'...
fatal: remote error:
is not a valid repository name
Email support#github.com for help
)
If you take a look at this line of docker CLI code, it only do recursive git clone against given URL of a git repo (even no --depth=1) when using docker build <git-repo-url>.
However, it could be an interesting improvement to docker (if people want it) since #<branch-name> and #<commit> are popular syntax to github URL adopted by lots of tools like npm and bower.
Well it works more or less depending on versions
For recent versions : (docker-engine 1.5.0-0~trusty and+)
docker build https://github.com/rzr/iotjs.git#master
docker build https://github.com/rzr/iotjs.git
docker build github.com/rzr/iotjs.git
For older ones: (docker.io 1.4-5ubuntu1 and -)
docker build https://github.com/rzr/iotjs.git
docker build git://github.com/rzr/iotjs.git
docker build github.com/rzr/iotjs.git
Maybe this can be handled in helper script like:
curl -sL https://rawgit.com/rzr/iotjs/master/run.sh | bash -x -
Related
English is not my native language, so I apologize in advance for very much possible errors.
I'm new to Docker and Linux in general, and trying to learn rn.
I have a task, I need to create dockerfile based upon tomcat:9.0-alpine, then in this exact file clone provided repository. Then I need to build image from this dockerfile, run container and then visit index.html in a browser, where I will see a specific page.
This how my dockerfile looks like:
FROM tomcat:9.0-alpine
RUN apk update
RUN apk add git
RUN git clone https://github.com/a1qatraineeship/docker_task.git $TOMCAT_HOME/webapps/whateverApp/
When I build image from this, I see that repo was cloned in a directory, that I specified:
#7 [4/4] RUN git clone https://github.com/a1qatraineeship/docker_task.git $TOMCAT_HOME/webapps/aquaApp/
#7 sha256:72b802c3b98dad7151daeba7db257b7b1f1089dc26fb5809fee52f881e19edb5
#7 0.319 Cloning into '/webapps/whateverApp'...
#7 DONE 1.9s
But when I run container and go to http://localhost:8888/whateverApp/, I get "404 - not found".
If I go to http://localhost:8888, I see Tomcat default page, so Tomcat is deffinetely working.
If I bash into container and go to /webapps - I don't see a folder that I specified (whateverApp) in there.
What I'm missing? All seems to work ok, there is no errors thrown, but don't see supposedly cloned repository. In examples that I saw there was no mention about any access restrictions or whatever. Even my teacher said before, that whole dockerfile will essentially consist of 4 lines, and only thing I really need to find out is to where to clone repo for everything to work properly. But If it don't clone at all, how can I verify that I placed files into right place?
The problem is your environment variable, is empty when the container run TOMCAT_HOME
FROM tomcat:9.0-alpine
ENV TOMCAT_HOME=/usr/local/tomcat
RUN apk update
RUN apk add git
RUN git clone https://github.com/a1qatraineeship/docker_task.git $TOMCAT_HOME/webapps/whateverApp/
and with that ENV should work, gook luck!
I have a repo that, when tested on Travis, consistently gives an error like:
$ if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
fatal: git fetch-pack: expected shallow list
The command "if [[ -a .git/shallow ]]; then git fetch --unshallow; fi" failed and exited with 128 during .
Your build has been stopped.
You can see an example here, although it's possible that link requires authorization.
What I find strange about this that it seems the git fetch --unshallow should run only if it's already determined that it is shallow. I should add that I've used this construct in many other repositories and never had a problem.
On my local machine, the contents of .git are:
myrepo.git$ ls -a
. branches config FETCH_HEAD HEAD index logs ORIG_HEAD
.. COMMIT_EDITMSG description gitk.cache hooks info objects refs
I am not hip to the internals of Git, nor do I really understand what git fetch --unshallow is actually supposed to do, but one possibility: check which version of git you have available to your Travis test runners.
I see the same fatal: git fetch-pack: expected shallow list error message with a GitLab CI/CD shell runner on CentOS 7 VMs. CentOS 7 only has Git 1.8.something in the official repos.
This blog post is where I initially became aware of this error message being related to an old version of Git. That post suggests upgrading the OS-default version of git with third-party repos, which I didn't want to do.
What I ended up doing instead, as advised here, was setting:
variables:
GIT_STRATEGY: clone
in the .gitlab-ci.yml. This is, of course, a GitLab-specific thing, but I'm pretty positive Travis gives you some option for setting environment variables for your test environment.
See also:
gitlab runner doesn`t work on a specific project (Ryan Daniels' answer)
I have a private git repository that I have to add to my docker image. For that I clone it locally in the same directory with the Dockerfile and then use the following docker command:
ADD my_repo_clone /usr/src/
My repo has a version tag that I clone, v1. So the files that I clone are always the same.
The problem is that when I build this docker image I always get a new image instead of replacing the old one:
docker build --rm -t "org_name/image_name" .
Apparently, because the ctime of the files change, the docker cache is not seeing my files as identical so I get always a new image, which I want to avoid.
I tries to touch the cloned repo and change atime and mtime to be a fixed date, but it is still not enough.
How can I stop Docker (without changing Docker source code that computes the file hashes and building it again) from creating all the time new images.
Or how can I clone the repo during the image building process? (For this I need SSH forwarding since the repo is private, and I could also not make SSH agent forwarding work during an image build process)
since you don't care about the repository itself and just need the files for tag v1, you could use git archive instead of git clone to produce a tar archive holding the files for tag v1.
Finally, the docker ADD directive to inject the archive into the image.
The mtime of the produced tar archive will be the time of the tag as documented:
git archive behaves differently when given a tree ID versus when given a commit ID or tag ID. In the first case the current time is used as the modification time of each file in the archive. In the latter case the commit time as recorded in the referenced commit object is used instead.
try:
git archive --remote=https://my.git.server.com/myoproject.git refs/tags/v1 --format=tar > v1.tar
I've got Jenkins set up to do 2 things in 2 separate jobs:
Build an executable jar and push to Ivy repo
Build a docker image, pulling in the jar from the Ivy repo, and push image to a private docker registry
During step 1 the jar will have some version which will be appended to the filename (e.g. my-app-0.1-SNAPSHOT, my-app-1.0-RELEASE, etc.). The problem that I'm facing is that in the Dockerfile we have to pull in the correct jar file based on the version number from the upstream build. Additionally, I would ideally like the docker image to be tagged with that same version number.
Would love to hear from the community about any possible solutions to this problem.
Thanks in advance!!
Obviously you need a unique version from (1) to refer to in (2).
0.1 -> 0.2 -> 0.3 -> ...
Not too complicated in terms of how things work together from a build / Docker point of view. I guess the far bigger challenge is to give up SNAPSHOT builds in the development workflow.
With your current Jenkins: release every build you create a container for.
Much better alternative: Choose a CI / CD server that uses build pipelines. And if you haven't already done so, take a look at the underlaying concept here.
You could use the Groovy Postbulid Plugin to extract with a regular expression the exact name of the generated .jar file at the end of step 1.
Then for step 2, you could have a Dockerfile template and replace in it some placeholder with the exact jar name, build the image and push it to your registry.
Or, if you don't use a Dockerfile you could have in your Docker registry a premade Docker image which has everything but the jar file and add the jar to it with those steps:
create a container from the image
add the jar file into the container using the docker cp command
commit the container into a new image
push the new image to your docker registry
Same need by my customer. We ended up by putting placeholders in the Dockerfile, which are replaced using sed just before the docker build.
This way, you can use the version in multiple locations, either in the FROM or in any filenames.
Example:
FROM custom:#placeholder#
ENV VERSION #placeholder#
RUN wget ***/myjar-${VERSION}.jar && ...
Regarding the consistency, a unique version is used:
from a job parameter (Jenkins)
to build the artifact (Maven)
to tag the Docker image (Docker)
to tag the Git repository containing the Dockerfile (Git)
I've stuck with this issue while configuring Jenkins for Nightly build. Please note that the repository project "project1" is large and is about 900MB. Please let me know how should I go around this problem.
Started by user anonymous
Building in workspace C:\Users\user1\.jenkins\jobs\Nightly Build\workspace
Fetching changes from the remote Git repository
Fetching upstream changes from git#github.com:MyOrg/projectgroup/project1
ERROR: Timeout after 10 minutes
FATAL: Failed to fetch from git#github.com:MyOrg/projectgroup/project1
hudson.plugins.git.GitException: Failed to fetch from git#github.com:MyOrg/projectgroup/project1
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:612)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:836)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:861)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1412)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:652)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:557)
at hudson.model.Run.execute(Run.java:1679)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:509)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:230)
Caused by: hudson.plugins.git.GitException: Command "fetch -t git#github.com:MyOrg/projectgroup/project1 +refs/heads/*:refs/remotes/origin/*" returned status code -1:
stdout:
stderr:
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:981)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:920)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.fetch(CliGitAPIImpl.java:187)
at hudson.plugins.git.GitAPI.fetch(GitAPI.java:229)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:610)
... 10 more
I was about to use Workaround #2 but it seems as of git plugin 2.0.3 this can now be configured in the UI, although it is a bit hidden away and is per project.
Go to the configure screen for a project, Source Code Management section, Git, Additional Behaviors, Add, Advanced clone behaviors, Timeout (in minutes) for clone and fetch operation.
I would use workaround #2 as suggested by DevHopeful_2012 if you want a global setting.
It looks like this is a known issue. See JENKINS-20445, JENKINS-20387, and several other issues, which seem to be popping up.
According to these bug reports, this is only happening in the newer version of the Git plugin, so you could downgrade, or try some workarounds:
Workaround #1 - create a local bare reference clone to reduce the time, as stated here.
If you have only 30 kb/second throughput from Jenkins to your git repository, you should probably consider cloning a bare copy of the
repository to central location on the Jenkins server, then perform the
clone with the "Advanced clone behaviour" to use a reference
repository. That will significantly reduce the amount of data
transferred from the git server to Jenkins.
On my Debian Jenkins machine, I do that with:
$ sudo mkdir -p /var/cache/git/mwaite
$ sudo chown mwaite.mwaite /var/cache/git/mwaite
$ cd /var/cache/git/mwaite
$ git clone --bare https://github.com/jenkinsci/jenkins.git
After that bare clone is available on the Jenkins machine, add that
advanced behavior to the job and it should perform much better.
Workaround #2 - increase the timeout to allow for the long initial clone process to complete, using the Git.timeOut property:
java -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar jenkins.war
This SO question is a good example of how to set properties for the Jenkins service. Also, note that this value is in minutes and not seconds. This workaround is courtesy of David.
The latter workaround worked for me. Although, I must admit that I'd prefer to use the reference repo, but I don't believe that it works with a multiconfiguration job, such as mine. And, be forewarned that it was surprisingly painful to set a property for every slave node and then internally document that we have to set this on all nodes, using large Git repos, going forward.
WINDOWS ONLY
There is a possibility that your ssh connection is not configured properly. If the ssh connection to your git repository is not estabilished, Jenkins build may appear as if it hung and eventually timeout.
By default, the Jenkins installer sets up Jenkins to run as a service,
which runs as the “Local System account”, NOT your user account. Since
the “Local System account” does not have SSH keys or known_hosts set
up, “git clone” will fail.
You can just copy C:\Program Files (x86)\Git\.ssh to
C:\Windows\SysWOW64\config\systemprofile\.ssh (the “Local System
account” home)
Also,
IMPORTANT: make sure your ssh keys do NOT have a password! Jenkins
will appear to hang when cloning the repository, but really it’s ssh
blocking in the background waiting for you to input your password.
Detailed steps in http://computercamp-cdwilson-us.tumblr.com/post/48589650930/jenkins-git-clone-via-ssh-on-windows-7-x64
I just found the solution for jenkins timout #10 min errors. This error was coming because of low Internet connection or may be the size of you project in github is large, so jenkins not able load that project from git server.For resolve the problem we have to edit "Additional Behaviours" in Git in jenkins project configuration and increase "Timeout (in minutes) for clone and fetch operations" to "60" or "120" minutes. So now whenever jenkins clone the git project from git server, its enough time to load/clone a project from git server.
Workaround #2 worked for me, I had to change jenkins.xml file.
On Windows:
Go to C:\Program Files (x86)\Jenkins
open jenkins.xml file
add -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 inside <arguments> tag
This is how my configuration looks like
Before:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>
After:
<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=60 -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>
In Ubuntu 14.04 lts
Goto /etc/default/jenkins edit with vi or gedit
update java orgs like below
Before update, it will be like
JAVA_ARGS="-Djava.awt.headless=true"
Update like below
JAVA_ARGS="-Djava.awt.headless=true -Dorg.jenkinsci.plugins.gitclient.Git.timeOut=30"
and restart jenkins like
sudo /etc/init.d/jenkins start
On CentOS/RedHat 7.4 Linux with Jenkins-2.138
Edit /etc/sysconfig/jenkins, modify the JENKINS_ARGS (last line) with the timeout setting mentioned by others in their answers:
# Pass arbitrary arguments to Jenkins.
# Full option list: java -jar jenkins.war --help
#
JENKINS_ARGS="-Dorg.jenkinsci.plugins.gitclient.Git.timeOut=25"