I am exploring the automated build features of the "New Docker Hub".
I see that it is possible to trigger automated tests for each new pull request that is submitted against a specific branch.
Is it possible to push/publish the images created from a specific pull request?
If this was possible, an open source project could invite folks to test pull requests without needing to re-build the code for a pull request.
Related
We have our base images set up on Dockerhub, and they will rebuild whenever the upstream repositories change.
In our Dockerfiles we install several packages with apt-get.
We'd like to have the most recent versions of these packages at all times. Since these packages have nothing to do with the upstream repo's, we'd need to reinstall them into our base image regularly.
One seemingly simple solution would be to have a scheduled rebuild of our images, for example daily or hourly. Each run would pull in the latest versions and bake it into the base image.
However, I can't find any way to do this. There's no option for it in the Dockerhub UI, and I can't find any reference for an API call or webhook that I can trigger from a cron job.
Has anyone come across a way to set up scheduled builds, or a reason why something this (seemingly) straightforward is unsupported?
There are Build triggers (Trigger your Automated Build by sending a POST to a specific endpoint. in Configure Automated Builds. Unfortunately, this feature was changed recently and I'm not able to find current documentation. There was option to POST some data, e.g. 'docker_tag=dev' to trigger specific build by docker tag/branch/...
I am trying to build a jenkins job(trigger builds remotely) on docker image build, build all I am getting on docker hub is following:
HISTORY
ID Status Date & Time
7345... ! ERROR 10/12/17 10:03
Reason (I assume): Docker is not authenticated to post to the jenkins url.
Question: How can I trigger the job automatically when an image gets pushed to docker hub?
Pull and run Watchtower docker image to poll any third-party public Docker image on Docker Hub or Quay that you need (typically as a base image of your own containers). Here's how. "Polling" here does not imply crudely pulling the whole image every 5 minutes or so - we are monitoring periodically for changes in the image, downloading only the checksum (SHA digest) most of the time (when there are no changes in the locally cached image).
Install the Build Token Root Plugin in your Jenkins server and set it up to receive Slack-formatted notifications secured with a token to trigger builds remotely or - safer - locally (those triggers will be coming from Watchtower container, not Slack). Here's how.
Set up Watchtower to post Slack messages to your Jenkins endpoint upon every change in the image(s) (tags) that you want. Here's how.
Optionally, if your scale is so large that you could end up overloading and bringing down the entire Docker Hub with a flood HTTP GET requests (should the time triggers go wrong and turn into a tight loop) make sure to build in some safety checks on top of Watchtower to "watch the watchman".
You can try the following plugin: https://wiki.jenkins.io/display/JENKINS/CloudBees+Docker+Hub+Notification
Which claims to do what you're looking for.
You can configure a WebHook in DockerHub wich will trigger the Jenkins-Build.
Docker Hub webhooks targeting your Jenkings server endpoint require making periodic copies of the image to another repo that you own [see my other answer with Docker Hub -> Watchman -> Jenkins integration through Slack notifications].
More details
You need to set up a cron job with periodic polling (docker pull) of the source repo to [docker] pull its `latest' tag, and if a change is detected, re-tag it as your own and [docker] push to a repo you own (e.g. a "clone" of the source Docker Hub repo) where you have set up a webhook targeting your Jenkings build endpoint.
Then and only then (in a repo you own) will Jenkins plugins such as Docker Hub Notification Trigger work for you.
Polling for Dockerfile / release changes
As a substitute of polling the registry for image changes (which need not generate much network traffic thanks to the local cache of docker images) you can also poll the source Dockerfile on Github using wget. For instance Dockerfiles of the official Docker Hub images are here. In case when the Github repo makes releases, you can get push notifications of them using Github Watch > Releases Only feature and if they have CI docker builds. Docker images will usually be available with a delay after code releases, even with complete automation, so image polling is more reliable.
Other projects
There was also a proposal for a 2019 Google Summer of Code project called Polling Docker Registries for Image Changes that tried to solve this problem for Jenkins users (incl. apparently Google), but sadly it was not taken up by participants.
Run a cron job with a periodic docker search to list all tags in the docker image of interest (here's the script). Note that this script requires the substitution of the jannis/jq image with an existing image (e.g. docker run --rm -i imega/jq).
Save resulting tags list to a file, and monitor it for changes (e.g. with inotifywait).
Fire a POST request using curl to your Jenkins server's endpoint using Generic Webhook Trigger plugin.
Cautions:
for efficiency reasons this tags listing script should be limited to a few (say, 3) top pages or simple repos with a few tags,
image tag monitoring relies on tags being updated correctly (automatically) after each image change, rather than being stuck in the past, like say Ubuntu tags (e.g. trusty-20190515 was updated a few days ago - late November, without the change in its mid-May tag).
I run a docker image for data processing on a windows server 2016 (single vm, On Premises). My image is stored in a Azure Container Registry. The code does not change often. To get security Updates I like to get a rebuild and release after the microsoft/windowsservercoreis updated.
Is there a Best Practice Way to do this?
I thought about 3 ways of solving this:
Run a scheduled build every 24h, pull the microsoft/windowsservercore, pull my custom image, run powershell to get the build dates and compare then (or use some of the histroy ids). If a rebuild is needed, build the new image and tag the build. Configure the Release to run only on this tag.
Run a Job to check the update time of the docker image and trigger the build with a REST request.
Put a basic Dockerfile on github. Set up automated Build with a trigger to microsoft/windowsservercore and configure the webhook to a WebService, which start the Build with REST.
But I really like non of these Ideas. Is there a better option?
You can use Azure Container Registry webhooks directly, the simple workflow:
Build a Web Api project to queue build per to detail request (webhook request) through Queue a build Rest API
Create an Azure Container Registry webhook to call Web API (step1)
I choose option three. Therefore I set up a github repository with a one line Dockerfile:
FROM alpine
I used the alpine image and not the windowsservercore, because automated build does currently does not support windows images. I configured a automated build in the docker hub and add a Linked Repositories to microsoft/windowsservercore.
Then I set up a MS Flow with a HTTP Request Trigger to start the Build. Add the Flow URL to a new webhook on the automated build.
For me this are to many moving parts that has to configured and work together, but I know no better way.
I have a CI/CD setup with a Jenkins server to manage our internal CI/CD. We have Codeship performing our CI/CD for our AWS work.
I'm looking to setup jobs on our Jenkins server to manage when new builds are triggered on Codeship.
The aim being, we will have our Jira dashboard integrated with Jenkins in such a way that as an issue's status changes, specific jobs are executed.
So I'm trying to create a job that uses Codeship's API to trigger a new build, but it appears that you can only rerun an old build? How do you trigger a fresh build?
From the docs enter link description here you can only retrieve information and restart previous builds.
You want to run specific jobs, but those must be associated with some specific commit on your repository. You can identify the build for that specific commit and restart it.
Builds are always triggered from your git repository (github or bitbucket), and Codeship is highly dependent on that to keep the flow as simple as possible. You don't need to upload anything anywhere and then command Codeship to run a build on that. All you need is specify a repository and push something.
You could create an internal git server where your developers push to and with jenkins you can push changes from there to a repository connected to Codeship. That way you could control indirectly what gets tested and what does not.
I have a public docker hub repository, automated build linked to a github repo.
I found I misnamed the tag of my last build.
Is that possible to re-edit the image name manually after building process without influencing the image ?
For the Automated builds, manually pulling, re-tagging and pushing won't work.
First, even if you pull and re-tag your image, you cannot push manually to an Automated Build. You will end up getting Error pushing to registry: Authentication is required.
The true solution would be to go to your Build Details Page, Click on Settings -> Automated Build -> Edit the tag name under Docker Tag Name and hit Save and trigger build. This will create a new tag and triggers the build.
Secondly, you cannot delete the tags (for Automated Builds) on your own. Please contact support#docker.com asking them to delete the tag.
Also, you should refrain from using HTTP DELETE request for Docker Hub. These API Endpoints are only meant for private registry and not for Docker Hub till date. Docker is planning to release the V2 registry Endpoint soon, after which you can safely use the API calls to delete/manipulate tags and images. Until then do not use V1/V2 Endpoints for deleting tags.