Get the last element of a docker image path using Go templates - docker

I would like to generate a synthetic docker ps table but my docker image paths are very long. The solution I am trying to do is to display only the last part of the image name:
This is what I have:
docker ps --format "table {{.Names}}\\t{{.Image}}\\t{{.Status}}\\t{{.Command}}"
NAMES IMAGE STATUS COMMAND
a_container a_local_image Up 36 minutes "python…"
another_container registry.example.com/group/subgroup/project/my_image:latest Up 38 minutes "go…"
I would like:
docker ps --format "table {{.Names}}\\t{{ <magic .Image> }}\\t{{.Status}}\\t{{.Command}}"
NAMES IMAGE STATUS COMMAND
a_container a_local_image Up 36 minutes "python…"
another_container my_image:latest Up 38 minutes "go…"
so basically, get what is after the last /, like basename does.
This is what I tried:
# No fixed length
docker ps --format "table {{ slice .Image 0 10}}"
# No last index available
docker ps --format 'table {{ (split "/" .Image) }}'
Any help or workaround is appreciated.
p.s.: since I use docker cli, I don't think I can define custom functions to use in the template expression.

There is no good way to get the last element of a template array. Even though you can use index and len built-ins, you can't use arithmetic to get len - 1.
This is a template-only trick:
{{ $image := "" }}{{ range split .Image "/" }}{{ $image = . }}{{ end }}{{ $image }}
Basically it does the following:
declare a var $image
split the .Image on /
iterate over the array string and just assign to $image
at the end of the range, $image will hold the value of the last element
print $image
Full command (with enclosing apex ' so you don't have to escape quotes):
docker ps --format 'table {{.Names}}\t{{ $image := "" }}{{ range split .Image "/" }}{{ $image = . }}{{ end }}{{ $image }}\t{{.Status}}\t{{.Command}}'

You may could use next workaround, I put it in a shell file just because it's too long, but you are ok to call it directly in shell.
test.sh
paste <(docker ps --format 'table {{.ID}}') \
<(docker ps --format 'table {{.Image}}' | awk -F '/' '{print $NF}') \
<(docker ps --format 'table {{.Command}} {{.CreatedAt}} {{.Status}} {{.Ports}} {{.Names}}')
Explain:
docker ps --format 'table {{.ID}}' get the ID column of docker ps
docker ps --format 'table {{.Image}}' | awk -F '/' '{print $NF}'), frist get the Image column, then use awk to split /, get the last sub column as you required.
the left is used to get remained columns as you needed of docker ps
Finally, use paste command to combine the output of above command.

Related

How to list paginated image tags from quay.io using regctl

Regctl provides - among other things - a CLI for listing image tags available on a registry, but when the image has a lot of tags, pagination gets in the way.
The problem is that so far I've not found a way to obtain other than the 1st 50 tags on quay.io.
i.e. if I get tags for calico/node image from dockerhub I get
/tmp$ regctl tag ls calico/node | wc -l
8225
but when it comes to quay.io it seems to return just the first 50 tags, that is why regctl provides a specific flag (from regctl tag ls --help)
--last string Specify the last tag from a previous request for pagination
but the --last param seems not working on quay as regctl seems to return the very same contents as when invoked without the flag
/tmp$ regctl tag ls quay.io/calico/node | wc -l
50
/tmp$ regctl tag ls quay.io/calico/node | tail -n 3
v3.4.0-0.dev-27-g319e739-ppc64le
v3.4.0-0.dev-28-g909229b-amd64
v3.4.0-0.dev-28-g909229b-arm64
/tmp$ regctl tag ls quay.io/calico/node --last 'v3.4.0-0.dev-28-g909229b-arm64' | tail -n 3
v3.4.0-0.dev-27-g319e739-ppc64le
v3.4.0-0.dev-28-g909229b-amd64
v3.4.0-0.dev-28-g909229b-arm64
The parameter is working, but only kinda:
$ regctl tag ls quay.io/calico/node --last v3.3.0-4-g70e7d79-ppc64le --limit 5
v3.3.0-2-g441d7c2-arm64
v3.4.0-0.dev-25-g17db8bd
v3.4.0-0.dev-25-g17db8bd-amd64
v3.4.0-0.dev-25-g17db8bd-arm64
v3.4.0-0.dev-25-g17db8bd-ppc64le
Something that jumps out there is that the last tag is included in the new list, which isn't valid from the OCI spec. Instead, the results should look like what Hub does:
$ regctl tag ls calico/node --limit 10
06d8348
0b3286c-e9613934-1eb2608
2606b97-e0ce0bc-7a66b57
2606b97-e0ce0bca-7a66b57
383a737
43b069c
4d0d1ec
4fac26c-0ff3c42a-f442384
53afe8a
542c040
$ regctl tag ls calico/node --limit 10 --last 542c040
684ef2b
6889069
69c3089
6ce4f0a
714dda5-36dc8f6f-aa95469
714dda5-48a4a83d-aa95469
714dda5-6af11396-aa95469
765dc25
91d54c3
9512289
I've also noticed when I increase the limit, I'll start seeing entries returned from even further before the "last" entry that I requested, and it's using a previous "last" parameter for these offsets. Most likely it's some weird caching happening, which is documented in this Jira issue. For right now, it appears that Quay is not following the OCI distribution-spec which makes this difficult to test and use, and will need to be fixed by Quay.

Getting Mount Points using awk & grep

I am trying to get mount points and their respective paths on linux. So when I run the mount -v command I get this example output
//cifst/FSR on /mnt/share/cifst/FSR type cifs ...
//sydatsttbsq01/TheBooks statements to be parsed on /mnt/share/TheBooks type cifs ...
I am trying to parse this text to display this output
/mnt/share/cifst/FSR;//cifst/FSR
/mnt/share/TheBooks;//sydatsttbsq01/TheBooks
But the /mnt on the first row is in column 3, while on the second row is in column 5 so how do I do this to get the /mnt part
mount -v | grep mnt | awk '{ print $1'} gets me the path but how do I get the mount points.
Lots of assumptions, but this works for your sample input/output:
$ cat << EOF | awk '{print $(NF-2), $1}' OFS=\;
> //cifst/FSR on /mnt/share/cifst/FSR type cifs
> //sydatsttbsq01/TheBooks statements to be parsed on /mnt/share/TheBooks type cifs
> EOF
/mnt/share/cifst/FSR;//cifst/FSR
/mnt/share/TheBooks;//sydatsttbsq01/TheBooks
The trick is to notice that it's not column 3 and 5 you're interested in, but in each case it is column NF - 2.
In this particular case, the grep is redundant because it matches each line of input, and in general grep is (almost) always redundant with awk. If you need to add the filter, do it with awk and use:
awk '/mnt/{print $(NF-2), $1}' OFS=\;
If the fields you are interested in are #1 and the next after the first field equal to "on", and they do not contain spaces, you could try this:
mount -v | awk '{a="";for(i=2;i<=NF;i++){if(a=="on")break;a=$i};print $i";"$1}'
If we add one more hypothesis that there is only one field equal to "on", another possibility is to use gensub:
mount -v | awk '{print gensub(/^(\S+).*\<on\>\s+(\S+).*/,"\\2;\\1",1)}'
Which brings us to a sed equivalent:
mount -v | sed -r 's/^(\S+).*\<on\>\s+(\S+).*/\2;\1/'
For this particular output something like this will work; bear in mind that it will break if any of the paths with spaces that you use have the word "on" in them.
mount -v | awk 'BEGIN{FS="( on | type )"; OFS=";"} $3 ~ /cifs/ {print $2,$1}'
/mnt/share/cifst/FSR;//cifst/FSR
/mnt/share/TheBooks;//sydatsttbsq01/TheBooks statements to be parsed
P.S.: You'd be much better off if you didn't use spaces in paths, replace them with ., or _, or camelcase them ...

Docker: How to print the last n characters of a field using --format

I'm trying to write a docker formatter that will make the output of docker container ls more readable. I would like to truncate the Image field so that only the last n characters are printed (some of the images i deal with have quite long paths).
E.g: How do I modify {{.Image}} so that output goes from this:
docker container ls --format "table {{.ID}}\t{{.Image}}"
43b15196a114 nginx:1.17.2-alpine
5cfb448d2aa5 this/container/has/a/very/long/path/elasticsearch:6.8.0
To this:
43b15196a114 nginx:1.17.2-alpine
5cfb448d2aa5 th/elasticsearch:6.8.0
There are slice function for this, but there are several caveats:
Slice indexes should not exceed array (string) bounds.
.Image field is of some strange type (not string), so you should evaluate it to string (using print) and assign some name to it.
docker container ls --format 'table {{.ID}}\t{{with $i:=(print .Image)}}{{if le (len $i) 20}}{{$i}}{{else}}{{slice $i (slice $i 20|len)}}{{en
d}}{{end}}'

docker inspect: how to select only certain keys from range

I created an docker image that has few labels, here is my Dockerfile section on LABELS:
grep LABEL Dockerfile
LABEL "css1"="/var/www/css1"
LABEL "css2"="/var/www/css2"
LABEL "img"="/var/www/img"
LABEL "js"="/var/www/js"
Then:
docker image inspect --format='{{.Config.Labels}}' labels-test
map[css1:/var/www/css1 css2:/var/www/css2 img:/var/www/img js:/var/www/js]
I need to get for example all labels starting with css. This is as far as i was able to figure:
docker image inspect --format='{{range $k,$v:=.Config.Labels}}{{$k}}:{{$v}} {{end}}' labels-test
css1:/var/www/css1 css2:/var/www/css2 img:/var/www/img js:/var/www/js
Desired output would be:
css1:/var/www/css1 css2:/var/www/css2
The Go template functions are available in golang docco
eq can test if arg1 == arg2.
printf "%.3s" $k will give you the first 3 chars of a string.
docker image inspect \
--format='{{ range $k,$v:=.Config.Labels }}{{ if eq (printf "%.3s" $k) "css" }}{{ $k }}:{{ $v }} {{end}}{{end}}' \
IMAGE
You might want to look at the querying the Docker API images endpoint /images/IMAGE/json directly or processing the JSON output somewhere if you need to do any more advanced processing:
docker image inspect \
--format='{{json .Config.Labels}}' \
IMAGE
You can do something like
docker inspect --format='{{index (index (.Config.Labels)).css1 }}' labels-test
which shows for me
/var/www/css1
and also
docker inspect --format='{{index (index (.Config.Labels)).css2 }}' labels-test
which shows for me
/var/www/css2
See my previous answer on that subject
How to get ENV variable when doing Docker Inspect
Edit
The following gives exactly what you ask for
docker inspect --format='{{index (index (.Config.Labels)).css1 }} {{index (index (.Config.Labels)).css2 }} labels-test
as i get
/var/www/css1 /var/www/css2

Escaping double curly braces in Ansible

How to escape double curly braces in Ansible 1.9.2?
For instance, how can I escape double curly braces in the following shell command?
- name: Test
shell: "docker inspect --format '{{ .NetworkSettings.IPAddress }}' instance1"
Whenever you have problems with conflicting characters in Ansible, a rule of thumb is to output them as a string in a Jinja expression.
So instead of {{ you would use {{ '{{' }}:
- debug: msg="docker inspect --format '{{ '{{' }} .NetworkSettings.IPAddress {{ '}}' }}' instance1"
Topic "Escaping" in the Jinja2 docs.
This:
- name: Test
shell: "docker inspect --format {% raw %}'{{ .NetworkSettings.IPAddress }}' {% endraw %} instance1"
Should work
Another way to do is using backslashes like \{\{ .NetworkSettings.IPAddress \}\}
Hope it helps
Tried on with ansible 2.1.1.0
{%raw%}...{%endraw%} block seems the clear way
- name: list container images and name date on the server
shell: docker ps --format {%raw%}"{{.Image}} {{.Names}}"{%endraw%}
Only need to escape leading '{{'
tasks:
- name: list container images and names
shell: docker ps --format "{{'{{'}}.Image}} {{'{{'}}.Names}}"
No harm to escap the tailing '}}', except more difficult to read.
tasks:
- name: list container images and names
shell: docker ps --format "{{'{{'}}.Image{{'}}'}} {{'{{'}}.Names{{'}}'}}"
Backslash '\' seems do not work
New in Ansible 2.0 is the ability to declare a value as unsafe with the !unsafe tag.
In your example you could do:
- name: Test
shell: !unsafe "docker inspect --format '{{ .NetworkSettings.IPAddress }}' instance1"
See the docs for details.
I have a similar issue: i need to post a JSON doc made from a jinja2 template containing some go templates variables (yes, i know :-P), such as
"NAME_TEMPLATE": %{{service_name}}.%{{stack_name}}.%{{environment_name}}
Trying to fence this part of the template between
{% raw %} ... {% endraw %}
didn't work because there is some sort of magic in ansible which will run the template and variable substition twice (i'm not sure about that, but it definitively looks like this)
You end up with "undefined variable service_name" when trying to use the template...
So i ended up using a combination of !unsafe and {% raw %} ... {% endraw %} to define a fact that's later used in the template.
- set_fact:
__rancher_init_root_domain: "{{ rancher_root_domain }}"
#!unsafe: try to trick ansible into not doing substitutions in that string, then use %raw% so the value won't substituted another time
__rancher_init_name_template: !unsafe "{%raw%}%{{service_name}}.%{{stack_name}}.%{{environment_name}}{%endraw%}"
- name: build a template for a project
set_fact:
__rancher_init_template_doc: "{{ lookup('template', 'templates/project_template.json.j2') }}"
the template contains this:
"ROOT_DOMAIN":"{{__rancher_init_root_domain}}",
"ROUTE53_ZONE_ID":"",
"NAME_TEMPLATE":"{{__rancher_init_name_template }}",
"HEALTH_CHECK":"10000",
and the output is ok:
"NAME_TEMPLATE": "%{{service_name}}.%{{stack_name}}.%{{environment_name}}",
Here's a shorter alternative to udondan's answer; surround the whole string with double brackets:
shell: "docker inspect --format {{ '{{ .NetworkSettings.IPAddress }}' }} instance1"
The solution by using raw has been already mentioned but the command in the answer before unfortunately didn't work for me.
Without ansible:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' docker_instance_name
With ansible:
- name: Get ip of db container
shell: "{% raw %}docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' docker_instance_name{% endraw %}"
register: db_ip_addr
- debug:
var: db_ip_addr.stdout
I managed to work around my issue using a small script:
#!/usr/bin/env bash
docker inspect --format '{{ .NetworkSettings.IPAddress }}' "$1"
And the following Ansible play
- copy:
src: files/get_docker_ip.sh
dest: /usr/local/bin/get_docker_ip.sh
owner: root
group: root
mode: 0770
- shell: "/usr/local/bin/get_docker_ip.sh {{ SWIFT_ACCOUNT_HOSTNAME }}"
register: swift_account_info
Nevertheless, it's very surprising that Ansible doesn't allow escaping double curly braces!
I was unable to get #Ben's answer to work (shell: !unsafe ...)
What follows here is a complete (and working!) answer to the OP's question, updated for Ansible >2.0
---
# file: play.yml
- hosts: localhost
connection: local
gather_facts: no
vars:
# regarding !unsafe, please see:
# https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html
#
- NetworkSettings_IPAddress: !unsafe "{{.NetworkSettings.IPAddress}}"
tasks:
- shell: "docker inspect --format '{{NetworkSettings_IPAddress}}' instance1"
register: out
- debug: var="{{item}}"
with_items:
- out.cmd
- out.stdout
outputs: ([WARNINGS] removed)
# ansible-playbook play.yml
PLAY [localhost] ***************************************************************
TASK [shell] *******************************************************************
changed: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item=out.cmd) => {
"item": "out.cmd",
"out.cmd": "docker inspect --format '{{.NetworkSettings.IPAddress}}' instance1"
}
ok: [localhost] => (item=out.stdout) => {
"item": "out.stdout",
"out.stdout": "172.17.0.2"
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0
# ansible --version | head -1
ansible 2.6.1
Here is a mostly clean and Ansible native workaround not depending on docker --inspect with curly braces. We assume we have just referenced one container with the Ansible docker module before:
- name: query IP of client container
shell: "docker exec {{ docker_containers[0].Id }} hostname -I"
register: _container_query
- name: get IP of query result
set_fact:
_container_ip: "{{ _container_query.stdout | regex_replace('\\s','') }}"
You now have the IP of the Docker container in the Variable _container_ip. I also published this workaround on my article The Marriage of Ansible with Docker.
[Update 2015-11-03] Removed whitespaces of the stdout of the container query.
[Update 2015-11-04] BTW, there were two pull requests in the official Ansible repository, that would made this workaround needless by recovering the facts returned by the Docker module. So you could acces the IP of a docker container via docker_containers[0].NetworkSettings.IPAddress. So please vote for those pull requests:
fixed broken facts #1457
docker module: fix regressions introduced by f38186c and 80aca4b #2093

Resources