Grep for mismatched values in a line - grep

I have a Kubernetes cluster with lots of pods. Each pod has an arbitrary number of containers. From the command line I can list all the pods and get an output like:
pod1 2/2
pod2 1/3
pod3 1/2
pod4 4/5
pod5 5/5
pod5 0/1
...etc...
I want a grep that would show me the pods that have containers that have not started yet. i.e. List all podN x/y where x!=y. The closest I have got is:
kubectl get pods | grep -e "1/[^1]" -e "2/[^2]" -e "3/[^3]" -e "4/[^4]" -e "5/[^5]"
This works but I'm hoping there is a way in grep (or other Linux CLI tools) I could do this without the endless -e terms e.g. by assigning the match to a variable
kubectl get pods | grep "${var=[0-9]+}/[^${var}]"

Suppresses all lines with identical numbers with grep:
kubectl get pods | grep -vE '([0-9]+)/\1'
or
kubectl get pods | awk -F ' +|/' '$2!=$3'
or
kubectl get pods | sed -E '/([0-9]+)\/\1/d'
Output:
pod2 1/3
pod3 1/2
pod4 4/5
pod5 0/1

Related

Docker ps -a sort by date

docker ps sorts by time, but the most recent docker instance is at the very top. This means if you started very many instances you have to scroll all the way to the top to see them. How do we output "docker ps -a" in reverse order, so that the most recent instance is printed at the bottom?
You can pipe the output to tac[1] like:
docker ps -a | tac
[1] From man tac: tac - concatenate and print files in reverse
Latest created container:
docker ps -a -l
Latest 5 created containers:
docker ps -a -n 5
As far as I know ordering is not possible but maybe you don't really need it...
It's enough to get what you want.
$ docker ps -a --format "table {{.ID}}\t{{.Names}}\t{{.CreatedAt}}" | (read -r; printf "%s\n" "$REPLY"; sort -k 3 -r )
See also
How to sort or order results docker ps --format?

Docker get real-time logs of one of the tasks of a service

I can't ssh into docker manager and get service logs. I get "docker service logs" requires exactly 1 argument. error, when I ssh into my manager with
docker-machine ssh manager1
and run
docker logs --follow $(docker ps | grep redis | head -n1 | cut -d " " -f1)
everything works, but when I run the following command I get "docker service logs" requires exactly 1 argument.
docker-machine ssh manager1 "docker service logs $(docker ps | grep redis | head -n1 | cut -d ' ' -f1) --follow"
how can I ssh into manager1 and pass the command to run?
docker ps has some filtering and output options that mean awk/grep etc are rarely needed.
docker-machine ssh manager1 sh -uexc 'echo;
cid=$(docker ps -q -f ancestor=redis -l);
docker service logs -f $cid || echo cid[$cid]'
docker ps options.
--filter , -f Filter output based on conditions provided
--format Pretty-print containers using a Go template
--last , -n Show n last created containers (includes all states)
--latest , -l Show the latest created container (includes all states)

docker find container by pid of inside process

I have docker containers. Inside them launched a process.
From the host machine the command top outputs pid of all processes launched in within containers.
How can I find a container in which the process with this PID is running?
Thank you.
Thank you #Alex Past and #Stanislav for the help. But I did not get full answers for me. I combined them.
In summary I has got next.
First
pstree -sg <PID>
where PID is the process's PID from the command top
In output I am getting parent PID for the systemd parent process. This PID is docker container's PID.
After I execute
docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}' | grep "^%PID%"
where %PID% is this parent PID.
In result I have docker's CONTAINER ID.
That's what I wanted
I suppose you need something like this:
docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}' | grep "%PID%"
You can find all parents for this process:
pstree -sg <PID>
This chain will be contains the container
You should be able to use exec against each running container checking if the pid exists. Of course the same process id could exists in more than one container. Here is a small bash script that search for a running process based on the supplied pid in each container:
#!/bin/bash
for container in $(docker ps -q); do
status=`docker exec $container ls /proc/$1 2>/dev/null`
if [ ! -z "$status" ]; then
name=`docker ps --filter ID=$container --format "{{.Names}}"`
echo "PID: $1 found in $container ($name)"
break;
fi
done;
For example:
./find-process.sh 1
I kinda combined all of these and wrote this two liner. Hopefully useful to someone.
#!/bin/bash
SCAN_PID=`pstree -sg $1 | head -n 1 | grep -Po 'shim\([0-9]+\)---[a-z]+\(\K[^)]*'`
docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}' | grep "${SCAN_PID}"
First line finds the container entry script and feeds it to the docker inspect.
You can cycle through the parent processes of the target process using ps -o ppid= and at each step check if the PID of the parent matches one of the containers.
#!/bin/bash
targetpid=$1
parentpid=0
while [ $parentpid != 1 ]; do
parentpid=$(ps -o ppid= $targetpid)
docker ps -q | xargs docker inspect --format '{{.State.Pid}}, {{.Name}}' | grep "^$parentpid"
targetpid="$parentpid"
done

Get Docker container id from container name

What is the command to get the Docker container id from the container name?
In Linux:
sudo docker ps -aqf "name=containername"
Or in OS X, Windows:
docker ps -aqf "name=containername"
where containername is your container name.
To avoid getting false positives, as #llia Sidorenko notes, you can use regex anchors like so:
docker ps -aqf "name=^containername$"
explanation:
-q for quiet. output only the ID
-a for all. works even if your container is not running
-f for filter.
^ container name must start with this string
$ container name must end with this string
You can try this:
docker inspect --format="{{.Id}}" container_name
This approach is OS independent.
Get container Ids of running containers ::
$docker ps -qf "name=IMAGE_NAME"
-f: Filter output based on conditions provided
-q: Only display numeric container IDs
Get container Ids of all containers ::
$docker ps -aqf "name=IMAGE_NAME"
-a: all containers
You could use the following command to print the container id:
docker container ls | grep 'container-name' | awk '{print $1}'
As a bonus point, if you want to login to the container with a container name:
docker exec -it $(docker container ls | grep 'container-name' | awk '{print $1}') /bin/bash
The following command:
docker ps --format 'CONTAINER ID : {{.ID}} | Name: {{.Names}} | Image: {{.Image}} | Ports: {{.Ports}}'
Gives this output:
CONTAINER ID : d8453812a556 | Name: peer0.ORG2.ac.ae | Image: hyperledger/fabric-peer:1.4 | Ports: 0.0.0.0:27051->7051/tcp, 0.0.0.0:27053->7053/tcp
CONTAINER ID : d11bdaf8e7a0 | Name: peer0.ORG1.ac.ae | Image: hyperledger/fabric-peer:1.4 | Ports: 0.0.0.0:17051->7051/tcp, 0.0.0.0:17053->7053/tcp
CONTAINER ID : b521f48a3cf4 | Name: couchdb1 | Image: hyperledger/fabric-couchdb:0.4.15 | Ports: 4369/tcp, 9100/tcp, 0.0.0.0:5985->5984/tcp
CONTAINER ID : 14436927aff7 | Name: ca.ORG1.ac.ae | Image: hyperledger/fabric-ca:1.4 | Ports: 0.0.0.0:7054->7054/tcp
CONTAINER ID : 9958e9f860cb | Name: couchdb | Image: hyperledger/fabric-couchdb:0.4.15 | Ports: 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp
CONTAINER ID : 107466b8b1cd | Name: ca.ORG2.ac.ae | Image: hyperledger/fabric-ca:1.4 | Ports: 0.0.0.0:7055->7054/tcp
CONTAINER ID : 882aa0101af2 | Name: orderer1.o1.ac.ae | Image: hyperledger/fabric-orderer:1.4 | Ports: 0.0.0.0:7050->7050/tcp
If you want to get complete ContainerId based on Container name then use following command
docker ps --no-trunc -aqf name=containername
In my case I was running Tensorflow Docker container in Ubuntu 20.04 :Run your docker container in One terminal , I ran it with
docker run -it od
And then started another terminal and ran below docker ps with sudo:
sudo docker ps
I successfully got container id:
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
e4ca1ad20b84 od "/bin/bash" 18 minutes ago
Up 18 minutes unruffled_stonebraker
Thanks for the answer of https://stackoverflow.com/a/65513726/889126, it gave me an idea to make a complete bash script as it is
export api_image_id=$(docker inspect --format="{{.Id}}" <image-name> | sed '/^[[:space:]]*$/d')
sudo docker exec -i -t ${api_image_id} /bin/bash
I need a specific container and make a script to extract some info from it in a quick sight.
Hope this would help others.
I tried sudo docker container stats, and it will give out Container ID along with details of memory usage and Name, etc. If you want to stop viewing the process, do Ctrl+C. I hope you find it useful.
I also need the container name or Id which a script requires to attach to the container.
took some tweaking but this works perfectly well for me...
export svr=$(docker ps --format "table {{.ID}}"| sed 's/CONTAINER ID//g' | sed '/^[[:space:]]*$/d')
docker exec -it $svr bash
The sed command is needed to get rid of the fact that the words CONTAINER ID gets printed too ... but I just need the actual id stored in a var.
To have container id and image Id -
$ docker container ls -a | awk 'NR>1 {print $1, $2}'
Docker image inspect ImageName\ImageId --format={{'.ConatinerConfig.Hostname'}}
The simplest way I can think of is to parse the output of docker ps
Let's run the latest ubuntu image interactively and connect to it
docker run -it ubuntu /bin/bash
If you run docker ps in another terminal you can see something like
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8fddbcbb101c ubuntu:latest "/bin/bash" 10 minutes ago Up 10 minutes gloomy_pasteur
Unfortunately, parsing this format isn't easy since they uses spaces to manually align stuff
$ sudo docker ps | sed -e 's/ /#/g'
CONTAINER#ID########IMAGE###############COMMAND#############CREATED#############STATUS##############PORTS###############NAMES
8fddbcbb101c########ubuntu:latest#######"/bin/bash"#########13#minutes#ago######Up#13#minutes###########################gloomy_pasteur######
Here is a script that converts the output to JSON.
https://gist.github.com/mminer/a08566f13ef687c17b39
Actually, the output is a bit more convenient to work with than that. Every field is 20 characters wide.
[['CONTAINER ID',0],['IMAGE',20],['COMMAND',40],['CREATED',60],['STATUS',80],['PORTS',100],['NAMES',120]]

Use last container name as default in docker client

docker client for docker ps has very useful flag -l which shows container information which was run recently. However all other docker commands requires providing either CONTAINER ID or NAME.
Is there any nice trick which would allow to call:
docker logs -f -l
instead of:
docker logs -f random_name
You can you docker logs -f `docker ps -ql`
For the last container
docker ps -n 1
or variants such as
docker ps -qan 1
can be handy
After a while playing with docker tutorial, I created small set of aliases:
alias docker_last="docker ps -l | tail -n +2 | awk '{ print \$(NF) }' | xargs docker $1"
alias docker_all="docker ps -a | tail -n +2 | awk '{ print \$(NF) }' | xargs docker $1"
alias docker_up="docker ps | tail -n +2 | awk '{ print \$(NF) }' | xargs docker $1"
alias docker_down="docker ps -a | tail -n +2 | grep -v Up | awk '{ print \$(NF) }' | xargs docker $1"
Which allow to call command on last, all, up and down containers:
docker_last logs # Display logs from last created container
docker_down rm # Remove all stopped containers
docker_up stop # Stop all running containers

Resources