I have a terratest code to test some Kubernetes resources, terraform resources.
I want to run the terratest code inside a docker container against existing terraform resources. The statefile exists in remote s3 backend. Please advice how to run terratest from a container.
I tried below code to create the docker image. But from inside docker container, when I run
FROM golang:1.19
RUN apt-get update
RUN apt-get install unzip
RUN wget https://releases.hashicorp.com/terraform/1.3.0/terraform_1.3.0_linux_amd64.zip
RUN unzip terraform_1.3.0_linux_amd64.zip
RUN rm terraform_1.3.0_linux_amd64.zip
RUN mv terraform /usr/local/go/bin/terraform
RUN apt-get install git bash openssl
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY *.go ./ ##Copies my go terratest code inside container
RUN go build -o /app/complete-test-docker
CMD [ "/complete-test-docker" ]
I want to run the terratest commands from container,
go test -v -run TestTagging -timeout 30m
But I am not sure how to run them from container. I dont want to run terraform init, apply as I have my resources already built and I have statefile in remote s3 backend.
Any pointers on this to do it in right way?
Related
I've created a docker image with Artifactory and Terraform to be used by pods in a K8s Cluster but it wont persist, the pod gets deleted immediately after spinning up and wasn't able to execute the job it was assigned to. Upon checking the pushed image in Artifactory converts the WORKDIR and CMD to RUN, is there anything I'm missing?
Here's the Dockerfile:
FROM alpine:3.16.2
LABEL maintainer="Platform Engineering"
# install dependencies
RUN apk add terraform
RUN apk add curl
RUN apk add tree
RUN curl -fL https://install-cli.jfrog.io | sh
RUN curl --location --output /usr/local/bin/release-cli "https://release-cli-downloads.s3.amazonaws.com/latest/release-cli-linux-amd64"
RUN chmod +x /usr/local/bin/release-cli
# check version of installed dependencies
RUN terraform -v
RUN jf -v
RUN release-cli -v
# target ci workspace under /tmp directory
WORKDIR /tmp/ci-workspace
CMD ["/bin/sh"]
Here's how the layers look like in Artifactory:
Tried rebuilding in Windows and other machine and installing one binary at a time, nothing worked.
I just started learning docker. To teach myself, I managed to containerize bandit (a python code scanner) but I'm not able to see the output of the scan before the container destroys itself. How can I copy the output file from inside the container to the host, or otherwise save it?
Right now i'm just using bandit to scan itself basically :)
Dockerfile
FROM python:3-alpine
WORKDIR /
RUN pip install bandit
RUN apk update && apk upgrade
RUN apk add git
RUN git clone https://github.com/PyCQA/bandit.git ./code-to-scan
CMD [ "python -m bandit -r ./code-to-scan -o bandit.txt" ]
You can mount a volume on you host where you can share the output of bandit.
For example, you can run your container with:
docker run -v $(pwd)/output:/tmp/output -t your_awesome_container:latest
And you in your dockerfile:
...
CMD [ "python -m bandit -r ./code-to-scan -o /tmp/bandit.txt" ]
This way the bandit.txt file will be found in the output folder.
Better place the code in your image not in the root directory.
I did some adjustments to your Dockerfile.
FROM python:3-alpine
WORKDIR /usr/myapp
RUN pip install bandit
RUN apk update && apk upgrade
RUN apk add git
RUN git clone https://github.com/PyCQA/bandit.git .
CMD [ "bandit","-r",".","-o","bandit.txt" ]`
This clones git in your WORKDIR.
Note the CMD, it is an array, so just devide all commands and args as in the Dockerfile about.
I put the the Dockerfile in my D:\test directory (Windows).
docker build -t test .
docker run -v D:/test/:/usr/myapp test
It will generate you bandit.txt in the test folder.
After the code is execute the container exits, as there are nothing else to do.
you can also put --rm to remove the container once it finishs.
docker run --rm -v D:/test/:/usr/myapp test
I have a nodejs app wrapped in a docker image. Whenever I do any change in the code, even adding few console.logs, I need to rebuild the whole image, a long process of over 10 minutes, during the CI process.
Is there a way to build one image on top of another, say adding only the delta?
Thanks
EDIT in response to #The Fool comment
I'm running the CI using aws tools, mainly CodeBuild. I can use the latest docker image I created (all are stored in aws ecr) and build based on it, would it know to take only the delta even if it conflicts with the new code?
Following is my Dockerfile:
FROM node:15.3.0
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app
RUN apt-get update
RUN apt-get install -y build-essential libcairo2-dev libpango1.0-dev libjpeg-dev libgif-dev librsvg2-dev
RUN npm install
COPY . /usr/src/app
EXPOSE 3000
CMD bash -c "npm run sequelize db:migrate&&npm run sequelize db:seed:all&&npm run prod"
Everytime I build the container I have to wait for apk add docker to finish which takes a long time.
Since everytime it downloads the same thing, can I somehow force Docker to cache apk's downloads for development purposes?
Here's my Dockerfile:
FROM golang:1.13.5-alpine
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
RUN apk add --update docker
CMD ["app"]
BTW, I am using this part volumes: - /var/run/docker.sock:/var/run/docker.sock in my docker-compose.yml to use sibling containers, if that matters.
EDIT: I've found google to copy docker.tgz in Chromium:
# add docker client -- do not install docker via apk -- it will try to install
# docker engine which takes a lot of space as well (we don't need it, we need
# only the small client to communicate with the host's docker server)
ADD build/docker/docker.tgz /
What is that docker.tgz? How can I get it?
Reorder your Dockerfile and it should work.
FROM golang:1.13.5-alpine
RUN apk add --update docker
WORKDIR /go/src/app
COPY src .
RUN go get -d -v ./...
RUN go install -v ./...
CMD ["app"]
As you are copying before installation, so whenever you change something in src the cache will invalidate for docker installtion.
Whenever you have a COPY command, if any of the files involve change, it causes every command after that to get re-run. If you move your RUN apk add ... command to the start of the file before it COPYs anything, it will get cached across runs.
A fairly generic recipe for most Dockerfiles to accommodate this pattern looks like:
FROM some-base-image
# Install OS-level dependencies
RUN apk add or apt-get install ...
WORKDIR /app
# Install language-level dependencies
COPY requirements.txt requirements.lock ./
RUN something install -r requirements.txt
# Install the rest of the application
COPY main.app ./
COPY src src/
# Set up standard run-time metadata
EXPOSE 12345
CMD ["/app/main.app"]
(Go and Java applications need the additional step of compiling the application, which often lends itself to a multi-stage build, but this same pattern can be repeated in both stages.)
You can download Docker x86_64 binaries for mac, linux, windows and unzip/untar and make it executable.
Whenever you are installing any packages in Docker container those should go at the beginning of Dockerfile, so it won’t ask you again to install same packages and COPY command part must be at the end of Dockerfile.
I want to put my code inside a docker container, I have created dockerfile and when I run, I got an error
internal/server/handlers.go:16:2: cannot find package "github.com/lib/pq" in any of:
/usr/local/go/src/github.com/lib/pq (from $GOROOT)
/go/src/github.com/lib/pq (from $GOPATH)
but when I launch my code locally without docker by typing go run main.go everything is fine
Make sure you installed all your package inside container. Because your docker container is a different machine with your current computer. You need to make sure that all dependencies package installed in your docker image. For an Dockefile example, install my package at Dockerfile as you can see:
FROM golang:latest
# Create working folder
RUN mkdir /app
COPY . /app
RUN apt -y update && apt -y install git
RUN go get github.com/go-sql-driver/mysql
RUN go get github.com/gosimple/slug
RUN go get github.com/gin-gonic/gin
RUN go get gopkg.in/russross/blackfriday.v2
RUN go get github.com/gin-gonic/contrib/sessions
WORKDIR /app
Now you run docker run -it -p 8080:8080 your_docker_image_name go run main.go should work.