I have a Go Lambda function. I want to host that function in a Docker image/container so that I can test it locally. In this effort, I have followed the instructions provided here. From those instructions, I have the following files:
.
Dockerfile
go.mod
go.sum
main.go
Those files contain the following:
Dockerfile (a copy of the Dockerfile in this section)
FROM alpine as build
# install build tools
RUN apk add go git
RUN go env -w GOPROXY=direct
# cache dependencies
ADD go.mod go.sum ./
RUN go mod download
# build
ADD . .
RUN go build -o /main
# copy artifacts to a clean image
FROM alpine
COPY --from=build /main /main
ENTRYPOINT [ "/main" ]
go.mod (an updated version of this go.mod)
module main
go 1.18
require (
github.com/aws/aws-lambda-go v1.32.1
github.com/aws/aws-sdk-go v1.44.60
)
require github.com/jmespath/go-jmespath v0.4.0 // indirect
go.sum (a modified version of this go.sum)
github.com/aws/aws-lambda-go v1.32.1 h1:ls0FU8Mt7ayJszb945zFkUfzxhkQTli8mpJstVcDtCY=
github.com/aws/aws-lambda-go v1.32.1/go.mod h1:jwFe2KmMsHmffA1X2R09hH6lFzJQxzI8qK17ewzbQMM=
github.com/aws/aws-sdk-go v1.44.60 h1:KTTogelVR+4dWiIPl7eyxoxaJkziChON6/Y/hVfTipk=
github.com/aws/aws-sdk-go v1.44.60/go.mod h1:y4AeaBuwd2Lk+GepC1E9v0qOiTws0MIWAX4oIKwKHZo=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.2 h1:4jaiDzPyXQvSd7D0EjG45355tLlV3VOECpq10pLC+8s=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
main.go (a copy of this main.go)
package main
import (
"context"
"encoding/json"
"log"
"os"
"github.com/aws/aws-lambda-go/events"
runtime "github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-lambda-go/lambdacontext"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/lambda"
)
var client = lambda.New(session.New())
func callLambda() (string, error) {
input := &lambda.GetAccountSettingsInput{}
req, resp := client.GetAccountSettingsRequest(input)
err := req.Send()
output, _ := json.Marshal(resp.AccountUsage)
return string(output), err
}
func handleRequest(ctx context.Context, event events.SQSEvent) (string, error) {
// event
eventJson, _ := json.MarshalIndent(event, "", " ")
log.Printf("EVENT: %s", eventJson)
// environment variables
log.Printf("REGION: %s", os.Getenv("AWS_REGION"))
log.Println("ALL ENV VARS:")
for _, element := range os.Environ() {
log.Println(element)
}
// request context
lc, _ := lambdacontext.FromContext(ctx)
log.Printf("REQUEST ID: %s", lc.AwsRequestID)
// global variable
log.Printf("FUNCTION NAME: %s", lambdacontext.FunctionName)
// context method
deadline, _ := ctx.Deadline()
log.Printf("DEADLINE: %s", deadline)
// AWS SDK call
usage, err := callLambda()
if err != nil {
return "ERROR", err
}
return usage, nil
}
func main() {
runtime.Start(handleRequest)
}
I can successfully run:
go mod tidy
go build
I can also successfully build and run my Docker image using:
docker build -t lambda-fn .
docker run -d -v ~/.aws-lambda-rie:/aws-lambda --entrypoint /aws-lambda/aws-lambda-rie -p 9000:8080 lambda-fn:latest /main
I can see a container based on the lambda-fn image listed, with a status of Running, in Docker desktop. However, when I send the following cURL request, nothing happens:
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
I was expecting some logs to be written based on the contents of the handleRequest function. What am I doing wrong?
If you are running in demon mode (-d) you cannot see the logs.
Remove -d and rerun the command
This is due the empty handler registration. You can set the handler name by passing the extra argument
docker run -d -v ~/.aws-lambda-rie:/aws-lambda --entrypoint /aws-lambda/aws-lambda-rie -p 9000:8080 lambda-fn:latest /main handleRequest
Inside my Dockerfile:
ENV PROJECTNAME mytestwebsite
CMD ["django-admin", "startproject", "$PROJECTNAME"]
Error:
CommandError: '$PROJECTNAME' is not a valid project name
What is the quickest workaround here? Does Docker have any plan to "fix" or introduce this functionality in later versions of Docker?
NOTE: If I remove the CMD line from the Docker file and then run the Docker container, I am able to manually run Django-admin startproject $PROJECTNAME from inside the container and it will create the project...
When you use an execution list, as in...
CMD ["django-admin", "startproject", "$PROJECTNAME"]
...then Docker will execute the given command directly, without involving a shell. Since there is no shell involved, that means:
No variable expansion
No wildcard expansion
No i/o redirection with >, <, |, etc
No multiple commands via command1; command2
And so forth.
If you want your CMD to expand variables, you need to arrange for a shell. You can do that like this:
CMD ["sh", "-c", "django-admin startproject $PROJECTNAME"]
Or you can use a simple string instead of an execution list, which gets you a result largely identical to the previous example:
CMD django-admin startproject $PROJECTNAME
If you want to use the value at runtime, set the ENV value in the Dockerfile. If you want to use it at build-time, then you should use ARG.
Example :
ARG value
ENV envValue=$value
CMD ["sh", "-c", "java -jar ${envValue}.jar"]
Pass the value in the build command:
docker build -t tagName --build-arg value="jarName"
You also can use exec
This is the only known way to handle signals and use env vars simultaneously.
It can be helpful while trying to implement something like graceful shutdown according to Docker github
Example:
ENV PROJECTNAME mytestwebsite
CMD exec django-admin startproject $PROJECTNAME
Lets say you want to start a java process inside a container:
Example Dockerfile excerpt:
ENV JAVA_OPTS -XX +UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm
...
ENTRYPOINT ["/sbin/tini", "--", "entrypoint.sh"]
CMD ["java", "${JAVA_OPTS}", "-myargument=true"]
Example entrypoint.sh excerpt:
#!/bin/sh
...
echo "*** Startup $0 suceeded now starting service using eval to expand CMD variables ***"
exec su-exec mytechuser $(eval echo "$#")
For the Java developers, following my solution below gonna work:
if you tried to run your container with a Dockerfile like below
ENTRYPOINT ["/docker-entrypoint.sh"]
# does not matter your parameter $JAVA_OPTS wrapped as ${JAVA_OPTS}
CMD ["java", "$JAVA_OPTS", "-javaagent:/opt/newrelic/newrelic.jar", "-server", "-jar", "app.jar"]
with an ENTRYPOINT shell script below:
#!/bin/bash
set -e
source /work-dir/env.sh
exec "$#"
it will build the image correctly but print the error below during the run of container:
Error: Could not find or load main class $JAVA_OPTS
Caused by: java.lang.ClassNotFoundException: $JAVA_OPTS
instead, Java can read the command line parameters either through the command line or by _JAVA_OPTIONS environment variable. so, it means we can pass the desired command line parameters through _JAVA_OPTIONS without changing anything on Dockerfile as well as to allow it to be able to start as parent process of container for the valid docker signalization via exec "$#".
The below one is my final version of the Dockerfile and docker-entrypoint.sh files:
...
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["java", "-server", "-jar", "app.jar"]
#!/bin/bash
set -e
source /work-dir/env.sh
export _JAVA_OPTIONS="-XX:+PrintFlagsFinal"
exec "$#"
and after you build your docker image and tried to run it, you will see the logs below that means it worked well:
Picked up _JAVA_OPTIONS: -XX:+PrintFlagsFinal
[Global flags]
int ActiveProcessorCount = -1 {product} {default}
Inspired on above, I did this:
#snapshot by default. 1 is release.
ENV isTagAndRelease=0
CMD echo is_tag: ${isTagAndRelease} && \
if [ ${isTagAndRelease} -eq 1 ]; then echo "release build"; mvn -B release:clean release:prepare release:perform; fi && \
if [ ${isTagAndRelease} -ne 1 ]; then echo "snapshot build"; mvn clean install; fi && \
.....
I'm trying to run a tcpdump command in docker container but I always get this error: "tcpdump: invalid number of output files ="
I'm running docker in Ubuntu.
This is my container run:
sudo docker run --name pcap_log -it --network=host -e "log_int=any" -e "log_rot=30" -e "log_fil=120" --mount source=/home/docker/container_data/pcap_data, target=/pcap_captures, type=bind pcap_log:12345
This is my image:
FROM debian:10.3
RUN apt-get update \
&& apt-get install -y \
tcpdump
RUN mv /usr/sbin/tcpdump /usr/bin/tcpdump
RUN mkdir /pcap_captures
WORKDIR /pcap_captures
ARG log_int
ARG log_rot
ARG log_fil
ENV log_int = $log_int
ENV log_rot = $log_rot
ENV log_fil = $log_fil
CMD tcpdump -i $log_int -G $log_rot -w '%Y-%m-%d_%H:%M:%S.pcap' -W $log_fil -U
"tcpdump: invalid number of output files ="
As you can see, there is an = sign left somewhere that is not interpreted correctly.
The problem is in your ENV declaration with an extra space before and after the = sign. For example
ENV log_int = $log_int
is instructing docker that the environment var log_int should be set to the value = $log_ing. Either you completely drop the equal sign
ENV log_int $log_int
or you remove the spaces arround it
ENV log_int=$log_int
Both are totally equivalent.
I actually suggest your keep the second form, declare all your envs in a single action and secure your vars with quotes:
ENV log_int="$log_int" log_rot="$log_rot" log_fil="$log_fil"
I am running the following Docker container from these Dockerfiles:
FROM debian:wheezy
MAINTAINER authors "authors#gm.com"
RUN groupadd -r -g 2200 example && \
useradd -rM -g example -u 2200 example
ENV APPROOT="/app" \
APP="mailer.sh" \
VERSION="0.6"
LABEL base.name="Mailer Archetype" \
base.version="${VERSION}"
WORKDIR $APPROOT
ADD . $APPROOT
ENTRYPOINT ["/app/mailer.sh"]
EXPOSE 33333
and
FROM dockerinaction/mailer-base:0.6
COPY ["./log-impl", "${APPROOT}"]
RUN chmod a+x ${APPROOT}/${APP} && \
chown example:example /var/log
USER example:example
VOLUME ["/var/log"]
CMD ["/var/log/mailer.log"]
where the mailer.sh is:
#!/bin/sh
printf "Logging Mailer has started.\n"
while true
do
MESSAGE=$(nc -l -p 33333)
printf "[Message]: %s\n" "$MESSAGE" > $1
sleep 1
done
All starts. But I want to test it. So i tried:
from the host to run "nc 33333" -> nothing happens!
to attach to the container: "docker exec -it /bin/bash" but then it does not recognize basic commands like "ps", "vi", "nc"...
So I am afraid my script is just not running.
Why is that?
thanks
The error I'm getting for my Go Module project
/bin/sh: microservice: not found
Dockerfile
FROM golang:1.7.4-alpine
MAINTAINER John Doe
ENV SOURCES /go/src/github.com/john/app/
COPY . ${SOURCES}
RUN cd ${SOURCES} && cgo_enabled=0 go install
ENV PORT 8080
EXPOSE 8080
ENTRYPOINT microservice
microservice.go
package main
import (
"fmt"
"net/http"
"os"
)
func main() {
http.HandleFunc("/", index)
http.ListenAndServe(port(), nil)
}
func port() string {
port := os.Getenv("PORT")
fmt.Println(port)
if len(port) == 0 {
port = "8080"
}
return ":" + port
}
func index(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, "Hello World.")
}
It's a Go module project. I've created an image using following command.
docker build -t app:1.0.3 .
and running it via
docker run -it -p 8080:8080 app:1.0.3
Created executable file is at /go/bin/app
and current working directory is /go.
So, change the last line of your Dockerfile to this
ENTRYPOINT ./bin/app