Go Lambda function no request body from docker - docker

I am trying to deploy Go in a dockerized lambda function that responds to LambdaFunctionURLRequest. When I test locally I do not get the request body.
If I deploy a normal zip to lambda it works just fine.
I do get the error back and the docker does print
Here is the basic code + docker
main.go
package main
import (
"context"
"errors"
"log"
"net/http"
"time"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
)
func main() {
lambda.Start(HandleRequest)
}
func init() {
log.Println("Init Function")
}
func HandleRequest(ctx context.Context, request events.LambdaFunctionURLRequest) (ev events.LambdaFunctionURLResponse, err error) {
log.Println("Received Handle Request")
log.Println(request.Body)
if request.Body == ""{
return ev, errors.New("No request body")
}
return
}
Docker
FROM golang AS build
WORKDIR /app
COPY . .
RUN go mod download
RUN go build -o main
ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
RUN chmod 755 /usr/bin/aws-lambda-rie
COPY entry.sh /
RUN chmod 755 /entry.sh
ENTRYPOINT [ "/entry.sh" ]
Entry.sh
#!/bin/sh
if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
exec /usr/bin/aws-lambda-rie "./main"
else
exec "$#"
fi
Curl
curl --location --request POST 'http://localhost:9000/2015-03-31/functions/function/invocations' \
--header 'Content-Type: application/json' \
--data-raw '{
"Hello There": "Some Data"
}'

Related

Deploy Go Lambda within Docker container

I have a Go Lambda function. I want to host that function in a Docker image/container so that I can test it locally. In this effort, I have followed the instructions provided here. From those instructions, I have the following files:
.
Dockerfile
go.mod
go.sum
main.go
Those files contain the following:
Dockerfile (a copy of the Dockerfile in this section)
FROM alpine as build
# install build tools
RUN apk add go git
RUN go env -w GOPROXY=direct
# cache dependencies
ADD go.mod go.sum ./
RUN go mod download
# build
ADD . .
RUN go build -o /main
# copy artifacts to a clean image
FROM alpine
COPY --from=build /main /main
ENTRYPOINT [ "/main" ]
go.mod (an updated version of this go.mod)
module main
go 1.18
require (
github.com/aws/aws-lambda-go v1.32.1
github.com/aws/aws-sdk-go v1.44.60
)
require github.com/jmespath/go-jmespath v0.4.0 // indirect
go.sum (a modified version of this go.sum)
github.com/aws/aws-lambda-go v1.32.1 h1:ls0FU8Mt7ayJszb945zFkUfzxhkQTli8mpJstVcDtCY=
github.com/aws/aws-lambda-go v1.32.1/go.mod h1:jwFe2KmMsHmffA1X2R09hH6lFzJQxzI8qK17ewzbQMM=
github.com/aws/aws-sdk-go v1.44.60 h1:KTTogelVR+4dWiIPl7eyxoxaJkziChON6/Y/hVfTipk=
github.com/aws/aws-sdk-go v1.44.60/go.mod h1:y4AeaBuwd2Lk+GepC1E9v0qOiTws0MIWAX4oIKwKHZo=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.2 h1:4jaiDzPyXQvSd7D0EjG45355tLlV3VOECpq10pLC+8s=
golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.8 h1:obN1ZagJSUGI0Ek/LBmuj4SNLPfIny3KsKFopxRdj10=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
main.go (a copy of this main.go)
package main
import (
"context"
"encoding/json"
"log"
"os"
"github.com/aws/aws-lambda-go/events"
runtime "github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-lambda-go/lambdacontext"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/lambda"
)
var client = lambda.New(session.New())
func callLambda() (string, error) {
input := &lambda.GetAccountSettingsInput{}
req, resp := client.GetAccountSettingsRequest(input)
err := req.Send()
output, _ := json.Marshal(resp.AccountUsage)
return string(output), err
}
func handleRequest(ctx context.Context, event events.SQSEvent) (string, error) {
// event
eventJson, _ := json.MarshalIndent(event, "", " ")
log.Printf("EVENT: %s", eventJson)
// environment variables
log.Printf("REGION: %s", os.Getenv("AWS_REGION"))
log.Println("ALL ENV VARS:")
for _, element := range os.Environ() {
log.Println(element)
}
// request context
lc, _ := lambdacontext.FromContext(ctx)
log.Printf("REQUEST ID: %s", lc.AwsRequestID)
// global variable
log.Printf("FUNCTION NAME: %s", lambdacontext.FunctionName)
// context method
deadline, _ := ctx.Deadline()
log.Printf("DEADLINE: %s", deadline)
// AWS SDK call
usage, err := callLambda()
if err != nil {
return "ERROR", err
}
return usage, nil
}
func main() {
runtime.Start(handleRequest)
}
I can successfully run:
go mod tidy
go build
I can also successfully build and run my Docker image using:
docker build -t lambda-fn .
docker run -d -v ~/.aws-lambda-rie:/aws-lambda --entrypoint /aws-lambda/aws-lambda-rie -p 9000:8080 lambda-fn:latest /main
I can see a container based on the lambda-fn image listed, with a status of Running, in Docker desktop. However, when I send the following cURL request, nothing happens:
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
I was expecting some logs to be written based on the contents of the handleRequest function. What am I doing wrong?
If you are running in demon mode (-d) you cannot see the logs.
Remove -d and rerun the command
This is due the empty handler registration. You can set the handler name by passing the extra argument
docker run -d -v ~/.aws-lambda-rie:/aws-lambda --entrypoint /aws-lambda/aws-lambda-rie -p 9000:8080 lambda-fn:latest /main handleRequest

how to make docker keep running in frontend and not exit so that I could see the running log output

Now I want to make a docker command run in frontend so that I could see the log output. Now I am using this command to run my docker container:
docker run -p 11110:11110 -p 11111:11111 -p 11112:11112 --name canal-server dolphinjiang/canal-server:v1.1.5
this is the Dockerfile of my project:
FROM centos:7
RUN cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo ZONE=\"Asia/Shanghai\" > /etc/sysconfig/clock
RUN rm -rf /etc/yum.repos.d/*.repo
COPY CentOS6-Base-163.repo /etc/yum.repos.d/
RUN yum clean all
RUN groupadd -g 2500 canal; useradd -u 2501 -g canal -d /home/canal -m canal
RUN echo canal:De#2018er | chpasswd; echo root:dockerroot | chpasswd
RUN yum -y update && yum -y install wget vi openssl.x86_64 glibc.x86_64 tar tar.x86_64 inetutils-ping net-tools telnet which file
RUN yum clean all
COPY jdk-8u291-linux-x64.tar.gz /opt
RUN tar -zvxf /opt/jdk-8u291-linux-x64.tar.gz -C /opt && \
rm -rf /opt/jdk-8u291-linux-x64.tar.gz && \
chmod -R 755 /opt/jdk1.8.0_291 && \
chown -R root:root /opt/jdk1.8.0_291
RUN echo 'export JAVA_HOME=/opt/jdk1.8.0_291' >> /etc/profile
RUN echo 'export JRE_HOME=$JAVA_HOME/jre' >> /etc/profile
RUN echo 'export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH' >> /etc/profile
RUN echo 'export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH' >> /etc/profile
RUN source /etc/profile
RUN yum install kde-l10n-Chinese -y
RUN yum install glibc-common -y
RUN localedef -c -f UTF-8 -i zh_CN zh_CN.utf8
ENV JAVA_HOME /opt/jdk1.8.0_291
ENV PATH $PATH:$JAVA_HOME/bin
ENV LANG zh_CN.UTF-8
ENV LC_ALL zh_CN.UTF-8
ADD canal-server /home/canal/
RUN chmod 755 /home/canal/bin
WORKDIR /home/canal/bin
RUN chmod 777 /home/canal/bin/restart.sh
RUN chmod 777 /home/canal/bin/startup.sh
RUN chmod 777 /home/canal/bin/stop.sh
RUN chmod 777 /home/canal/bin/config.sh
CMD /home/canal/bin/config.sh
this is the config.sh:
cat > /home/canal/conf/canal.properties <<- EOF
# register ip
canal.register.ip = ${HOSTNAME}.canal-server-discovery-svc-stable.testcanal.svc.cluster.local
# canal admin config
canal.admin.manager = canal-admin-stable:8089
canal.admin.port = 11110
canal.admin.user = admin
canal.admin.passwd = 4ACFE3202A5FF5CF467898FC58AAB1D615029441
# admin auto register
canal.admin.register.auto = true
canal.admin.register.cluster =
EOF
sh /home/canal/bin/restart.sh
and this is the restart.sh:
#!/bin/bash
args=$#
case $(uname) in
Linux)
bin_abs_path=$(readlink -f $(dirname $0))
;;
*)
bin_abs_path=$(cd $(dirname $0) ||exit ; pwd)
;;
esac
sh "$bin_abs_path"/stop.sh $args
sh "$bin_abs_path"/startup.sh $args
and this is the start.sh:
#!/bin/bash
current_path=`pwd`
case "`uname`" in
Linux)
bin_abs_path=$(readlink -f $(dirname $0))
;;
*)
bin_abs_path=`cd $(dirname $0); pwd`
;;
esac
base=${bin_abs_path}/..
canal_conf=$base/conf/canal.properties
canal_local_conf=$base/conf/canal_local.properties
logback_configurationFile=$base/conf/logback.xml
export LANG=en_US.UTF-8
export BASE=$base
if [ -f $base/bin/canal.pid ] ; then
echo "found canal.pid , Please run stop.sh first ,then startup.sh" 2>&2
exit 1
fi
if [ ! -d $base/logs/canal ] ; then
mkdir -p $base/logs/canal
fi
## set java path
if [ -z "$JAVA" ] ; then
JAVA=$(which java)
fi
ALIBABA_JAVA="/usr/alibaba/java/bin/java"
TAOBAO_JAVA="/opt/taobao/java/bin/java"
if [ -z "$JAVA" ]; then
if [ -f $ALIBABA_JAVA ] ; then
JAVA=$ALIBABA_JAVA
elif [ -f $TAOBAO_JAVA ] ; then
JAVA=$TAOBAO_JAVA
else
echo "Cannot find a Java JDK. Please set either set JAVA or put java (>=1.5) in your PATH." 2>&2
exit 1
fi
fi
case "$#"
in
0 )
;;
1 )
var=$*
if [ "$var" = "local" ]; then
canal_conf=$canal_local_conf
else
if [ -f $var ] ; then
canal_conf=$var
else
echo "THE PARAMETER IS NOT CORRECT.PLEASE CHECK AGAIN."
exit
fi
fi;;
2 )
var=$1
if [ "$var" = "local" ]; then
canal_conf=$canal_local_conf
else
if [ -f $var ] ; then
canal_conf=$var
else
if [ "$1" = "debug" ]; then
DEBUG_PORT=$2
DEBUG_SUSPEND="n"
JAVA_DEBUG_OPT="-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=$DEBUG_PORT,server=y,suspend=$DEBUG_SUSPEND"
fi
fi
fi;;
* )
echo "THE PARAMETERS MUST BE TWO OR LESS.PLEASE CHECK AGAIN."
exit;;
esac
str=`file -L $JAVA | grep 64-bit`
if [ -n "$str" ]; then
JAVA_OPTS="-server -Xms2048m -Xmx3072m -Xmn1024m -XX:SurvivorRatio=2 -XX:PermSize=96m -XX:MaxPermSize=256m -Xss256k -XX:-UseAdaptiveSizePolicy -XX:MaxTenuringThreshold=15 -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError"
else
JAVA_OPTS="-server -Xms1024m -Xmx1024m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:MaxPermSize=128m "
fi
JAVA_OPTS=" $JAVA_OPTS -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8"
CANAL_OPTS="-DappName=otter-canal -Dlogback.configurationFile=$logback_configurationFile -Dcanal.conf=$canal_conf"
if [ -e $canal_conf -a -e $logback_configurationFile ]
then
for i in $base/lib/*;
do CLASSPATH=$i:"$CLASSPATH";
done
CLASSPATH="$base/conf:$CLASSPATH";
echo "cd to $bin_abs_path for workaround relative path"
cd $bin_abs_path
echo LOG CONFIGURATION : $logback_configurationFile
echo canal conf : $canal_conf
echo CLASSPATH :$CLASSPATH
$JAVA $JAVA_OPTS $JAVA_DEBUG_OPT $CANAL_OPTS -classpath .:$CLASSPATH com.alibaba.otter.canal.deployer.CanalLauncher 2>&1
echo $! > $base/bin/canal.pid
echo "cd to $current_path for continue"
cd $current_path
else
echo "canal conf("$canal_conf") OR log configration file($logback_configurationFile) is not exist,please create then first!"
fi
after I start the docker, it exit automaticlly, and the docker not startup, no log output. what should I do to make it run in frontend. after successs, switch to the backend. I also tried to run in deamon like this(make the container run background and not exit):
docker run -it -d -p 11110:11110 -p 11111:11111 -p 11112:11112 --name canal-server canal/canal-server:v1.1.5
the process still exit automaticlly. and docker container did not startup.
Basically, you should get the point (based on your latest comment).
Docker is based on some command, when it's done - it stops the container.
So to make it continuously running you should have command and run infinitely.
Also check this answer as well, there are more explanation
Why docker exiting with code 0
One of the easiest solution is to tail some logs.
Like,
tail -f /dev/null
Taken from here
you can use tail -f /dev/null to keep the container from stopping, try this
docker run -it -d -p 11110:11110 -p 11111:11111 -p 11112:11112 --name canal-server canal/canal-server:v1.1.5 tail -f /dev/null
see also this post

Docker throwing "{binary} not found" error

I have a very simple Go app that I'm trying to Dockerize.
package main
import (
"fmt"
"net/http"
"time"
)
func main() {
http.HandleFunc("/", date)
http.HandleFunc("/health", healthCheck)
http.ListenAndServe(":8080", nil)
}
func healthCheck(w http.ResponseWriter, r *http.Request){
fmt.Fprintf(w, "I'm alive!")
}
func date(w http.ResponseWriter, r *http.Request) {
now := time.Now()
fmt.Fprintf(w, now.Format("2006-01-02 15:04:05"))
}
My Dockerfile:
FROM golang:1.14.6
RUN mkdir /app
COPY date.go /app
WORKDIR /app
RUN go build date.go
RUN groupadd -g 999 appuser && \
useradd -r -u 999 -g appuser appuser
USER appuser
EXPOSE 8080
CMD ['/app/date']
However, when I use docker run to run the app, I get the following error:
/bin/sh: 1: [/app/date]: not found
I've commented out the CMD ['/app/date'] line and rebuilt the image, and then was able to exec into it by running
docker run -dit goapp
docker exec -ti [containerid] /bin/bash
This takes me into the /app folder where I do see the date file. And I am able to run /app/date without any issues. I'm not sure what I'm doing wrong.
The CMD is interpreted as JSON, so you need to change the single quotes to double quotes.
CMD ["/app/date"]
This is specified in the Dockerfile documentation:
The exec form is parsed as a JSON array, which means that you must use double-quotes (") around words not single-quotes (').

Jenkins: Run a curl command within groovy scrpt

I have a requirement where I need to send the status of the jenkins slave to influxdb. To do so I need to run a curl command from Jenkins Groovy script.
My script looks like this :
int value=0;
for (Node node in Jenkins.instance.nodes) {
if (!node.toComputer().online){
value=1;
}
else{
value=0;
}
curl -i -XPOST http://localhost:8086/write?db=jenkins_db&u=user&p=pass --data-binary 'mymeas,tag=$node.nodeName status=$value'
But after running the script values do not appear in influxdb.
Any Idea what might be wrong here?
PS I also tried
def response = [ 'bash', '-c', "curl", "-i", "-XPOST", "http:/localhost:8086/write?db=jenkins_db&u=user&p=pass", "--data-binary", "\'mymeas tag=$node.nodeName status=$value"\' ].execute().text
You just need to echo your curl command
echo curl -i -XPOST http://localhost:8086/write?db=jenkins_db&u=user&p=pass --data-binary 'mymeas,tag=$node.nodeName status=$value'

Docker: How to create a table for local dynamo DB?

I'm trying to create a docker container with local Amazon Dynamo DB. And it actually works. But I can not understand how to create a table for this image in Docker file?
Through the javascript I create a table like this:
var params = {
TableName: 'UserActivity',
KeySchema: [
{
AttributeName: 'user_id',
KeyType: 'HASH',
},
{
AttributeName: 'user_action',
KeyType: 'RANGE',
}
],
AttributeDefinitions: [
{
AttributeName: 'user_id',
AttributeType: 'S',
},
{
AttributeName: 'user_action',
AttributeType: 'S',
}
],
ProvisionedThroughput: {
ReadCapacityUnits: 2,
WriteCapacityUnits: 2,
}
};
dynamodb.createTable(params, function(err, data) {
if (err) ppJson(err); // an error occurred
else ppJson(data); // successful response
});
And here is my Docker file:
FROM openjdk:8-jre-alpine
ENV DYNAMODB_VERSION=latest
RUN apk add --update curl && \
rm -rf /var/cache/apk/* && \
curl -O https://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_${DYNAMODB_VERSION}.tar.gz && \
tar zxvf dynamodb_local_${DYNAMODB_VERSION}.tar.gz && \
rm dynamodb_local_${DYNAMODB_VERSION}.tar.gz
EXPOSE 8000
ENTRYPOINT ["java", "-Djava.library.path=.", "-jar", "DynamoDBLocal.jar", "--sharedDb", "-inMemory", "-port", "8000"]
You need to point your DynamoDB client to the local DynamoDB endpoint.
Do something like:
dynamodb = new AWS.DynamoDB({
endpoint: new AWS.Endpoint('http://localhost:8000')
});
dynamodb.createTable(...);
UPDATE
Alternatively you can use AWS CLI to create tables in your local DynamoDB without using JavaSciprt code. You need to use the "--endpoint-url" to point CLI to your local instance like this:
aws dynamodb list-tables --endpoint-url http://localhost:8000
You need to use create-table command to create a table.
The main problem was in base image which I used for local dynamodb. I mean openjdk:8-jre-alpine
Looks like the alpine distribution does not have something important for functioning of local dynamodb.
So here is a working Dockerfile for local dynamodb:
FROM openjdk:8-jre
ENV DYNAMODB_VERSION=latest
COPY .aws/ root/.aws/
RUN curl -O https://s3-us-west-2.amazonaws.com/dynamodb-local/dynamodb_local_${DYNAMODB_VERSION}.tar.gz && \
tar zxvf dynamodb_local_${DYNAMODB_VERSION}.tar.gz && \
rm dynamodb_local_${DYNAMODB_VERSION}.tar.gz
EXPOSE 8000
ENTRYPOINT ["java", "-Djava.library.path=.", "-jar", "DynamoDBLocal.jar", "--sharedDb", "-inMemory"]
I'd be happy to read what is the main problem of alpine in this particular case.

Resources