I am having trouble figuring out how we can use docker build behind a SOCKS5 proxy.
Example Dockerfile:
FROM clearlinux:base
RUN swupd bundle-add c-basic
Doing docker build --build-arg="ALL_PROXY='socks5://proxy.company.com'" -t test-socks . does not work i.e. the RUN command times out.
However, if I define ENV ALL_PROXY='socks5://proxy.company.com' within the Dockerfile, build works fine.
I've noticed that it is the case only with ALL_PROXY environment variable.
Related
I am facing the issue
(gcloud.run.deploy) Cloud Run error: Container failed to start. Failed
to start and then listen on the port defined by the PORT environment
variable. Logs for this revision might contain more information.
There are a few post with this error but I couldn't find my particular case.
I am running a background task, nothing to expose, it connects to firebase process some data and store it back. I wanted this process to run on a container on Cloud Run so I made it a container, which runs perfectly locally, but when uploading it to CR it fails with the above error.
I tried to expose 8080 on dockerfile and a few more things but if you try to connect to they container it has no server running to connect to. It is a batch task.
Can anyone tell me if it is possible at all to upload this type of tasks to Cloud Run, I do not know how to solve the issue. I wouldnt believe google requires a server running on the container to allow it, I saw some posts with dev pulling an nginx on the image so they can expose the port but this would be totally unnecessary in my case.
Thanks for your advice
UPDATE
Cloud Logging: The error simply say there was a fail to start the container, which is funny because the container starts and also shows some logs like if it were working but then it stops.
Build on MAC yes.
DockerFile is pretty simple.
FROM openjdk:11
ENV NOTIFIER_HOME /opt/app/
ENV NOTIFIER_LOGS /opt/notifications/logs/
RUN mkdir -p $NOTIFIER_HOME RUN mkdir -p $NOTIFIER_LOGS
RUN apt update
#RUN apt install curl
COPY docker/* $NOTIFIER_HOME
EXPOSE 8080
ENV TMP_OPTS -Djava.io.tmpdir=/tmp ENV LOG4j_OPTS
-Dlog4j.configurationFile=$NOTIFIER_HOME/logback.xml ENV NOTIFIER_OPTS $TMP_OPTS $LOG4j_OPTS
ENV JAVA_GC_OPTS -Xms1g -Xmx1g
WORKDIR $NOTIFIER_HOME ENTRYPOINT ["sh", "-c", "/opt/app/entrypoint.sh"]
You can't run background jobs on Cloud Run. Wrap it in a webserver as proposed by MBHA if the process take less than 1h.
Else you can you GKE Autopilot to run your container for a while. you pay only when your container run. And the first cluster is free. You can have a try on it!
As hack you can run your container in Cloud Build also, or in Vertex AI custom container training.
I've run in to a similar issue with building custom image on MAC + deploying in to Cloud Run. In my case, it turned out to be the docker platform causing the problem. The way I isolated this was by building the same image in Cloud Shell and that would work perfectly fine in Cloud Run.
Now, if you need to build it locally on MAC go ahead and test it by changing the Docker platform:
export DOCKER_DEFAULT_PLATFORM=linux/amd64
docker build -t mytag:myver .
Once the image has been built, you can inspect the architecture:
docker image inspect mytag:myver | grep -i Architecture
Then deploy it to Cloud Run.
The explanation is in your question:
I am running a background task, nothing to expose
A cloud run application, so your container, must be listening for incoming HTTP requests as stated in the Container runtime contract. That's why in all cloud run examples, java in your case, spring boot is used with #RestController. Other explanation can be found in this answer.
Update:
So the solution is either to
add a webserver to your code and wrap it with spring boot and controller logic
use Cloud Function rather than Cloud Run and get rid of the Dockerfile and in the same time have simpler code and less configuration
What I'd like to do is this:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-alpine
# more stuff
But, this needs to run on an isolated clean build machine which does not have internet access, so I need to route it through a local mirror server (e.g. Artifactory or Nexus or another similar thing)
If the docker image were hosted on docker hub (e.g. FROM ubuntu) then the --registry-mirrors docker configuration option would solve it for us.
BUT because microsoft have decided not to use docker hub, the registry mirror thing isn't working either.
After much effort, I've set up a a custom domain mirror for mcr.microsoft.com and so now I can do this:
FROM microsoft-docker-mirror.local.domain/dotnet/core/aspnet:3.0-alpine
# more stuff
That works. But we have remote workers who may not be on the local office LAN and can't see my local mirror server. What I now want to do is vary it depending on environment. E.g.
ENV MICROSOFT_DOCKER_REPO mcr.microsoft.com
FROM ${MICROSOFT_DOCKER_REPO}/dotnet/core/aspnet:3.0-alpine
My isolated build machine would set the MICROSOFT_DOCKER_REPO environment variable, and everyone else's machines would use the default mcr.microsoft.com
Anyway. Adding the ENV line to the dockerfile results in docker throwing this error:
Error response from daemon: No build stage in current context
There seem to be lots of references to the fact that the FROM line must be the first line in the file, before even any comments...
How can I reference an environment variable in my FROM statement? Or alternatively, how can I make registry mirrors work for things that aren't docker hub?
Thanks!
"ARG is the only instruction that may precede FROM in the Dockerfile" - Dockerfile reference.
ARG MICROSOFT_DOCKER_REPO=mcr.microsoft.com
FROM ${MICROSOFT_DOCKER_REPO}/dotnet/core/aspnet:3.0-alpine
Build with a non-default MICROSOFT_DOCKER_REPO using the --build-arg: docker build --rm --build-arg MICROSOFT_DOCKER_REPO=repo.example.com -t so:58196638 .
Note: you can pass the --build-arg from the hosts environment i.e. MICROSOFT_DOCKER_REPO=repo.example.com docker build --rm --build-arg MICROSOFT_DOCKER_REPO=${MICROSOFT_DOCKER_REPO} -t so:58196638 .
I would like to launch two services using docker-compose using the same image passing to the same image two different parameters, with eth and btc values. The jar expects an argument with one of those two values.
The services are demo-quartz-btc and demo-quartz-eth.
When I launch docker-compose, I can see in the log the following messages:
demo-quartz-btc_1 | Error: Unable to access jarfile /opt/app/demo-quartz.jar $CRYPT_TYPE
demo-quartz-eth_1 | Error: Unable to access jarfile /opt/app/demo-quartz.jar $CRYPT_TYPE
This is the link that shows the docker-compose.yml.
This is the link that shows the Dockerfile.
I followed the solution of this link, , but it doesn't work for me. Also I followed the official help, but it does not work.
Docker engine version is 18.09.2, compose version is 1.23.2.
Can someone help me? What do am I doing wrong?
EDIT
I can see in logs that no argument is being applied using docker-compose up. Is there any recommended way to run two services that need another services to run, like zookeeper, kafka, Eureka or others?
EDIT 1
Ok, now I know that I can run a container with the service using a entry-point.sh script file passing the argument to the container. Now, ¿how can I do the same using docker-compose up command?. I need all services/containers running in the same process.
EDIT 2
Do i have to put in script file the necessary docker run -it commands to have everything up and running?
// 31 July 2019 new Dockerfile. I can run this container in isolated mode,
//without kafka,zookeeper and Eureka server.
// I can pass arguments to Dockerfile running the next commands:
// mvn clean package -DskipTests
// docker build -t aironman/demo-quartz .
// docker run -it aironman/demo-quartz:latest eth
// where eth is the argument.
~/D/demo-quartz> cat Dockerfile
FROM adoptopenjdk/openjdk12:latest
ARG JAR_FILE
ARG CRYPT_TYPE
RUN mkdir /opt/app
COPY target/demo-quartz-0.0.2-SNAPSHOT.jar /opt/app/demo-quartz.jar
COPY entry-point.sh /
RUN chmod +x entry-point.sh
ENTRYPOINT ["/entry-point.sh"]
The solution is to have two Dockerfile files, each with an entry-point.sh file, each with the parameters you need. I realize that I have to adapt to the way Docker/Kubernetes wants you to work and not the other way around.
Working behind a corporate proxy - I need to build my docker images like
docker build --build-arg http_proxy=http://my.proxy:80 .
and that's fine.
I have a script that I've checked out that does a bunch of docker builds - and that's fail because it's not reaching the proxy.
Is there a way to set my local environment to always use my proxy settings when doing docker build?
I did look at creating an alias - but that seems a bit gnarly giving there's a space between the commands? Is there a simple global config I can modify?
First of all make sure to configure the http_proxy setting for the docker deamon as described in HTTP/HTTPS proxy.
This configuration should be enough for docker to pick it up and use it when building the image. However, If the internal commands that the dockerfile is running are creating some custom connections, this configuration may not be picked up properly.
The proxy settings can be picked up from docker info:
$ docker info | grep Proxy
Http Proxy: http://localhost:3128
Https Proxy: http://localhost:3128
You can use the values provided picked up by docker info.
However, what I recommend is to install a tool to transparently route all the traffic to the http proxy. That way you can forget about the proxy and all tools on your machine should work seemlessly.
If you are on linux, there is redsocks. The is also a docker image for it if you don't want to install it directly on the machine. For other platforms you can use proxycap.
I want to run apt from within a docker container, within a vagrant machine (running on virtualbox), but this fails because I'm behind a proxy.
I use vagrant-proxyconf to allow the vagrant machine itself to connect to the internet, which works fine:
if Vagrant.has_plugin?("vagrant-proxyconf")
config.proxy.http = ...
config.proxy.https = ...
config.proxy.no_proxy = "localhost,127.0.0.1,.example.com"
end
However, these settings aren't carried through to docker containers started within the vagrant machine. When I start a debian-based docker container with
docker run -it debian /bin/bash
and within the bash I run
apt-get update
then apt can't establish a connection. I can fix this problem by adding the following to my Dockerfile
ENV http_proxy <myproxy>
but adjusting all Dockerfile's would be cumbersome, and I'd prefer not to hardcode my proxy into the Dockerfile's themselves, as those are also used in a different setup.
I've also tried telling docker what proxy to use using: https://docs.docker.com/engine/admin/systemd/
However, this appears not to have any effect on the proxy that apt uses within the docker container.
Is there a way to pass the http_proxy environment variable to all docker containers started within my machine by default? Alternatively, would it be possible to configure vagrant / virtualbox to "emulate" a "proxyless" internet connection so that I don't have to reach the proxy settings down through all the virtualization layers?
You can add the vars passing them as arguments on docker build command. In that way it will work and the proxy ip won't be on Dockerfile. In this way:
docker build -t --build-arg http_proxy="http://yourIp" yourImage
Then on Dockerfile you must set the var as argument:
ARG http_proxy
Automatically the var is able to be used in this way:
RUN echo ${http_proxy}
But in your case you don't need to use it, only setting the proxy var is enough to be used during building.
This technique could be very useful too in order to avoid write passwords on Dockerfiles.
Hope it helps