I am using Artifactory for storing docker images. Artifactory setup is using v1 repository to store images. When working from one of he linux machine i am able to pull and push the images from the Artifactory. But when working on my Windows laptop if I am trying to pull the image from the Artifactory it gives me below error
akash#AKASH-WS01 MINGW64 ~
$ docker pull mydocker.abc.com:5903/ubuntu
Using default tag: latest
Error response from daemon: unknown: Unsupported docker v2 repository request for 'demo-docker'
I am using .dockercfg file for authentication and have information stored to it. "demo-docker" is a user
Why docker pull command is using v2 repository when mydocker.abc.com:5903/ubuntu is on v1.Is there any way to make docker pull to use v1
I had the same problem, I adjusted my nginx to resolve the issue:
Artifactory Version: 4.15.0
Docker Version: 1.12.0
Stop Nginx service (service nginx stop)
Open your conf file in nginx (/etc/nginx/sites-enabled/default.conf) and change following line in it:
rewrite ^/(v1|v2)/(.*) /api/docker/build-images/$1/$2;
to
rewrite ^/(v2)/(.*) /api/docker/build-images/$1/$2;
Example below:
server {
listen 8000 ssl;server_name artifactory.corpintra.net;
if ($http_x_forwarded_proto = '') {
set $http_x_forwarded_proto $scheme;
}
## Application specific logs
access_log /var/log/nginx/build-docker-access.log;
error_log /var/log/nginx/build-docker-error.log;
rewrite ^/(v2)/(.*) /api/docker/build-images/$1/$2;
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
proxy_read_timeout 900;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_pass http://localhost:8081/artifactory/;
proxy_set_header X-Artifactory-Override-Base-Url $http_x_forwarded_proto://$host:$server_port;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}}
Restart Nginx (service nginx restart)
Related
I am working with Magento 2.4.2 (Adobe Commerce Enterprise Edition) and have a local site set up using the Magento Cloud Docker setup. I would like to change the nginx timeout setting to be long enough to let a page I'm testing run for as long as it needs to but still render the page on the browser in the frontend.
Is there a specific environment variable that I can set in my docker-compose.yml file to accomplish this? I'm not seeing anything that would make this update in the docker-environment or Dockerfile files. Do I just have to add my own custom lines to either of these files to update the timeout setting?
if you use magento cloud docker development, no you can't without overriding the docker image.
if you want to set nginx timeout, you need to override the nginx docker image and include it in docker-compose.override.yml. here are the step :
copy vendor/magento/magento-cloud-docker/images/nginx to .docker/images/nginx, i.e like this
edit .docker/images/nginx/1.19/etc/nginx.conf and .docker/images/nginx/1.19/etc/vhost.conf
create docker-compose.override.yml , like this
and run docker-compose up --build --force-recreate --no-deps --remove-orphans -d
check this link for the full example.
Note : the .docker/config.env file will be overwritten when you run ./vendor/bin/ece-docker 'build:compose'
According to Adobe Commerce support, this isn't possible on their Cloud platform which is very unfortunate
In terms of a local environment for testing, a method which is quicker and hackier than the one presented by Deki above is below:
ssh into your tls docker container
edit the /etc/nginx/conf.d/default.conf file as per below:
server {
listen 80;
listen 443 ssl;
server_name _;
ssl_certificate /etc/nginx/ssl/magento.crt;
ssl_certificate_key /etc/nginx/ssl/magento.key;
**# Add the 3 lines below**
proxy_read_timeout NEW_TIMEOUT_VALUE;
proxy_connect_timeout NEW_TIMEOUT_VALUE;
keepalive_timeout NEW_TIMEOUT_VALUE;
location / {
proxy_pass http://varnish:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}
I have different versions of my web application running in Docker containers. And nginx is running on my host machine.
Is it possible to access the desired deployed version of my web application with the help of sub-domain such as v1.myapp.io, v2.myapp.io without reconfiguring and restarting the nginx?
I also want to access future versions in the same way?
Could anyone tell me if there is any way to achieve it?
Please consider me a newbie to Docker/nginx world.
Thanks in Advance.
Yes, although it can be done but its very difficult to achieve with docker only. kubernetes will make this very easy and everything like dns, service mapping is provided out of the box. I will include both docker and kubernetes approach:
Docker approach:
A first draft will look like this, use regex in nginx server_name and set the docker container names with a pattern. Create a /etc/hosts entry for different containers like:
172.16.0.1 v1.docker.container
172.16.0.2 v2.docker.container
And nginx server conf look like:
server {
listen 80;
server_name "~^(?<ns>[a-z]+.+)\.myapp\.io";
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$ns.docker.container";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}
Kubernetes approach:
Create different service and deployment for different versions in a namespace. Lets say namespace is 'app-namespace'. Service names are self explanatory:
APP version v1: v1-app-service
APP version v2: v2-app-service
To make nginx more flexible you can add the service name as namespace to $proxyserver
Nginx rule:
server {
listen 80;
server_name "~^(?<version>[a-z]+.+)\.myapp\.io";
# you can replace this with kubernetes dns server IP
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$version.app-namespace.svc.kubernetes";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}
I found another solution to this problem after digging a lot. It can be easily be done with Automated Nginx Reverse Proxy for Docker.
Once docker container for ngninx was up and running on my system.
I spun two docker containers of my webapp(diff versions) with the following command:
docker run -e VIRTUAL_HOST=v1.myapp.io --name versionOne -d myapp.io:v1
docker run -e VIRTUAL_HOST=v2.myapp.io --name versionTwo -d myapp.io:v2
and it worked out for me.
Additional notes:
1. I am using dnsmasq for handling all dns queries
My Goal
I want to use a single NGinx docker container as a proxy.
I want it to respond to traffic on my domain: "sub.domain.com" and listen on port 80 and 443.
When traffic comes in on /admin I want it to direct all traffic to one docker container (say... admin_container:6000).
When traffic comes in on /api I want it to direct all traffic to another docker container (say... api_container:5500).
When traffic comes in on any other path (/anything_else) I want it to direct all traffic to another docker container (say... website_container:5000).
Some Helpful Context
Real quick, let me provide some context in case it's helpful. I have a NodeJS website running in a docker container. I'd like to also have an Admin section, created with ASP.NET Core that runs in a second docker container. I'd like both of these websites to share and make use of a single ASP.NET Core Web Api project, running in a third docker container. So, one NodeJS project and two ASP.NET Core projects, that all live on a single subdomain:
sub.domain.com/
Serves the main website
sub.domain.com/admin
Serves the Admin website
sub.domain.com/api
Serves API endpoints and handles Database connectivity
What I have So Far
So far I have the NGinX reverse proxy set up and a single docker container for the NodeJS application. Currently all traffic on :80 is redirected to 443. All traffic on 443 is directed to the NodeJS docker container, running privately on :5000. I'll admit I'm not great with NGinx and don't fully understand how this works.
The NGinx.conf file
worker_processes 2;
events { worker_connections 1024; }
http {
sendfile on;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
upstream docker-nodejs {
server nodejs_prod:5000;
}
server {
listen 80;
server_name sub.domain.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name sub.domain.com;
ssl_certificate /etc/nginx/ssl/combined.crt;
ssl_certificate_key /etc/nginx/ssl/mysecretkeyfile.key;
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_pass http://docker-nodejs;
}
}
}
Quick Note: In the above code "nodejs_prod:5000" refers to the container who's name is "nodejs_prod" and listens on port 5000. I don't understand how that works, but it is working. Somehow docker is creating DNS entries in the private network, one for each container name. I'm actually using docker-compose.
My Actual Question: How will this NGinx.conf file look when I have 2 more websites (each a docker container). It's important that /admin and /api are sent to the correct docker containers, and not handled by the "catch all" location. I'm imagining that I'll have a "catch all" location which captures all traffic that DOESN'T START WITH /admin OR /api.
Thank you!
In your nginx config you want to add location rules, such as
location /admin {
proxy_pass http://admin_container:6000/;
}
location /api {
proxy_pass http://api_container:5500/;
}
This redirects /admin to admin_container port 6000, and /api to api_container and port 5500.
docker-compose creates a network between all it's containers. Which is why http://api_container:5500/ points to the container named api_container and the port 5500. This can be used for communication between containers. You can read more about it here https://docs.docker.com/compose/networking/
I've deployed an on prem instance of Nexus OSS, that is reached behind a Nginx reverse proxy.
On any attempt to push docker images to a repo created on the Nexus registry I'm bumping into a
413 Request Entity Too Large in the middle of the push.
The nginx.conf file is looking like so:
http {
client_max_body_size 0;
upstream nexus_docker {
server nexus:1800;
}
server {
server_name nexus.services.loc;
location / {
proxy_pass http://nexus_docker/;
proxy_set_header Host $http_post;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
The nginx is deployed using docker, and I've successfully logged in to it using docker login.
I've tried multiple other flags, such as the chunkin and such. But nothing seems to work.
That's due to your server block having a default value for client_max_body_size of around 1MB in size when unset.
To resolve this, you will need to add the following line to your server block:
# Unlimit large file uploads to avoid "413 Request Entity Too Large" error
client_max_body_size 0;
http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size
As it turns out, the linux distro running the containered nginx server was itself running a variation of nginx for any incoming request.
Once we set the client_max_body_size to 0 on the nginx configuration file which the OS ran, it worked.
I have Nexusv3.6 and created a Docker repo docker-repo (type: hosted) and a Docker group docker-group (type: group).
For both I enabled HTTPS connector:
docker-repo on Port 8101 and docker-group on Port 8102.
I added docker-repo to my docker-group.
Now I am able to push/pull an image to/from docker-repo directly like:
docker push myhost.com:8101/mymimage:latest
But when I try to push to the group like this:
docker push myhost.com:8102/docker-repo/mymimage:latest
I get an error saying: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value
Any ideas what's the problem here?
I solved this problem with NGINX as follows:
Updated.
In the following example, "repository/docker" is a group that combines docker-proxy and docker-hosted repositories.
All HEAD* and GET requests are proxied to the docker repository group (hosted + proxy).
All "changing" requests are proxied to the docker-hosted repository directly.
*One exception. HEAD /v2/.../blobs/ should be proxied to the hosted repo because it called before push blobs to the hosted repo and we have to check the blob existence in the hosted repo. Otherwise we get an error: blob unknown: blob unknown to registry
server {
listen *:443 default_server ssl;
.........................
location ~ ^/(v1|v2)/[^/]+/?[^/]+/blobs/ {
if ($request_method ~* (POST|PUT|DELETE|PATCH|HEAD) ) {
rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
}
rewrite ^/(.*)$ /repository/docker/$1 last;
}
location ~ ^/(v1|v2)/ {
if ($request_method ~* (POST|PUT|DELETE|PATCH) ) {
rewrite ^/(.*)$ /repository/docker-hosted/$1 last;
}
rewrite ^/(.*)$ /repository/docker/$1 last;
}
location / {
proxy_pass http://nexus:8081/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto "https";
}
}
You can verify the settings by running:
# pull via proxy
docker pull nexus.your.domain/ubuntu
# push to the hosted repository
docker push nexus.your.domain/ubuntu
According to the official documentation about repository groups for docker:
A repository group is the recommended way to expose all your
repositories for read access to your users.
and, from the documentation about pushing images in private registries
You can not push to a repository group or a proxy repository.