Redirecting to docker registry with nginx - docker

All I would like to do is control the top endpoint (MY_ENDPOINT where users will login and pull images. The registry and containers are being hosted (DOCKER_SAAS), so all I need is a seemingly simple redirect. Concretely, where you would normally do:
docker login -u ... -p ... DOCKER_SAAS
docker pull DOCKER_SAAS/.../...
I would like to allow:
docker login -u ... -p ... MY_ENDPOINT
docker pull MY_ENDPOINT/.../...
And even more optimally I would prefer:
docker login MY_ENDPOINT
docker pull MY_ENDPOINT/.../...
where the difference in the last item is that the endpoint contains a hashed version of the username and password, which is set into an Authorization header (using Basic) - so the user doesn't even need to worry about username and password, just their URL. I've tried a proxy_pass as we are already doing for basic packaging (using HTTPS), but that fails with a 404 (in part because we do not handle /v2 - do I need to redirect that through, also?). This led me to https://docs.docker.com/registry/recipes/nginx/, but this seems to only be pertinent if you are hosting the registry. Is what I am trying to do even possible?

It sounds like there is also an Nginx or similar reverse-proxy-server in front of the DOCKER_SAAS. Does the infrastructure look like this?
[MY_ENDPOINT: nginx] <--> ([DOCKER_SAAS ENDPOINT: ?] <--> [DOCKER_REGISTRY])
My guess is that since the server [DOCKER_SAAS ENDPOINT: ?] is apparently configured with a fixed domain name, it expects exactly that domain name in the request header (e.g. Host: DOCKER_SAAS.TLD). So the problem is probably that when proxying from [MY_ENDPOINT: nginx] to [DOCKER_SAAS ENDPOINT: ?] the wrong Host header is sent along, i.e. by default the host header MY_ENDPOINT.TLD is sent along, but it should be DOCKER_SAAS.TLD instead. E.g.:
upstream docker-registry {
server DOCKER_SAAS.TLD:8443;
}
server {
...
server_name MY_ENDPOINT.TLD;
location / {
proxy_pass https://docker-registry/;
proxy_set_header Host DOCKER_SAAS.TLD; # set the header explicitly
...
}
}
or
server {
...
server_name MY_ENDPOINT.TLD;
location / {
proxy_pass https://DOCKER_SAAS.TLD:8443/;
proxy_set_header Host DOCKER_SAAS.TLD; # set the header explicitly
...
}
}
Regarding this:
And even more optimally I would prefer: docker login MY_ENDPOINT
This could be set on the proxy server ([MY_ENDPOINT: nginx]), yes. (The Authorization: "Basic ..." can be dynamically filled with the respective token extracted from the MY_ENDPOINT, and so on). However, the docker CLI would still ask for a username and password anyway. Yes, the user can enter dummy values (to make the CLI happy), or this would also work though:
docker login -u lalala -p 123 MY_ENDPOINT
But this would be inconsistent, and would rather confuse the users, imho. So better let it be...

This simple config works both with GitHub and Amazon ECR:
server {
listen 80;
server_name localhost;
location / {
proxy_set_header Authorization "Basic ${NGINX_AUTH_CREDENTIALS}";
proxy_pass https://registry.example.com;
}
}
${NGINX_AUTH_CREDENTIALS} is a placeholder for actual hash that Docker uses to authenticate. You can get it from $HOME/.docker/config.json after using docker login once:
{
"auths": {
"registry.example.com": {
"auth": "THIS STRING"
}
}
Since proxy injects/replaces authentication header, there is no need to use docker login, just pull using the address of the proxy instead of registry address.
Why 404?
I had several 40X errors trying to test the proxy to GitHub with curl:
bad credentials - 404, not 401 or 403 as it normally is.
GET /v2/_catalog - 404 (not supported on GitHub yet, in backlog). Use GET /v2/repo_name/image_name/tags/list instead.
curl without -XGET - 405, it gives response anyway but to get 200 you need to explicitly use GET (-XGET)
Despite all that docker pull worked flawlessly from the beginning, so I recommend using it for testing.
How to handle /v2/
location / matches everything, including /v2/, so there is no particular need for that in proxy.

Related

How to have an AWS ELB forward the actual host name to the target group instead of the ELB's host name?

We have a Ruby/Rails website we're migrating from Heroku to AWS. The original dev is not available. I'm now trying to complete the migration. My background is in the Windows / .NET world. This Linux / Ruby/Rails environment is quite foreign to me...
Here's the current environment I've set-up:
Route 53
Record Name
Record Type
Alias
Alias Route Traffic To
foo.example.com
A
yes
cloudfront: xyz.cloudfront.net
CloudFront
Domain Name
Alternate Domain Names
Origin Domain
Origin Protocol
Behavior Protocol
xyz.cloudfront.net
foo.example.com
foo.us-west-2.elb.amazonaws.com
HTTP only
Redirect HTTP to HTTPS
The CloudFront distribution:
uses an AWS issued SSL cert
handles the http to https redirect
forwards the request to the ELB over http (not https)
Load Balancer
DNS Name
Listener Rule
Forward To
foo.us-west-2.elb.amazonaws.com
HTTP 80: default action
Target Group: foo-ec2
Target Group: foo-ec2 contains a single Ubuntu ec2 instance running nginx/1.18.0 + Phusion Passenger 6.0.10 to serve up the Ruby/Rails site.
nginx config
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL Config - we should NEVER receive 443/https traffic;
# CloudFront manages https traffic => AWS ELB => http to this server
#listen 443 ssl default_server;
#listen [::]:443 ssl default_server;
server_name foo.example.com
foo.us-west-2.elb.amazonaws.com;
# Tell Nginx and Passenger where the app's 'public' directory is
root /var/www/foo_example/public;
# Turn on Passenger
passenger_enabled on;
passenger_app_env production;
passenger_ruby /home/ubuntu/.rbenv/versions/2.6.8/bin/ruby;
}
Issue
The rails app starts up without error and is served over https. However, when a user attempts to log in / authenticate, the Devise gem sends back a redirect using http and the ELB's DNS name.
Example
sign_in request
Request URL: https://foo.example.com/users/sign_in
Request Method: POST
Status Code: 302
sign_in response
location: http://foo.us-west-2.elb.amazonaws.com/users
server: nginx/1.18.0 + Phusion Passenger(R) 6.0.10
status: 302 Found
Notice the request was over https and our domain:
https://foo.example.com
But now we're over http and the ELB's domain:
http://foo.us-west-2.elb.amazonaws.com
My assumption
The devise gem is seeing the host from the ELB and then generates the URL from the ELB host, creating two issues:
we are now on http since the ELB communicates with the ec2 instance over http
we are now on the ELB's host name, foo.us-west-2.elb.amazonaws.com, instead of our name, foo.example.com
I've looked into the devise documentation to see if we can just pass in the http protocol and domain to use when creating the post back, but my ruby knowledge is limited. Plus, I think this would be the "bad" path; where the "good" path would be to have the AWS ELB forward the actual domain name, instead of it's own.
I've reviewed several SO and related stack sites with similar questions, but I've either ended up with an infinite loop redirect, or the various config changes have resulted in the same behavior of the devise gem creating the wrong URL post back.
These two questions seem to be the closest, but I'm not quite able to make the "connection" between the answers and my limited knowledge of this ecosystem.
AWS Cloudfront + Load Balancer, url changes from main domain to load balancer subdomain
cloudfront domain replaced by application load balancer dns name when redirecting from http to https
Question
How can I get the AWS ELB to forward our domain, foo.example.com, to the ec2 target group and not the ELB's domain?
After more experimentation with AWS settings, the solution is actually rather simple. The other answers I posted in the question were vague in the actual settings, so here's the concrete solution.
In CloudFront, you need to create a new origin request policy, not a cache policy:
Open up CloudFront
go to Policies (left nav)
click the "Origin Request" tab
click the "create origin request policy" button
name the policy whatever you want, i.e., "my origin request policy"
under "Origin request settings" > Headers: select "Include the following headers"
under "Add header": check the "Host" option
click the "Create" button
The policy will look like this:
Once the new origin request policy has been created:
head back to the CloudFront distributions
click your distribution's Id so you can edit it
click the "Behaviors" tab
select your behavior and edit
scroll down to "Cache key and origin requests"
make sure the "Cache policy and origin request policy (recommended)" is selected
under the "Origin request policy - optional", select your new policy, i.e., "my origin request policy"
save changes
The behavior will look like this (I'm using no caching for now to verify the ec2 instance is getting all the requests):
That's it. The host header is now correctly passed through to the ELB and ec2 instance. Nothing else needs to be done with the ELB.
I verified the host header was being used in all requests by modifying the nginx logging option to include the $host variable in the log file (and did a bit more customization to the OOB format):
# prefixed log with '[my-log]', but it's not needed; remove.
log_format my-log '[my-log] $http_x_forwarded_for - $remote_user [$time_local] '
'"$request_method $scheme://$host$request_uri $server_protocol" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent" $request_time';
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL Config - we should NEVER receive 443/https traffic;
# CloudFront manages https traffic => AWS ELB => http to this server
#listen 443 ssl default_server;
#listen [::]:443 ssl default_server;
server_name foo.example.com
foo.us-west-2.elb.amazonaws.com;
# create the our log file
access_log /var/log/nginx/my-log.access.log my-log;
# Tell Nginx and Passenger where the app's 'public' directory is
root /var/www/foo_example/public;
# Turn on Passenger
passenger_enabled on;
passenger_app_env production;
passenger_ruby /home/ubuntu/.rbenv/versions/2.6.8/bin/ruby;
}
Surely this will help future me as well as others.
For infinite redirects, I made this additional change and the site is serving fine.
Select Origin. Edit origin
Select Match viewer in Protocol. Save. Invalidate. Done

How do I map a location to an upstream server in Nginx?

I've got several Docker containers acting as web servers on a bridge network. I want to use Nginx as a proxy that exposes a service (web) outside the bridge network and embeds content from other services (i.e. wiki) using server side includes.
Long story short, I'm trying to use the configuration below, but my locations aren't working properly. The / location works fine, but when I add another location (e.g. /wiki) or change / to something more specific (e.g. /web) I get a message from Nginx saying that it "Can't get /wiki" or "Can't get /web" respectively:
events {
worker_connections 1024;
}
http {
upstream wiki {
server wiki:3000;
}
upstream web {
server web:3000;
}
server {
ssi on;
location = /wiki {
proxy_pass http://wiki;
}
location = / {
proxy_pass http://web;
}
}
}
I've attached to the Nginx container and validated that I can reach the other containers using CURL- they appear to be working properly.
I've also read the Nginx pitfalls and know that using hostnames (wiki, web) isn't ideal, but I don't know the IP addresses ahead of time and have tried to counter any DNS issues by telling docker-compose that the nginx container depends on web and wiki.
Any ideas?
You must turn proxy_pass http://wiki; to proxy_pass http://wiki/;.
As I know, Nginx would take two different way with/without backslash at the end of uri. You may find more details about proxy_pass directive on nginx.org.
In your case, a backslash(/) is essential as a uri to be passed to server. You've already got error message "Can't get /wiki". In fact, this error message means that there is no /wiki in server wiki:3000, not in Nginx scope.
Getting better knowing about proxy_pass directive with/without uri would help you much.
I hope this would help.

serving web app and python using nginx on remote server

Setup:
1> Web GUI using Angular JS hoasted in tomcat server and Python app using Flask is running on an AWS server.
2> I am working in a secure server hence am unable to access AWS directly.
3> I have setup NGINX to access GUI app from my local secured network. GUI app is running on awsserver:9506/appName
4> Flask app is running in AWS server hosted on 127.0.0.1:5000. This app has 2 uri's cross and accross:
127.0.0.1:5000/cross
127.0.0.1:5000/accross
Now in my GUI after NGINX setup i am able to access it using domain name and without port:
doman.name/appName
Now when i try to use it send a request to server my url changes to:
doman.name/cross. I did the changes in NGINX config and am able to access it but am not able to get a response back. Please find below my NGINX config file:
server {
listen 80;
server_name domain.name;
root /home/Tomcat/webapps/appName;
location / {
proxy_pass http://hostIP:9505/; #runs the tomcat home page
}
location /appName/ {
proxy_pass http://hostIP:9505/appName; #runs the application home page
}
location /cross/ {
proxy_pass http://127.0.0.1:5000/cross; #hits the python flask app and am trying to send post
}
}
Also what i noticed is that my POST request is being converted to GET at the server by NGINX
You need to be consistent with your use of the trailing /. With the proxy_pass statement (as with alias) nginx performs text substitution to form the rewritten URI.
Is the URI of the API /cross or /cross/? POST is converted to GET when the server is forced to perform a redirect (for example, to append a /).
Specifying the same URI on the location and proxy_pass is unnecessary as no changes are made.
If the hostIP in your first two location blocks is the same, and assuming that the missing trailing / is accidental, they can be combined into a single location block.
For example:
location / {
proxy_pass http://hostIP:9505;
}
location /cross {
proxy_pass http://127.0.0.1:5000;
}
See this document for more.

How can I host my API and web app on the same domain?

I have a Rails API and a web app(using express), completely separate and independent from each other. What I want to know is, do I have to deploy them separately? If I do, how can I make it so that my api is in mysite.com/api and the web app in mysite.com/
I've seen many projects that do it that way, even have the api and the app in separate repos.
Usually you don't expose such web applications directly to clients. Instead you use a proxy server, that forwards all incoming requests to the node or rails server.
nginx is a popular choice for that. The beginners guide even contains a very similar example to what you're trying to do.
You could achieve what you want with a config similar to this:
server {
location /api/ {
proxy_pass http://localhost:8000;
}
location / {
proxy_pass http://localhost:3000;
}
}
This is assuming your API runs locally on port 8000 and your express app on port 3000. Also this is not a full configuration file - this needs to be loaded in or added to the http block. Start with the default config of your distro.
When there are multiple location entries nginx chooses the most specific one. You could even add further entries, e.g. to serve static content.
While Svens answer is completely correct for the question given. I'd prefer doing it at the DNS level so that I can change the server to a new location just in case my API or Web App experience heavy load. This helps us to run our APIs without affecting WebApp and vice-versa
DNS Structure
api.mysite.com => 9.9.9.9 // public IP address of my server
www.mysite.com = > 9.9.9.9 // public IP address of my server
Since now you'd want both your WebApp and API to run on the same server, you can use nginx to forward requests appropriately.
server {
listen 80;
server_name api.mysite.com;
# ..
# Removed for simplicity
# ..
location / {
proxy_pass http://localhost:3000;
}
}
server {
listen 80;
server_name www.mysite.com;
# ..
# Removed for simplicity
# ..
location / {
proxy_pass http://localhost:8000;
}
}
Any time in future if you are experiencing overwhelming traffic, you can just alter the DNS to point to a new server and you'd be good.

OAuth token validation from HAProxy or Apache mod_proxy

I have a microservice deployed on 3 nodes sitting behind a HAProxy load balancer all inside internal network. The services are protected using OAuth2 APIS authorization server. Now, I want to move the HAProxy to DMZ. And I want to reject requests that do not have auth token in the header and also validate the auth token by calling OAuth REST API.
In HAProxy I couldn't find a way to do this. There is an option httpchk which can be used for healthcheck. I'm looking for a similar feature that can be used to validate each incoming request.
Can anyone please help suggest me on how to implement this using HAProxy or Apache mod_proxy?
There's the Apache module mod_auth_openidc that would allow you to validate OAuth 2.0 tokens against an Authorization Server, see: https://github.com/zmartzone/mod_auth_openidc. That module can be combined with mod_proxy to achieve what you are looking for.
In HAProxy I couldn't find a way to do this.
For the record, as of 2021 you can. Here's a HAProxy official blog post about using OAuth https://www.haproxy.com/blog/using-haproxy-as-an-api-gateway-part-2-authentication/.
TL;DR: install this haproxy-lua-oauth script, then you can come up with conf like this snippet
frontend api_gateway
# Always use HTTPS to protect the secrecy of the token
bind :443 ssl crt /usr/local/etc/haproxy/pem/test.com.pem
# Accept GET requests and skip further checks
http-request allow if { method GET }
# Deny the request if it's missing an Authorization header
http-request deny unless { req.hdr(authorization) -m found }
# Verify the token by invoking the jwtverify Lua script
http-request lua.jwtverify
# Deny the request unless 'authorized' is true
http-request deny unless { var(txn.authorized) -m bool }
# (Optional) Deny the request if it's a POST/DELETE to a
# path beginning with /api/hamsters, but the token doesn't
# include the "write:hamsters" scope
http-request deny if { path_beg /api/hamsters } { method POST DELETE } ! { var(txn.oauth_scopes) -m sub write:hamsters }
# If no problems, send to the apiservers backend
default_backend apiservers

Resources