Update
The details in this question are getting long, but I think it narrows down to this:
For some reason the host name matters to Nginx when it's trying to figure out whether to proxy the request. If the host name is set to git.example.com the request does not seem to go through, but if it's set to 203.0.113.2 then it goes through. Why does the host name matter?
Filed an issue with Nginx here
And docker compose
Start of original question
When I type in the IP address of the reverse proxy directly into my browser bar, it does perform the redirect.
When using a URL that is resolved via the /etc/hosts entry 203.0.113.2 git.example.com the "Welcome to Ngnix page" is shown. Any ideas? This is the configuration:
server {
listen 203.0.113.2:80 default_server;
server_name 203.0.113.2 git.example.com;
proxy_set_header X-Real-IP $remote_addr; # pass on real client IP
location / {
proxy_pass http://203.0.113.1:3000;
}
}
This is the docker-compose.yml file that is used to launch the whole thing:
version: '3'
services:
gogs-nginx:
build: ./proxy
ports:
- "80:80"
networks:
mk1net:
ipv4_address: 203.0.113.2
gogs:
image: gogs/gogs
ports:
- "3000:3000"
volumes:
- gogs-data:/data
networks:
mk1net:
ipv4_address: 203.0.113.3
volumes:
gogs-data:
external: true
networks:
mk1net:
ipam:
config:
- subnet: 203.0.113.0/24
One interesting thing is that I can navigate to for example:
http://203.0.113.2/issues
The log for the above URL is:
gogs-nginx_1 | 203.0.113.1 - - [07/Oct/2018:11:28:06 +0000] "GET / HTTP/1.1" 200 38825 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"
If I then change 203.0.113.2 with git.example.com (So that the url ends up being git.example.com I get Nginxs "404 not found" page, and the log says:
gogs-nginx_1 | 2018/10/07 11:31:34 [error] 8#8: *10 open() "/usr/share/nginx/html/issues" failed (2: No such file or directory), client: 203.0.113.1, server: localhost, request: "GET /issues HTTP/1.1", host: "git.example.com"
If I only use http://git.example.com as the URL I get the NGINX welcome page, and the following log:
gogs-nginx_1 | 203.0.113.1 - - [07/Oct/2018:11:34:39 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36" "-"
It looks like Nginx understands that the request is for the proxy because it logs the IP of the proxy, but it does not redirect to the proxy and returns a 304 ...
Using Curl to perform requests
Using curl with a host name parameter that targets the proxy like this:
curl -H 'Host: git.example.com' -si http://203.0.113.2
Results in the Nginx welcome page:
ole#mki:~/Gogs/.gogs/docker$ curl -H 'Host: git.example.com' -si http://203.0.113.2
HTTP/1.1 200 OK
Server: nginx/1.15.1
Date: Sun, 07 Oct 2018 17:09:11 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 03 Jul 2018 13:27:08 GMT
Connection: keep-alive
ETag: "5b3b79ac-264"
Accept-Ranges: bytes
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
nginx.org.<br/>
Commercial support is available at
nginx.com.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
But if I change the host name to the ip address like this:
Using curl with a host name parameter that targets the proxy like this:
curl -H 'Host: 203.0.113.2' -si http://203.0.113.2
Then the proxy works as it should:
ole#mki:~/Gogs/.gogs/docker$ curl -H 'Host: 203.0.113.2' -si http://203.0.113.2
HTTP/1.1 302 Found
Server: nginx/1.15.1
Date: Sun, 07 Oct 2018 17:14:46 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 34
Connection: keep-alive
Location: /user/login
Set-Cookie: lang=en-US; Path=/; Max-Age=2147483647
Set-Cookie: i_like_gogits=845bb09d69587b81; Path=/; HttpOnly
Set-Cookie: _csrf=neGgBfG4LdOcdrdeA0snHjVGz4s6MTUzODkzMjQ4NjE5MzEzNzI3OQ%3D%3D; Path=/; Expires=Mon, 08 Oct 2018 17:14:46 GMT; HttpOnly
Set-Cookie: redirect_to=%252F; Path=/
Found.
I am sorry, I failed to realize what's happening on your side because the information is sometimes confusing and sometimes incomplete. But Stackoverflow provides a great explanation on what is considered a good question: How to create a Minimal, Complete, and Verifiable example and so I have just tried to implement a minimal example of a system you are likely going to build.
Below I am providing all the files and will show you a test run as well.
File #1: docker-compose.yml
gogs:
image: gogs/gogs
web:
build: .
ports:
- 8000:80
links:
- gogs
I have outdated Docker at my computer and I do not want to bother with Docker networking, so I have just linked both containers using Docker links. This is the most important part and the link will ensure that (1) our web container depends on gogs; (2) we are able to reference gogs IP from inside web as just gogs. Docker will resolve the name to an IP assigned to the container.
Since I want a minimal example, I've skipped everything else as irrelevant. For example, volume.
File #2: Dockerfile
Newer Compose versions support config options specified right in docker-compose.yml, but I need a custom Dockerfile instead. It's trivial:
FROM nginx:stable-alpine
COPY gogs.conf /etc/nginx/conf.d
File #3: gogs.conf
And finally we need Nginx configuration for proxy:
server {
listen 80 default_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
proxy_pass http://gogs:3000;
}
}
You may notice here we are referring another container simply by name gogs and we need to know what port number it is exposes. We know: 3000.
Running
$ docker-compose build
$ docker-compose up
It's up and running:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f74293df630 g_web "nginx -g 'daemon off" 2 minutes ago Up 26 seconds 0.0.0.0:8000->80/tcp g_web_1
dfa2dbaa6074 gogs/gogs "/app/gogs/docker/sta" 2 minutes ago Up 26 seconds 22/tcp, 3000/tcp g_gogs_1
web container is exposed to the world at port number 8000.
Tests
by IP
Let's request it by IP:
$ curl -si http://192.168.99.100:8000/
HTTP/1.1 302 Found
Server: nginx/1.14.0
Date: Sun, 07 Oct 2018 15:13:55 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 31
Connection: keep-alive
Location: /install
Set-Cookie: lang=en-US; Path=/; Max-Age=2147483647
Set-Cookie: i_like_gogits=50411f542e2ae8f8; Path=/; HttpOnly
Set-Cookie: _csrf=ZJxRPqnqayIbpAYgZ22zrPIOaSo6MTUzODkyNTIzNTQ2NTg5MDE1NA%3D%3D; Path=/; Expires=Mon, 08 Oct 2018 15:13:55 GMT; HttpOnly
Found.
Corresponding log file:
web_1 | 192.168.99.1 - - [07/Oct/2018:15:14:24 +0000] "GET / HTTP/1.1" 302 31 "-" "curl/7.61.1" "-"
gogs_1 | [Macaron] 2018-10-07 15:14:24: Started GET / for 192.168.99.1
gogs_1 | [Macaron] 2018-10-07 15:14:24: Completed GET / 302 Found in 199.519µs
gogs_1 | 2018/10/07 15:14:24 [TRACE] Session ID: 38d06d393a9e9d21
gogs_1 | 2018/10/07 15:14:24 [TRACE] CSRF Token: Xth986dFWhhj8w8vBdIqRZu4SbI6MTUzODkyNTI2NDYxMDYzNzAyNA==
I can see from the log that (1) both containers work and they were used to process the request; (2) 192.168.99.1 is my host's IP address, which means "gogs" successfully gets a real request IP via X-Forwarded-For.
by domain name
OK, let's request using a domain name:
$ curl -H 'Host: g.example.com' -si http://192.168.99.100:8000/
Trust me, this is just sufficient. Host is an HTTP protocol header to pass domain name. And any browser will do the same under the hood.
and the corresponding log file is --
gogs_1 | [Macaron] 2018-10-07 15:32:49: Started GET / for 192.168.99.1
gogs_1 | [Macaron] 2018-10-07 15:32:49: Completed GET / 302 Found in 618.701µs
gogs_1 | 2018/10/07 15:32:49 [TRACE] Session ID: 81f64d97e9c3dd1e
gogs_1 | 2018/10/07 15:32:49 [TRACE] CSRF Token: X5QyHM4LMIfn8OSJD1gwSSEyXV46MTUzODkyNjM2OTgyODQyMjExMA==
web_1 | 192.168.99.1 - - [07/Oct/2018:15:32:49 +0000] "GET / HTTP/1.1" 302 31 "-" "curl/7.61.1" "-"
No changes, everything works as expected.
Related
Using this docker-compose file:
version: '3'
services:
hello:
image: nginxdemos/hello
ports:
- 7080:80
tool:
image: wbitt/network-multitool
tty: true
networks:
default:
name: test-network
If I curl from the host, it works.
❯ curl -s -o /dev/null -v http://192.168.1.102:7080
* Expire in 0 ms for 6 (transfer 0x8088b0)
* Trying 192.168.1.102...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x8088b0)
* Connected to 192.168.1.102 (192.168.1.102) port 7080 (#0)
> GET / HTTP/1.1
> Host: 192.168.1.102:7080
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.23.1
< Date: Sun, 10 May 2071 00:06:00 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Expires: Sun, 10 May 2071 00:05:59 GMT
< Cache-Control: no-cache
<
{ [6 bytes data]
* Connection #0 to host 192.168.1.102 left intact
If I try to contact another container from within the network, it fails.
❯ docker exec -it $(gdid tool) curl -s -o /dev/null -v http://hello
* Could not resolve host: hello
* Closing connection 0
Is this intended behaviour? I thought networks within the same network (and using docker-compose) are meant to be able to talk by their service name?
I am bringing the containers up with docker-compose up -d
I am running a moodle docker container and one other container to call moodle web service function in a same network. I'm pretty sure I have set up the moodle properly because I can call moodle web service from postman. I am also aware i need to use the container alias of the running moodle container, which in this case is webserver. This is what i have tried
calling http://localhost:8000/webservice/rest/server.php?moodlewsrestformat=json&wstoken=fa4e7222df472b032ca9b4bd6d17595a&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2 from Postman and it return the following response
{
"users": [
{
"id": 2,
"username": "admin",
"firstname": "Admin",
"lastname": "User",
"fullname": "Admin User",
"email": "abc#def.com",
"department": "",
"firstaccess": 1655901976,
"lastaccess": 1655907821,
"auth": "manual",
"suspended": false,
"confirmed": true,
"lang": "en",
"theme": "",
"timezone": "99",
"mailformat": 1,
"description": "",
"descriptionformat": 1,
"profileimageurlsmall": "http://localhost:8000/theme/image.php/boost/core/1655902245/u/f2",
"profileimageurl": "http://localhost:8000/theme/image.php/boost/core/1655902245/u/f1",
"preferences": [
{
"name": "core_message_migrate_data",
"value": "1"
},
{
"name": "auth_manual_passwordupdatetime",
"value": "1655902098"
},
{
"name": "email_bounce_count",
"value": "1"
},
{
"name": "email_send_count",
"value": "1"
},
{
"name": "login_failed_count_since_success",
"value": "0"
},
{
"name": "_lastloaded",
"value": 1655907864
}
]
}
],
"warnings": []
}
but when i try calling the same API from the other container i get a 403 response status, i also tried calling the same API with curl inside the container using curl -v -g 'http://webserver/webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2'
with result:
* Trying 172.18.0.8:80...
* Connected to webserver (172.18.0.8) port 80 (#0)
> GET /webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2 HTTP/1.1
> Host: webserver
> User-Agent: curl/7.80.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 403 Forbidden
< Date: Wed, 22 Jun 2022 14:42:38 GMT
< Server: Apache/2.4.53 (Debian)
< X-Powered-By: PHP/7.4.29
< Content-Length: 0
< Connection: close
< Content-Type: text/html; charset=utf-8
<
* Closing connection 0
so it is clearly connected but somehow still return 403 response, am i missing something? is there another moodle settings i have to tweak?
I am using moodle-docker with added
networks:
default:
name: network_name
external: true
at the end of base.yml file so it could connect to the current project I'm working on which is a Node.js project that will call one of the web service on moodle. Here is the docker-compose.yml which relevant to this question
version: "3.7"
services:
test:
build: .
env_file:
- "./.env"
ports:
- 8085:8085
networks:
- network_name
networks:
network_name:
driver: bridge
name: network_name
this is the result of calling docker logs of the moodle container
172.18.0.1 - - [22/Jun/2022:15:38:27 +0000] "GET /webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2 HTTP/1.1" 200 1464 "-" "PostmanRuntime/7.29.0"
[Wed Jun 22 15:38:30.315720 2022] [php7:notice] [pid 22] [client 172.18.0.3:34082] Debugging: The server died because the web services or the REST protocol are not enable in \n* line 39 of /webservice/rest/server.php: call to debugging()\n
172.18.0.3 - - [22/Jun/2022:15:38:30 +0000] "GET /webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2 HTTP/1.1" 403 199 "-" "curl/7.80.0"
This error doesn't make any sense to me because i already enabled the web service and the REST protocol on the site administration settings. I also tried to enable web services for mobile devices with no luck so far. I'm also new to moodle development so I might miss something simple so any input is appreciated.
UPDATE:
After restarting the whole docker setup and try to curl from the container again I got this message:
* Trying 172.19.0.8:80...
* Connected to webserver (172.19.0.8) port 80 ()
> GET /webservice/rest/server.php?wstoken=beebce17854eff740ba85ec016542cfe&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2 HTTP/1.1
> Host: webserver
> User-Agent: curl/7.80.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Thu, 23 Jun 2022 07:34:35 GMT
< Server: Apache/2.4.53 (Debian)
< X-Powered-By: PHP/7.4.29
< Vary: Accept-Encoding
< Content-Length: 284
< Content-Type: text/html; charset=UTF-8
<
Install Behat before enabling it, use:
php admin/tool/behat/cli/init.php
<br />
<b>Warning</b>: Cannot modify header information - headers already sent by (output started at /var/www/html/lib/testing/lib.php:169) in <b>/var/www/html/lib/testing/lib.php</b> on line <b>171</b><br />
* Connection #0 to host webserver left intact
After further investigation I found that in config.docker-template.php there is a setting for $CFG->behat_wwwroot which has default value to http://webserver. I am not familiar how to setup testing framework in Moodle so I'm not sure how can I change this.
Now my problem is how to setup the moodle docker to both be exposed from localhost and webserver. Is anyone familiar with this?
Problem
Since you have not mentioned the port cURL tries to connect to port 80 which is the default http port. You can observe it here:
* Connected to webserver (172.18.0.8) port 80 (#0)
Solution
You need to specifically set it as 8000:
Instead of
'http://webserver/webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2'
It should be
'http://webserver:8000/webservice/rest/server.php?wstoken=fa4e7222df472b032ca9b4bd6d17595a&moodlewsrestformat=json&wsfunction=core_user_get_users&criteria[0][key]=id&criteria[0][value]=2'
you must use docker-compose to create your network.
I am new to nginx and trying to understand what is going on here. I have a docker compose file that starts up a nginx container like so:
proxy:
image: nginx:alpine
container_name: proxy
restart: unless-stopped
ports:
- "80:80"
- "443:443"
volumes:
- ./proxy/default.conf:/etc/nginx/conf.d/default.conf
Which copies my default.conf into the nginx container, which looks like this:
server {
listen 80;
listen [::]:80;
server_name localhost testthis;
return 301 https://www.google.com$request_uri;
}
So if I run curl -I http://localhost, I see google.com as expected
HTTP/1.1 301 Moved Permanently
Server: nginx/1.21.6
Date: Fri, 25 Feb 2022 06:39:39 GMT
Content-Type: text/html
Content-Length: 169
Connection: keep-alive
Location: https://www.google.com/
But if I run curl -I http://testthis, I get this response:
curl: (6) Could not resolve host: testthis
Why is this happening if the server names are on the same server block? Eventually I am wanting to set up a custom domain and subdomains to forward requests to specific localhost ports per app but not understanding how this works too well.
curl -I http://localhost works because localhost, by default, resolves to an IP from your machine (127.0.0.1), not because it's listed in your nginx's default.conf. And Docker configures your machine to forward traffic on port 80 and 443 (the ones used for HTTP and HTTPS) to that container.
The server_name directive in nginx's configuration makes it recognize requests with that DNS in the request's Host: header. It does not advertise that as a name for your network.
For that to work, you need to make your computer recognize testthis as a name for your computer. On Linux, edit /etc/hosts and add this line:
127.0.0.1 testthis
On Windows, I don't know, but you can certainly search for "windows hosts file" and get a similar method.
curl --connect-to testthis:80:127.0.0.1:80 http://google.com should do the trick
I am trying to get Ory Hydra working in Docker-Compose with Nginx. Due to my iterative approach, I already had a working system before adding Nginx. In other words, it was working, now it isn't.
The changes which I think might affect this process are: Nginx, Hydra's host name, oauth2 config in my demo application. Also, my setup is based on the Kratos-Hydra integration demo. Of course Kratos and the UI are now also accessed from Nginx, so that obviously has changed as well, but I don't think that's causing problems.
So here's what happens when I try to access a secured endpoint in my demo app:
Redirect to kratos-ui for login
Enter details and send request
Login succeeds
Hydra returns 403: You are not allowed to perform this action.
Nginx:
# kratos-selfservice-ui-node
server {
server_name self.localhost;
proxy_set_header Host self.localhost;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
proxy_pass http://self:3000;
}
}
# kratos
server {
server_name login.localhost;
#proxy_set_header Host ...;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
proxy_pass http://kratos:4433;
}
}
#hydra
server {
server_name oidc.localhost;
#proxy_set_header Host 127.0.0.1:4444;
#proxy_set_header Host oidc.localhost;
#proxy_set_header X-Real-IP $remote_addr;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
proxy_pass http://hydra:4444;
}
}
Request:
GET /oauth2/auth?client_id=auth-code-client&login_verifier=8b5f6d3f964c4470ab2e42fac90ae1c2&nonce=XTr2FJETXFsr6kxw3SlZsbh7rbQ_RMw8SdK3MeMCAs0&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Flogin%2Foauth2%2Fcode%2Fhydra&response_type=code&scope=openid+profile&state=4OSX7C_A84-u-6MlUZOlzjAAXiBYIzbKGfGwcAp1n1M%3D HTTP/1.1
Host: hydra:4444
User-Agent: <stuff>
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://self.localhost/
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Pragma: no-cache
Cache-Control: no-cache
Hydra entry in docker-compose:
# OIDC Server
# Configured to use Kratos for identities
hydra:
image: oryd/hydra:v1.6.0-alpine
container_name: hydra
depends_on:
- hydra-migrate
#ports:
#- 4444:4444 # Public port
#- 4445:4445 # Admin port
#- 5555:5555 # Port for hydra token user
command:
serve all --dangerous-force-http
volumes:
-
type: bind
source: ./config/hydra
target: /home/ory
environment:
- DSN=postgres://pguser:secret#postgres:5432/hydra?sslmode=disable
- OIDC_SUBJECT_IDENTIFIERS_SUPPORTED_TYPES=public,pairwise
- LOG_LEAK_SENSITIVE_VALUES=true
##- URLS_SELF_ISSUER=http://127.0.0.1:4444
##- URLS_SELF_PUBLIC=http://127.0.0.1:4444
#- URLS_SELF_ISSUER=http://hydra:4444
#- URLS_SELF_PUBLIC=http://hydra:4444
- URLS_SELF_ISSUER=http://oidc.localhost
- URLS_SELF_PUBLIC=http://oidc.localhost
- URLS_CONSENT=http://self.localhost/auth/hydra/consent
- URLS_LOGIN=http://self.localhost/auth/hydra/login
- URLS_LOGOUT=http://self.localhost/logout
- SECRETS_SYSTEM=youReallyNeedToChangeThis
- OIDC_SUBJECT_IDENTIFIERS_PAIRWISE_SALT=youReallyNeedToChangeThis
- OAUTH2_EXPOSE_INTERNAL_ERRORS=true;
- OAUTH2_INCLUDE_LEGACY_ERROR_FIELDS=true
restart: on-failure
networks:
- <ory>
Spring Boot App config:
spring:
security:
oauth2:
client:
registration:
hydra:
client-name: Demo OIDC Client with Spring Boot :D
client-id: auth-code-client
client-secret: secret
provider:
hydra:
issuer-uri: http://oidc.localhost/
And here's the client that I created:
docker exec hydra \
hydra clients create \
--endpoint http://127.0.0.1:4445 \
--id auth-code-client \
--secret secret \
--grant-types authorization_code,refresh_token \
--response-types code,id_token \
--scope openid,profile \
--callbacks http://localhost:8080/login/oauth2/code/hydra
/etc/hosts stuff I added:
# Dev stuff
127.0.0.1 self.localhost
127.0.0.1 login.localhost
127.0.0.1 oidc.localhost
127.0.0.1 oidc-demo.localhost
127.0.0.1 hello.localhost
Hydra logs:
< THIS IS FROM THE INITIAL REQUEST TO THE KRATOS UI >
time=2022-01-24T12:49:00Z level=info msg=started handling request http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:response_type=code&client_id=auth-code-client&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D&redirect_uri=http://localhost:8080/login/oauth2/code/hydra remote:192.168.16.11:43608 scheme:http]
time=2022-01-24T12:49:00Z level=info msg=access allowed audience=audit http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:response_type=code&client_id=auth-code-client&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D&redirect_uri=http://localhost:8080/login/oauth2/code/hydra remote:192.168.16.11:43608 scheme:http] service_name=ORY Hydra service_version=v1.6.0
time=2022-01-24T12:49:00Z level=info msg=completed handling request http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:response_type=code&client_id=auth-code-client&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D&redirect_uri=http://localhost:8080/login/oauth2/code/hydra remote:192.168.16.11:43608 scheme:http] http_response=map[status:302 text_status:Found took:15.9869ms]
time=2022-01-24T12:49:00Z level=info msg=started handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:GET path:/oauth2/auth/requests/login query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54218 scheme:http]
time=2022-01-24T12:49:00Z level=info msg=completed handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:GET path:/oauth2/auth/requests/login query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54218 scheme:http] http_response=map[status:200 text_status:OK took:3.034ms]
< THIS IS AFTER LOGIN >
time=2022-01-24T12:49:59Z level=info msg=started handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:GET path:/oauth2/auth/requests/login query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54292 scheme:http]
time=2022-01-24T12:49:59Z level=info msg=completed handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:GET path:/oauth2/auth/requests/login query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54292 scheme:http] http_response=map[status:200 text_status:OK took:3.7631ms]
time=2022-01-24T12:49:59Z level=info msg=started handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:PUT path:/oauth2/auth/requests/login/accept query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54296 scheme:http]
time=2022-01-24T12:49:59Z level=info msg=completed handling request http_request=map[headers:map[accept:application/json] host:hydra:4445 method:PUT path:/oauth2/auth/requests/login/accept query:login_challenge=3a6891edb669434f821a0d5413519bfe remote:192.168.16.2:54296 scheme:http] http_response=map[status:200 text_status:OK took:8.8812ms]
time=2022-01-24T12:49:59Z level=info msg=started handling request http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache referer:http://self.localhost/ user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:client_id=auth-code-client&login_verifier=fedb596a040648b8b626e0f7e4f3f04a&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Flogin%2Foauth2%2Fcode%2Fhydra&response_type=code&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D remote:192.168.16.11:43694 scheme:http]
time=2022-01-24T12:49:59Z level=info msg=access denied audience=audit error=map[message:request_forbidden reason:You are not allowed to perform this action. status:Forbidden status_code:403] http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache referer:http://self.localhost/ user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:client_id=auth-code-client&login_verifier=fedb596a040648b8b626e0f7e4f3f04a&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Flogin%2Foauth2%2Fcode%2Fhydra&response_type=code&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D remote:192.168.16.11:43694 scheme:http] service_name=ORY Hydra service_version=v1.6.0
time=2022-01-24T12:49:59Z level=info msg=completed handling request http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;q=0.5 cache-control:no-cache referer:http://self.localhost/ user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:96.0) Gecko/20100101 Firefox/96.0] host:127.0.0.1:4444 method:GET path:/oauth2/auth query:client_id=auth-code-client&login_verifier=fedb596a040648b8b626e0f7e4f3f04a&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2Flogin%2Foauth2%2Fcode%2Fhydra&response_type=code&state=-__end_skoEpW7KSAfzng1yZyOdJoF2-Cfzls-dccD4%3D remote:192.168.16.11:43694 scheme:http] http_response=map[status:302 text_status:Found took:8.6448ms]
Update
In the course of trying out everything before posting this question to stackoverflow.com, I went back to an older git commit which was working.
Bad news, it doesn't work anymore. I have the official kratos-hydra integration checked out and built ($ git status -> On branch hydra-integration \n Your branch is up to date with 'origin/hydra-integration'.), and I did the required steps, and now I get this:
$ docker exec hydra_hydra_1 \
hydra token user \
--client-id auth-code-client \
--client-secret secret \
--endpoint http://127.0.0.1:4444/ \
--port 5555 \
--scope openid,offline
Config file not found because "Config File ".hydra" Not Found in "[/home/ory]""
Setting up home route on http://127.0.0.1:5555/
Setting up callback listener on http://127.0.0.1:5555/callback
Press ctrl + c on Linux / Windows or cmd + c on OSX to end the process.
If your browser does not open automatically, navigate to:
http://127.0.0.1:5555/
< then I navigate to 127.0.0.1:5555, click on authorize application, I have to enter log in details, and then I get redirected to an error page >
Got error: The request is not allowed
http: Server closed
The browser doesn't add much info to that:
An error occurred
request_forbidden
The request is not allowed
You are not allowed to perform this action.
I've tried deleting all containers, images, volumes, and networks, browser cookies, using a different browser, restarting docker, restarting my computer. Same problem.
Something that seems odd is that the app always asks me to log in, even if I am logged in already when I go to the UI url manually. I remember that if I was already logged in, it wouldn't ask me to log in again?
Update
I was on the hydra-integration branch for some reason, instead of the hydra-integration-2021, which is why going to back to the basics didn't work. That's my mistake.
The actual project is not working, but after reevaluating the work required and benefits/drawbacks/requirements I decided to switch from Kratos to werther.
To bring some sanity to this I would first update to good internal and external URLs. The crux of the problem feels like you need to configure Ory Hydra (running inside the cluster) with an internet URL used in browsers etc, and this will be different to Ory Hydra's physical URL.
SIMILAR CURITY EXAMPLE
This feels like a similar setup to yours - it's worth taking a little time to understand resources:
Docker Compose
Tutorial Article
Authorization Server Configuration
Look at the base-url property at the top of the third link above, which is what internet clients such as browsers use to connect to the Authorization Server. There will be a property like this that you can set in Hydra.
I have a Docker stack for my mail server.
My docker-compose.xml contains
version: '3.7'
services:
postfix:
...
dovecot:
....
ports:
- "110:110"
- "995:995"
- "143:143"
- "993:993"
networks:
- mail
....
roundcube:
image: roundcube/roundcubemail
container_name: roundcube
environment:
- ROUNDCUBEMAIL_DEFAULT_HOST=dovecot
# - ROUNDCUBEMAIL_DEFAULT_PORT=993
networks:
- proxy
- mail
I also have a Nginx container running as a proxy for all my web applications. For roundcube I have
set $roundcube_upstream http://roundcube;
location /roundcube/ {
rewrite ^/roundcube/(.*) /$1 break;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_pass $roundcube_upstream;
}
With that configuration it's working. I can go to https://www.mydomain.be/rouncube/ and I can login. The default port is 143. So roundcube si connecting to dovecot with imap.
Now, I'd like to use port 993 and ssl/tls.
I tried decommenting the ROUNDCUBEMAIL_DEFAULT_PORT=993, but also using ssl://dovecot or tls://dovecot or ssl://mail.mydomain.be, ... but nothing is working.
When I click on the connextion button, after a while I receive an nginx error page. In my proxy logs I can see
2019/01/31 09:29:25 [error] 460#460: *82483 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 194.197.210.75, server: www.mydomain.be, request: "POST /roundcube/?_task=login HTTP/1.1", upstream: "http://172.18.0.9:80/?_task=login", host: "www.mydomain.be", referrer: "https://www.mydomain.be/roundcube/"
And I don't understand where the http://172.18.0.9:80/?_task=login is coming from ?
With Thunderbird client I can connect on that port.
What's the problem ?
Edit
Using
- ROUNDCUBEMAIL_DEFAULT_HOST=ssl://dovecot
- ROUNDCUBEMAIL_DEFAULT_PORT=993
I now have a response : connection error to the storage server.
In my roundcube logs :
errors: <1db522a3> IMAP Error: Login failed for me#mydomain.be from 172.18.0.8(X-Real-IP: ...,X-Forwarded-For: ...). Could not connect to ssl://dovecot:993: Unknown reason in /var/www/html/program/lib/Roundcube/rcube_imap.php on line 196 (POST /?_task=login&_action=login)172.18.0.8 - - [31/Jan/2019:13:57:37 +0100] "POST /?_task=login HTTP/1.1" 200 3089 "https://www.mydomain.be/roundcube/?_task=login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0"
and in dovecot logs
2019-01-31T13:57:38.002653+01:00 536ff3507263 dovecot: auth: Debug: auth client connected (pid=35),
2019-01-31T13:57:38.010096+01:00 536ff3507263 dovecot: imap-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=192.168.240.3, lip=192.168.240.2, TLS, session=<nVssksCAT7LAqPAD>
So dovecot is well contacted but ... ? Don't know whats the problem.
Your issue is that roundcube requires TLS or SSL certificates to be verified by default. Either copy the certificate from the mail server, use letsencrypt to validate your certificate or turn off peer verification in your roundcube configuration.