How to display the entire line with grep matching the pattern?
example.log:
8.8.8.8 [24/May/2020:10:10:10] TLSv1.2 302
8.8.8.8 [24/May/2020:10:10:11] TLSv1.3 200
8.8.8.8 [24/May/2020:10:10:12] TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384
8.8.8.8 [24/May/2020:10:10:12] TLSv1.1 200
8.8.8.8 [24/May/2020:10:10:13] TLSv1 200
8.8.8.8 [24/May/2020:10:10:13] TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384
8.8.8.8 [24/May/2020:10:10:14] TLSv1 ECDHE-RSA-AES256-GCM-SHA384
How to display the entire line that has the TLSv1 pattern only? If I use grep -e 'TLSv1 it will display the other lines as well.
Like this:
grep 'TLSv1 '
Note the space in the regex
or
grep 'TLSv1\s'
or using awk:
awk '$3=="TLSv1"'
Related
We have a requirement to have a globalized cache machine i.e. Redis to be running on one server and the docker services running on other servers need to be able to access it. So we have created a docker container in Ec2 instance A and all the other services are in Instance B & C. We have installed redis-cli IP on Instnace A & Instance B and it's working. But we have a requirement to have DNS instead of IP here. So we have configured the records in Bastian host in the below format.
server {
listen 80 ;
server_name devtest-redis.xyz.com;
return 301 https://$server_name$request_uri ;
}
server {
listen 443;
server_name devtest-redis.xyz.com;
ssl on;
ssl_certificate /etc/letsencrypt/live/xyz.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/xyz.com/privkey.pem;
ssl_session_timeout 5m;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
ssl_prefer_server_ciphers on;
add_header 'Access-Control-Allow-Origin' '*';
location / {
proxy_pass http://10.63.3.10:6379;
}
}
But when we type redis-cli -h devtest-redis.xyz.com, I'm getting the below error. Even if I give redis-cli -h IP:PORT , I get the same error. I'm missing something here or how can this be resolved?
Could not connect to Redis at https://devtest-redis.xyz.com:6379: Name or service not known
I am trying to use NGINX as a proxy with a next.js frontend and FastAPI backend, each running in their own container.
I got everything working fine with HTTP, but having some issues getting things to work with HTTPS.
All containers start running without any issues, and things seem to be working, but when I try to communicate with the proxy, I get the following errors:
From host:
lafton#lafton-platform:~$ curl localhost -L
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:443
Form inside NGINX container using localhost:
root#6016e75698cf:/# curl localhost -L
curl: (60) SSL: no alternative certificate subject name matches target host name 'localhost'
More details here: https://curl.haxx.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
From inside NGINX container using lafton.io:
root#6016e75698cf:/# curl https://lafton.io
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to lafton.io:443
I tried to install NGINX locally instead of inside Docker and it works as expected. I tried to enable the SSL configuration which is commented out in the default configuration, and it worked perfectly with SSL locally.
I then tried to use the default SSL configuration with my setup, but it does not work.
This is the NGINX config I am running inside /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
load_module /etc/nginx/modules/ngx_http_js_module.so;
events {
worker_connections 1024;
}
http {
server {
listen 80;
server_name lafton.io;
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 443 ssl;
server_name lafton.io;
ssl_certificate /etc/certs/fullchain1.pem;
ssl_certificate_key /etc/certs/privkey1.pem;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK;
ssl_prefer_server_ciphers on;
location / {
proxy_pass http://lafton-website:3000;
}
location /api/albums {
proxy_pass http://lafton-albums:8000;
}
}
}
The port 80 part is just a redirect to https. It is the exact same without it.
The ciphers is from Mozillas recommendations. I tried to change this from the default as some of the troubleshooting I did seemed to indicate no matching ciphers.
I am really lost here and not sure where to look for further troubleshooting. Any help would be really appreciated!
Timo Stark's comment solved the issue.
It didn't work inside the container because the certificates CN was lafton.io, so I had to use the -k flag in the curl command.
When that worked, I saw a typo in my docker-compose file, so the container had exposed port 433, not 443.
Ok. So I am trying to enable HTTPS with Nginx using Docker container.
My nginx.conf now looks like this:
server {
listen 443 ssl;
server_name localhost;
ssl_certificate ssl/domain.crt;
ssl_certificate_key ssl/domain.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
Btw I have a working version with HTTP, which looks like this:
server {
listen 80;
server_name localhost;
When I am starting the docker container, I get an error saying: "[emerg] 1#1: cannot load certificate "/etc/nginx/ssl/domain.crt":"
I have created the encrypting with openssl, and it is put in the "ssl"-folder where the rest of my project is. But the problem seems to be here? Does anybody have a solution for this?
I am able to successfully get a NGINX reverse proxy running locally with SSL enabled. However, once I try to get it running on a Docker container, I get issues with SSL.
I portforwarded to port 9443 using docker run -it nginxProxy -v local_path_to_certs:container_path_to_certs -p 9443:443.
My Dockerfile:
FROM nginx:alpine
# Expost port 443
EXPOSE 443
RUN rm -v /etc/nginx/nginx.conf
RUN rm -rf /etc/nginx/conf.d
# Copy custom configuration file from the current directory
COPY nginx.conf /etc/nginx/nginx.conf
ENTRYPOINT [ "nginx", "-c", "/etc/nginx/nginx.conf", "-g", "daemon off;"]
My nginx.conf:
events {
worker_connections 1024;
}
http {
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4:!SHA:!SHA1:!SSLv2:#STRENGTH;
ssl_prefer_server_ciphers on;
ssl_certificate /path_to_cert/public.cert.pem;
ssl_certificate_key /path_to_private_key/private.key.pem;
ssl_password_file /path_to_password_file/password.txt;
server {
listen [::]:443 ssl;
server_name _;
location / {
proxy_pass https://example.com;
}
}
}
On my local machine, this works as fine as expected, but when running on a Docker container running curl https://localhost:9443 -vvv gives me
* Rebuilt URL to: https://localhost:9443/
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:#STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:9443
* Closing connection 0
curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:9443
I will asume that you have solved this by now. Either way Im gonna leave the solution i've found searching for the same problem.
I had this similar error and user mljrg find that it was happening when hostnames were being resolved to IPv6 addresses. He discover this by comparing the outputs in two machines, one was resolving to IPv4 and it worked there, the other was resolving to IPv6 and it failed there.
So the solution was to run networksetup -setv6off Wi-Fi on macOS High Sierra 10.13.6.
Maybe this helps a few others like me.
I'm trying to change the settings for Nginx ssl_protocols, but the changes don't reflect on the server.
At first I thought it was because we were using Ubuntu 12.04, but now we're updated to 14.04.
Nginx version:
nginx version: nginx/1.10.1
built by gcc 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --sbin-path=/usr/local/sbin/nginx --with-http_ssl_module --with-http_stub_status_module --with-http_gzip_static_module
Openssl version:
OpenSSL 1.0.1f 6 Jan 2014
Ngnix.conf:
http {
include /usr/local/nginx/conf/mime.types;
default_type application/octet-stream;
tcp_nopush on;
keepalive_timeout 65;
tcp_nodelay off;
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
error_log /var/log/nginx/error.log debug;
open_file_cache max=1000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
client_body_timeout 10;
client_header_timeout 10;
sendfile on;
# output compression
gzip on;
gzip_min_length 1100;
gzip_buffers 4 8k;
gzip_proxied any;
gzip_types text/plain text/html text/css text/js application/x-javascript application/javascript application/json;
# include config for each site here
include /etc/nginx/sites/*;
/etc/nginx/sites/site.conf:
server {
listen 443 ssl;
server_name server_name;
root /home/deploy/server_name/current/public;
access_log /var/log/nginx/server_name.access.log main;
ssl_certificate /usr/local/nginx/conf/ssl/wildcard.server_name.com.crt;
ssl_certificate_key /usr/local/nginx/conf/ssl/wildcard.server_name.com.key.unsecure;
ssl_client_certificate /usr/local/nginx/conf/ssl/geotrust.crt;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:DHE-RSA-AES256-SHA;
ssl_prefer_server_ciphers on;
location ~ ^/assets/ {
expires max;
add_header Cache-Control public;
add_header ETag "";
break;
}
location / {
try_files $uri #server_name;
proxy_set_header X-Forwarded-Proto https;
}
location #server_name {
include proxy.conf;
proxy_pass http://server_name;
proxy_set_header X-Forwarded-Proto https;
}
# stats url
location /nginx_stats {
stub_status on;
access_log off;
}
}
The config files get loaded properly and are both being used as intended. If it has any relevance the server is running Ruby on Rails with Unicorn.
Does anyone have an idea what could be wrong?
Description
I had a similar problem. My changes would be applied (nginx -t would warn about duplicate and invalid values), but TLSv1.0 and TLSv1.1 would still be accepted. My line in my sites-enabled/ file reads
ssl_protocols TLSv1.2 TLSv1.3;.
I ran grep -R 'protocol' /etc/nginx/* to find other mentions ssl_protocols, but I only found the main configuration file /etc/nginx/nginx.conf and my own site config.
Underlying problem
The problem was caused by a file included by certbot/letsencrypt, at /etc/letsencrypt/options-ssl-nginx.conf. In certbot 0.31.0 (certbot --version) the file includes this line:
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
This somewhat sneakily enabled these versions of TLS.
I was tipped off by Libre Software.
0.31.0 is the most up-to-date version I was able to get for Ubuntu 18.04 LTS
Solution
TLS versions <1.2 were disabled by default in the certbot nginx config starting from certbot v0.37.0 (thank you mnordhoff). I copied the file from there into the letsencrypt config (options-ssl-nginx.conf), added a note to myself and subsequent maintainers and everything was all right with the world again.
How to not get into this mess in the first place
grepping one level higher (/etc/* instead of /etc/nginx*) would have allowed me to find the culprit. But a more reliable and powerful tool is nginx -T, which prints out all the configuration files that are considered.
Other useful commands:
nginx -s reload after you change configs
nginx -v to find out your nginx version. To enable TSL version 1.3, you need version 1.13.0+.
openssl version: you need at least OpenSSL 1.1.1 "built with TLSv1.3 support"
curl -I -v --tlsv<major.minor> <your_site> for testing whether a certain version of TLS is in fact enabled
journalctl -u nginx --since "10 minutes ago" to make absolutely sure something else isn't going on.
Want to add another (somewhat obscure) possibility since the CERTbot one didn't cover me. Mine is only for NGINX installs with multiple domains. Basically the info you specify for your specific domain may be modified because of the server default. That default is set from the first domain name encountered when reading the config (basically alphabetically). Details on this page.
http://nginx.org/en/docs/http/configuring_https_servers.html
A common issue arises when configuring two or more HTTPS servers listening on a single IP address:
server {
listen 443 ssl;
server_name www.example.com;
ssl_certificate www.example.com.crt;
...
}
server {
listen 443 ssl;
server_name www.example.org;
ssl_certificate www.example.org.crt;
...
}
With this configuration a browser receives the default server’s certificate, i.e. www.example.com regardless of the requested server name. This is caused by SSL protocol behaviour. The SSL connection is established before the browser sends an HTTP request and nginx does not know the name of the requested server. Therefore, it may only offer the default server’s certificate.
The issue wasn't in the server itself, but instead in the AWS Load Balancer having wrong SSL Ciphers selected.