Docker Redis TLS authentication failure with .netcore app - docker

I am trying to use redis with tls with a netcore application and I get an authentication error
The Setup:
Docker:
I created a redis docker container using redis:6.2.0
docker-compose.yaml:
.
.
redis:
image: redis:6.2.0
command: redis-server /usr/local/etc/redis/redis.conf --appendonly yes
container_name: "cxm-redis"
ports:
- "6379:6379"
volumes:
- cxm-redis-data:/data
- C:/SaaS/certs/redis.conf:/usr/local/etc/redis/redis.conf
- C:/SaaS/certs/tests/tls/redis.crt:/usr/local/etc/redis/redis.crt
- C:/SaaS/certs/tests/tls/redis.key:/usr/local/etc/redis/redis.key
- C:/SaaS/certs/tests/tls/ca.crt:/usr/local/etc/redis/ca.crt
up to here all looks good, (as far as I can tell) I managed to authenticate using the following command
redis-cli --tls --cert ../usr/local/etc/redis/redis.crt --key /usr/local/etc/redis/redis.key --cacert /usr/local/etc/redis/ca.crt and I can succesfully ping and request keys.
I created the certificates with openssl and for the redis.conf i am using the redis.conf example from redis
The important bits:
### TLS
tls-port 6379
tls-cert-file /usr/local/etc/redis/redis.crt
tls-key-file /usr/local/etc/redis/redis.key
tls-ca-cert-file /usr/local/etc/redis/ca.crt
netcore:
For my .netcore application I am using the StackExchange library and for the TLS connection I followed the instructions here, like so
var options = new ConfigurationOptions
{
EndPoints = { "redis-test:6379" },
Password = "not-the-actual-password",
Ssl = true
};
options.CertificateSelection += delegate {
return new X509Certificate2("./redis_certificate.p12");
};
_db = ConnectionMultiplexer.Connect(options).GetDatabase();
the redis_certificate.p12 was generated using openssl with this command line
openssl pkcs12 -export -out sample_certificate.p12 -inkey redis.key -in redis.crt
The Issue:
When I make a request to redis from my app I get the following error:
It was not possible to connect to the redis server(s). There was an authentication failure; check that passwords (or client certificates) are configured correctly. AuthenticationFailure on redis-test:6379/Interactive, Initializing/NotStarted
in my apps logs, and I get the following in my redis logs:
Error accepting a client connection: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Error accepting a client connection: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca,
Error accepting a client connection: (null)
Are there any apparent mistakes in my setup I am failing to see? This is my first time trying this and maybe I am assuming too much or heading down the wrong way..
Trying to resolve this I found several questions with a similar issue but implementing their fixes did not resolve my issue..
a few of the things I tried
sending different ssl protocols from my .netcore app
sending the pfx/p12 certificate in different ways
several different redis configurations
Edit: I can provide as much code as needed!

For any one facing the same issue, it seems the server was using a non routed CA for the server certificates, the solution I found was to use the CertificateValidation callback of StackExchange.Redis library with the following code
private static bool CheckServerCertificate(object sender, X509Certificate certificate,
X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
if ((sslPolicyErrors & SslPolicyErrors.RemoteCertificateChainErrors) == SslPolicyErrors.RemoteCertificateChainErrors)
{
// check that the untrusted ca is in the chain
var ca = new X509Certificate2(_redisSettings.CertificatePath);
var caFound = chain.ChainElements
.Cast<X509ChainElement>()
.Any(x => x.Certificate.Thumbprint == ca.Thumbprint);
return caFound;
}
return false;
}
also an important part of the code being the condition
if((sslPolicyErrors & SslPolicyErrors.RemoteCertificateChainErrors) == SslPolicyErrors.RemoteCertificateChainErrors)

Related

Erlang :ssh authentication error. How to connect to ssh using identity file

I'm getting an authentication error when trying to connect ssh host.
The goal is to connect to the host using local forwarding. The command below is an example using drop bear ssh client to connect to host with local forwarding.
dbclient -N -i /opt/private-key-rsa.dropbear -L 2002:1.2.3.4:2006 -p 2002 -l
test_user 11.22.33.44
I have this code so far which returns empty connection
ip = "11.22.33.44"
user = "test_user"
port = 2002
ssh_config = [
user_interaction: false,
silently_accept_hosts: true,
user: String.to_charlist(user),
user_dir: String.to_charlist("/opt/")
]
# returns aunthentication error
{:ok, conn} = :ssh.connect(String.to_charlist(ip), port, ssh_config)
This is the error Im seeing
Server: 'SSH-2.0-OpenSSH_5.2'
Disconnects with code = 14 [RFC4253 11.1]: Unable to connect using the available authentication methods
State = {userauth,client}
Module = ssh_connection_handler, Line = 893.
Details:
User auth failed for: "test_user"
I'm a newbie to elixir and have been reading this erlang ssh document for 2 days. I did not find any examples in the documentation which makes it difficult to understand.
You are using non-default key name, private-key-rsa.dropbear. Erlang by default looks for this set of names:
From ssh module docs:
Optional: one or more User's private key(s) in case of publickey authorization. The default files are
id_dsa and id_dsa.pub
id_rsa and id_rsa.pub
id_ecdsa and id_ecdsa.pub`
To verify this is a reason, try renaming private-key-rsa.dropbear to id_rsa. If this works, the next step would be to add a key_cb callback to the ssh_config which should return the correct key file name.
One example implementation of a similar feature is labzero/ssh_client_key_api.
The solution was to convert dropbear key to ssh key. I have used this link as reference.
Here is the command to convert dropbear key to ssh key
/usr/lib/dropbear/dropbearconvert dropbear openssh /opt/private-key-rsa.dropbear /opt/id_rsa

Hyperledger Sawtooth - Preflight error while submitting transaction

I am trying to submit a transaction to Hyperledger Sawtooth v1.0.1 using javascript to a validator running on localhost. The code for the post request is as below:
request.post({
url: constants.API_URL + '/batches',
body: batchListBytes,
headers: { 'Content-Type': 'application/octet-stream' }
}, (err, response) => {
if (err) {
console.log(err);
return cb(err)
}
console.log(response.body);
return cb(null, response.body);
});
The transaction gets processed when submitted from an backend nodejs application, but it returns an OPTIONS http://localhost:8080/batches 405 (Method Not Allowed) error when submitted from client. These are the options that I have tried:
Inject Access-Control-Allow-* headers into the response using an extension: The response still gives the same error
Remove the custom header to bypass preflight request: This makes the validator throw an error as shown:
...
sawtooth-rest-api-default | KeyError: "Key not found: 'Content-Type'"
sawtooth-rest-api-default | [2018-03-15 08:07:37.670 ERROR web_protocol] Error handling request
sawtooth-rest-api-default | Traceback (most recent call last):
...
The unmodified POST request from the browser gets the following response headers from the validator:
HTTP/1.1 405 Method Not Allowed
Content-Type: text/plain; charset=utf-8
Allow: GET,HEAD,POST
Content-Length: 23
Date: Thu, 15 Mar 2018 08:42:01 GMT
Server: Python/3.5 aiohttp/2.3.2
So, I guess OPTIONS method is not handled in the validator. A GET request for the state goes through fine when the CORS headers are added. This issue was also not faced in Sawtooth v0.8.
I am using docker to start the validator, and the commands to start it are a slightly modified version of those given in the LinuxFoundationX: LFS171x course. The relevant commands are below:
bash -c \"\
sawadm keygen && \
sawtooth keygen my_key && \
sawset genesis -k /root/.sawtooth/keys/my_key.priv && \
sawadm genesis config-genesis.batch && \
sawtooth-validator -vv \
--endpoint tcp://validator:8800 \
--bind component:tcp://eth0:4004 \
--bind network:tcp://eth0:8800
Can someone please guide me as to how to solve this problem?
CORS issues are always the best.
What is CORS?
Your browser trying to protect users from bring directed to a page they think is the frontend for an API, but is actually fraudulent. Anytime a web page tries to access an API on a different domain, that API will need to explicitly give the webpage permission, or the browser will block the request. This is why you can query the API from Node.js (no browser), and can put the REST API address directly into your address bar (same domain). However, trying to go from localhost:3000 to localhost:8008 or from file://path/to/your/index.html to localhost:8008 is going to get blocked.
Why doesn't the Sawtooth REST API handle OPTIONS requests?
The Sawtooth REST API does not know the domain you are going to run your web page from, so it can't whitelist it explicitly. It is possible to whitelist all domains, but this obviously destroys any protection CORS might give you. Rather than try to weigh the costs and benefits of this approach for all Sawtooth users everywhere, the decision was made to make the REST API as lightweight and security agnostic as possible. Any developer using it would be expected to put it behind a proxy server, and they can make whatever security decisions they need on that proxy layer.
So how do you fix it?
You need to setup a proxy server that will put the REST API and your web page on the same domain. There is no quick configuration option for this. You will have to set up an actual server. Obviously there are lots of ways to do this. If you are already familiar with Node, you could serve the page from Node.js, and then have the Node server proxy the API calls. If you are already running all of the Sawtooth components with docker-compose though, it might be easier to use Docker and Apache.
Setting up an Apache Proxy with Docker
Create your Dockerfile
In the same directory as your web app create a text file called "Dockerfile" (no extension). Then make it look like this:
FROM httpd:2.4
RUN echo "\
LoadModule proxy_module modules/mod_proxy.so\n\
LoadModule proxy_http_module modules/mod_proxy_http.so\n\
ProxyPass /api http://rest-api:8008\n\
ProxyPassReverse /api http://rest-api:8008\n\
RequestHeader set X-Forwarded-Path \"/api\"\n\
" >>/usr/local/apache2/conf/httpd.conf
This is going to do a couple of things. First it will pull down the httpd module from DockerHub, which is just a simple static server. Then we are using a bit of bash to add five lines to Apache's configuration file. These five lines import the proxy modules, tell Apache that we want to proxy http://rest-api:8008 to the /api route, and set the X-Forwarded-Path header so the REST API can properly build response URLs. Make sure that rest-api matches the actual name of the Sawtooth REST API service in your docker compose file.
Modify your docker compose file
Now, to the docker compose YAML file you are running Sawtooth through, you want to add a new property under the services key:
services:
my-web-page:
build: ./path/to/web/dir/
image: my-web-page
container_name: my-web-page
volumes:
- ./path/to/web/dir/public/:/usr/local/apache2/htdocs/
expose:
- 80
ports:
- '8000:80'
depends_on:
- rest-api
This will build your Dockerfile located at ./path/to/web/dir/Dockerfile (relative to the docker compose file), and run it with its default command, which is to start up Apache. Apache will serve whatever files are located in /usr/local/apache2/htdocs/, so we'll use volumes to link the path to your web files on your host machine (i.e. ./path/to/web/dir/public/), to that directory in the container. This is basically an alias, so if you update your web app later, you don't need to restart this docker container to see the changes. Finally, ports will take the server, which is at port 80 inside the container, and forward it out to localhost:8000.
Running it all
Now you should be able to run:
docker-compose -f path/to/your/compose-file.yaml up
And it will start up your Apache server along with the Sawtooth REST API and validator and any other services you defined. If you go to http://localhost:8000, you should see your web page, and if you go to http://localhost:8000/api/blocks, you should see a JSON representation of the blocks on chain. More importantly you should be able to make the request from your web app:
request.post({
url: 'api/batches',
body: batchListBytes,
headers: { 'Content-Type': 'application/octet-stream' }
}, (err, response) => console.log(response) );
Whew. Sorry for the long response, but I'm not sure if it is possible to solve CORS any faster. Hopefully this helps.
The transaction Header should have details like, address of the block where it would be save. Here is example which I have used and is working fine for me :
String payload = "create,0001,BLockchain CPU,Black,5000";
logger.info("Sending payload as - "+ payload);
String payloadBytes = Utils.hash512(payload.getBytes()); // --fix for invaluid payload seriqalization
ByteString payloadByteString = ByteString.copyFrom(payload.getBytes());
String address = getAddress(IDEM, ITEM_ID); // get unique address for input and output
logger.info("Sending address as - "+ address);
TransactionHeader txnHeader = TransactionHeader.newBuilder().clearBatcherPublicKey()
.setBatcherPublicKey(publicKeyHex)
.setFamilyName(IDEM) // Idem Family
.setFamilyVersion(VER)
.addInputs(address)
.setNonce("1")
.addOutputs(address)
.setPayloadSha512(payloadBytes)
.setSignerPublicKey(publicKeyHex)
.build();
ByteString txnHeaderBytes = txnHeader.toByteString();
byte[] txnHeaderSignature = privateKey.signMessage(txnHeaderBytes.toString()).getBytes();
String value = Signing.sign(privateKey, txnHeader.toByteArray());
Transaction txn = Transaction.newBuilder().setHeader(txnHeaderBytes).setPayload(payloadByteString)
.setHeaderSignature(value).build();
BatchHeader batchHeader = BatchHeader.newBuilder().clearSignerPublicKey().setSignerPublicKey(publicKeyHex)
.addTransactionIds(txn.getHeaderSignature()).build();
ByteString batchHeaderBytes = batchHeader.toByteString();
byte[] batchHeaderSignature = privateKey.signMessage(batchHeaderBytes.toString()).getBytes();
String value_batch = Signing.sign(privateKey, batchHeader.toByteArray());
Batch batch = Batch.newBuilder()
.setHeader(batchHeaderBytes)
.setHeaderSignature(value_batch)
.setTrace(true)
.addTransactions(txn)
.build();
BatchList batchList = BatchList.newBuilder()
.addBatches(batch)
.build();
ByteString batchBytes = batchList.toByteString();
String serverResponse = Unirest.post("http://localhost:8008/batches")
.header("Content-Type", "application/octet-stream")
.body(batchBytes.toByteArray())
.asString()
.getBody();

Windows Etsy: Peer certificate cannot be authenticated with given CA certificates

In an effort to be OAuth'd with Etsy, I have tried countless solutions in C# to at least start the authentication process (ie get the login URL):
eg
mashery.com, http://term.ie/oauth/example/client.php and question #8321034
but the response is always the same:
oauth_problem=signature_invalid&debug_sbs=GET&https%3A%2F%2Fopenapi.etsy.com%2Fv2%2Foauth%2Frequest_token&oauth_consumer_key%3D...my-consumer-key...%26oauth_nonce%3D2de91e1361d1906bbae04b15f42ab38d%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1502362164%26oauth_version%3D1.0%26scope%3Dlistings_w%2520listings_r
and so I'm resorting to the dreaded world of PHP...
On my machine, I've installed the following (Windows 10):
XAMPP (xampp-win32-7.1.7-0-VC14-installer) with default options
JDK (jdk-8u144-windows-i586)
JRE (jre-8u144-windows-i586)
php_oauth.dll ([php_oauth-2.0.2-7.1-ts-vc14-x86.zip][4]) and copying it to C:\xampp\php\ext
[cacert.pem][4], (dated Jun 7 03:12:05 2017) and coping it to the following directories:
C:\xampp\perl\vendor\lib\Mozilla\CA
C:\xampp\phpMyAdmin\vendor\guzzle\guzzle\src\Guzzle\Http\Resources
Apache and Tomcat would not run to begin with from XAMPP because it said that ports 443 and 80 were being used/blocked and so I duly changed these to 444 and 122 in
C:\xampp\apache\conf\extra\httpd-ssl.conf
C:\xampp\apache\conf\httpd.conf
All good so far but when I run the following script in my browser (http://localhost:444/dashboard/etsy.php):
<?php
$base_uri = 'https://openapi.etsy.com';
$api_key = 'my-etsy-api-key';
$secret = 'my-etsy-api-secret';
$oauth = new OAuth($api_key, $secret, OAUTH_SIG_METHOD_HMACSHA1, OAUTH_AUTH_TYPE_URI);
$req_token = $oauth->getRequestToken($base_uri .= "/v2/oauth/request_token?scope=listings_w%20transactions_r", 'oob');
$login_url = $req_token['login_url'];
print "Please log in and allow access: $login_url \n\n";
$verifier = readline("Please enter verifier: ");
$verifier = trim($verifier);
$oauth->setToken($req_token['oauth_token'], $req_token['oauth_token_secret']);
$acc_token = $oauth->getAccessToken($base_uri .= "/v2/oauth/access_token", null, $verifier);
$oauth_token = $acc_token['oauth_token'];
$oauth_token_secret = $acc_token['oauth_token_secret'];
$oauth->setToken($oauth_token, $oauth_token_secret);
print "Token: $oauth_token \n\n";
print "Secret: $oauth_token_secret \n\n";
?>
I get the following error message:
Fatal error: Uncaught OAuthException: making the request failed (Peer
certificate cannot be authenticated with given CA certificates) in
C:\xampp\htdocs\dashboard\etsy.php:8 Stack trace: #0
C:\xampp\htdocs\dashboard\etsy.php(8):
OAuth->getRequestToken('https://openapi...', 'oob') #1 {main} thrown
in C:\xampp\htdocs\dashboard\etsy.php on line 8
I've tried running the script with each thread safe, x86 version of OAuth (http://windows.php.net/downloads/pecl/releases) - stop, restart Apache) but no luck.
I'm at my wits end.
How to I resolve this Peer certificate problem?
Simply disable the SSL on local.
$oauth->disableSSLChecks()
Oauth by default using CURL SSL Certificate. The simple way for local apache server is to disable it. Either configure the SSL for the CURL. It will also resolve the issue for oauth.
as per php documentation
we can set the certificate path simply.
$oauth->setCAPath("F:\xampp\php\extras\ssl\cacert.pem");
print_r($oauth->getCAPath());
You can also set the request engine to curl or php stream if the ssl is already configured.
Official PHP documentation

Ruby - Connect with SSL and authenticate via client certificate - sslv3 alert bad certificate

I'm trying to connect a webcrawler that accesses a certain site via SSL and queries my data on that site. The authentication of this site is via a self-signed Digital Certificate. At the moment I want to access the site, I upload this certificate in .pfx format to my api, convert it to .pem, and when I try to access the site with this certificate, the response comes with status 403 (forbidden ).
However, when I try to access the site through a browser with the certificate in .pfx format I usually get it.
I already tried using Mechanize, and it worked for a while (until a few months ago it worked), but then it started to give the error:
SSL_connect returned = 1 errno = 0 state = SSLv3 read finished A: sslv3 alert bad certificate
The site is old, it does not receive updates frequently.
After that I already tried to use the net / http lib and the error persisted, I tried to use the httprb gem and lastly I tried with Faraday. All attempts ended either in that error quoted above or with the response status == 403.
What can I do to be able to connect? Is there something wrong with my script? Is it missing any information I need to get through?
Code:
# Faraday customs method:
class FaradayHttp
def with_openssl
system "openssl pkcs12 -in my-certificate-path -out certificate-output-path -nodes -password pass:certificate-password"
def cert_object
OpenSSL::X509::Certificate.new File.read("certificate-output-path")
end
# create PKey
def key_object
OpenSSL::PKey.read File.read("certificate-output-path")
end
faraday = Faraday::Connection.new 'https://example-site.com',
:ssl => {
certificate: cert_object,
private_key: key_object,
version: :SSLv3,
verify: false
}
faraday
end
end
# Controller that try to connect with the ssl server:
agent = FaradayHttp.new.with_openssl
page = agent.get '/login_path'
# mypki will prompt you for certificates
require 'mypki'
# faraday will use certificates from mypki
require 'faraday'
faraday = Faraday::Connection.new 'https://example-site.com'
faraday.get '/login_path'

MQTT:SSL routines:SSL3_GET_RECORD:wrong version number

I am trying to use SSL with MQTT and gets following error,
1379677998: Client connection from 127.0.0.1 failed: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number.
1379678058: OpenSSL Error: error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure
1379678058: Socket read error on client (null), disconnecting.
My conf file has seetings as below,
port 8883
cafile /home/administrator/Downloads/mosquitto-1.2.1_dilip/test/ssl/test-root-ca.crt (also tried all-ca.crt here)
certfile /home/administrator/Downloads/mosquitto-1.2.1_dilip/test/ssl/server.crt
keyfile /home/administrator/Downloads/mosquitto-1.2.1_dilip/test/ssl/server.key
tls_version tlsv1
require_certificate true (tried commenting this too)
and I run, 08-ssl-connect-cert-auth.test from test folder. code snippet included below
mosquitto_tls_opts_set(mosq, 1, "tlsv1", NULL);
mosquitto_tls_set(mosq, "../ssl/test-root-ca.crt", "../ssl/certs", "../ssl/client.crt", "../ssl/client.key", NULL);
mosquitto_connect_callback_set(mosq, on_connect);
mosquitto_disconnect_callback_set(mosq, on_disconnect);
rc = mosquitto_connect(mosq, "localhost", 8883, 60);
Environment info:
Python 2.7.3
OpenSSL 1.0.1 14 Mar 2012
Description: Ubuntu 12.04.2 LTS
Release: 12.0
mosquitto-1.2.1/mosquitto-1.2
Please let me know what could go wrong here? I also tried generating certificates using gen.sh.
Thanks in advance.
-Dilip
On a general note - you should be generating your own certificates, don't use the ones provided by mosquitto for testing, or use the gen.sh script without modifying it for your own details.
To use the certificates provided, you should be using all-ca.crt in the broker. It contains the example root CA and intermediary CA certificates. The client only requires the root CA for verification purposes.
You are using a full path to the certificates in the broker configuration, but relative paths in the client. Are you sure that they are both correct?
Have you tried running the tests unmodified with make test? This would tell you if there is a problem with your openssl installation for example.

Resources