I'm having plenty of trouble trying to use a certificate that has a password on ruby. I'm using rest-client, but that's not a requirement.
This is the cURL-equivalent of the call I need to make:
curl -E certificate.pem:PASSWORD -d ident=language -d data="test" "https://theurl"
I tried many things, but I can't get the password part working. Here's what I have so far:
cert = OpenSSL::X509::Certificate.new(File.read("#{RAILS_ROOT}/certificate.pem"))
reply = RestClient.post("https://theurl", {:ident => 'language', :data => 'test'}, {:ssl_client_cert => cert})
I've tried putting the password everywhere, as :password and :ssl_client_key, I've looked through all documentation I could find but there's nowhere that will accept this password.
This is the error I always get:
SSL_connect returned=1 errno=0 state=SSLv3 read finished A: sslv3 alert handshake failure
What am I missing?
The way you uses curl with option -E, you are specifying a private key with a certificate.
(from the cURL man page)
-E/--cert
(SSL) Tells curl to use the specified
client certificate file when getting a
file with HTTPS, FTPS or another
SSL-based protocol. The certificate
must be in PEM format. If the optional
password isn't specified, it will be
queried for on the terminal. Note that
this option assumes a "certificate"
file that is the private key and the
private certificate concatenated! See
--cert and --key to specify them independently.
So in order to do the samething with RestClient, you can try using the ssl_client_key option. Like:
:ssl_client_key => OpenSSL::PKey::RSA.new(File.read("key.pem"), "passphrase, if any"),
Related
I'm trying to connect a webcrawler that accesses a certain site via SSL and queries my data on that site. The authentication of this site is via a self-signed Digital Certificate. At the moment I want to access the site, I upload this certificate in .pfx format to my api, convert it to .pem, and when I try to access the site with this certificate, the response comes with status 403 (forbidden ).
However, when I try to access the site through a browser with the certificate in .pfx format I usually get it.
I already tried using Mechanize, and it worked for a while (until a few months ago it worked), but then it started to give the error:
SSL_connect returned = 1 errno = 0 state = SSLv3 read finished A: sslv3 alert bad certificate
The site is old, it does not receive updates frequently.
After that I already tried to use the net / http lib and the error persisted, I tried to use the httprb gem and lastly I tried with Faraday. All attempts ended either in that error quoted above or with the response status == 403.
What can I do to be able to connect? Is there something wrong with my script? Is it missing any information I need to get through?
Code:
# Faraday customs method:
class FaradayHttp
def with_openssl
system "openssl pkcs12 -in my-certificate-path -out certificate-output-path -nodes -password pass:certificate-password"
def cert_object
OpenSSL::X509::Certificate.new File.read("certificate-output-path")
end
# create PKey
def key_object
OpenSSL::PKey.read File.read("certificate-output-path")
end
faraday = Faraday::Connection.new 'https://example-site.com',
:ssl => {
certificate: cert_object,
private_key: key_object,
version: :SSLv3,
verify: false
}
faraday
end
end
# Controller that try to connect with the ssl server:
agent = FaradayHttp.new.with_openssl
page = agent.get '/login_path'
# mypki will prompt you for certificates
require 'mypki'
# faraday will use certificates from mypki
require 'faraday'
faraday = Faraday::Connection.new 'https://example-site.com'
faraday.get '/login_path'
I am connecting to an external API using Ruby SSL two way authentication.
My latest script is here:
namespace :rnif_message do
# With Proxy
task send_test_index: :environment do
our_cert = File.read(File.join(Rails.root, 'ssl', 'invoice', 'test', 'cert20190116_ourcert.der'))
their_test_cert = File.read(File.join(Rails.root, 'ssl', 'invoice', 'test', 'testcert2016_theircert.der'))
cert_store = OpenSSL::X509::Store.new
# Contains their intermediate CA files
cert_store.add_path File.join(Rails.root, 'ssl', 'invoice', 'test', 'ca')
cert_store.add_cert OpenSSL::X509::Certificate.new(their_test_cert)
uri = URI("https://xml.digital.com/wm.rn/receive")
proxy_host = "us-static-02.qg.com"
proxy_port = "port"
proxy_user = "user"
proxy_pass = "pass"
proxy_request = Net::HTTP.new(uri.hostname, '443', proxy_host, proxy_port, proxy_user, proxy_pass)
proxy_request.verify_mode = OpenSSL::SSL::VERIFY_PEER
proxy_request.use_ssl = true
proxy_request.ssl_version = :TLSv1_2
proxy_request.ciphers = ["AES256-SHA:AES128-SHA:DES-CBC3-SHA"]
proxy_request.cert = OpenSSL::X509::Certificate.new(our_cert)
proxy_request.cert_store = cert_store
post_request = Net::HTTP::Post.new(uri)
response = proxy_request.request(post_request)
end
Response back (since I updated the ciphers) is now
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=unknown state
Instead of the older from my two previous questions
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A
# /Users/me/projects/proj/lib/tasks/rnif_message_builder.rake:217:in `block (2 levels) in <top (required)>'
Here is my latest wireshark
In the initial configuration of my certificate and IP on THEIR server configuration, I may have given them the wrong IP address, so I may be getting blocked by their firewall. Is there ways using openssl s_client I can test this?
So far i've been trying
openssl s_client -showcerts -connect xml.digitaloilfield.com:https
But I am not very familiar with using openssl s_client
Any help on troubleshooting this would be greatly appreciated!
Update
Thanks you very much for your help so far. I am experimenting with those commands you sent me and trying to see what info I can get from them to help me with this. Currently, after they changed my IP address and allowed me through the firewall, I am getting this
EOFError: end of file reached /Users/me/projects/xtiri/xtiri.com/lib/tasks/rnif_message_builder.rake:219:in `block (2 levels) in <top (required)>'
This will usually connect to nearly all servers. It uses TLS 1.2 and SNI. That should establish the TCP connection and start the TLS handshake. The handshake may fail later, but that's a different problem.
$ openssl s_client -connect xml.digitaloilfield.com:443 -tls1_2 \
-servername xml.digitaloilfield.com -debug
<hang>
connect: Connection timed out
connect:errno=110
However, while s_client is hanging, jump over to another terminal and issue:
$ sudo netstat -a | grep openssl
$
Netstat does not show you the SYN_SEND state, so use tcptrack:
$ sudo tcptrack -i eth0
# <next, use s_client>
172.16.2.4:43302 208.38.22.37:443 SYN_SENT 15s 0 B/s
You are in TCP's wait timer. The other side did not perform the three-way handshake with you. In fact, they did not acknowledge your SYN. There could be a few reasons for it, but ...
Given the target, it looks like you encountered a firewall. Rather than Reject'ing connections, it is Drop'ing connections. Its sometimes called "Stealth Mode"; it makes it appear there's no server running on the machine. That's consistent with OpenSSL's connect: Connection timed out message.
The problem could be with the proxy. You really want to run the tests from there, but you probably won't be able to. It could be you are using the ciphers, protocols and ports as specified by the remote site; but the proxy is doing its own thing. Also see Jarmock's SSL Interception Proxies and Transitive Trust.
Here are a couple of references:
How can I monitor network connections for an app
Is it better to set -j REJECT or -j DROP in iptables?
TCP 3-way handshake on the Wireshark Wiki
I've had this issue for about 2 weeks, when I suddenly stopped sending notifications in production. I am using the django-push-notifications library and by django admin I can send a test message, but it does not send messages through the system.
On my local computer, everything works flawlessly. I discovered a command to test the certificate:
openssl s_client -connect gateway.push.apple.com:2195 -cert apns-cert.pem
With this one I had the return: Timeout: 7200 (sec) Verify return
code: 20 (unable to get local issuer certificate) Extended master
secret: yes
So with a lot of research, I discovered that I needed to put the path of "CA":
openssl s_client -CApath /etc/ssl/certs/ -connect gateway.push.apple.com:2195 -cert apns-cert.pem
Who was taking me to: Verify return code: 0 (ok)
However, for use in the library, I needed to put the full path of a .pem file. Then I found this command:
ls /etc/ssl/certs/Entrust*
I tested all the .pem files that were there, until I reached what appeared to have worked perfectly:
openssl s_client -CAfile /etc/ssl/certs/Entrust.net_Premium_2048_Secure_Server_CA.pem -connect gateway.push.apple.com:2195 -cert apns-cert.pem
Soon, I formatted my PUSH_NOTIFICATIONS_SETTINGS:
PUSH_NOTIFICATIONS_SETTINGS = {
"GCM_API_KEY": "xxxx",
"APNS_CERTIFICATE": os.path.join(BASE_DIR, "apns-cert.pem"),
"APNS_CA_CERTIFICATES": "/etc/ssl/certs/Entrust.net_Premium_2048_Secure_Server_CA.pem",
"APNS_ERROR_TIMEOUT": 3,
}
IOS_VERIFY_RECEIPT_API = 'https://buy.itunes.apple.com/verifyReceipt'
ANDROID_VERIFY_RECEIPT_API = 'https://www.googleapis.com/androidpublisher/v2/applications/{packageName}/purchases/subscriptions/{subscriptionId}/tokens/{token}'
Unfortunately it still does not send PUSH, and no error because I have configured it to pop errors to send by email.
PS: Remembering that by sending a test text via django admin: OK. Sending via sandbox (debug): OK.
In fact it was not an SSL issue, it was a bulk upload error by the library.
The tokens registered in the system were expired and the library does not know how to work with it and canceled the action, causing no other token to be attempted. I corrected the problem by looping and ignoring the individual error by sending a test to my email:
def send_push(self):
errors = []
# IOS
queryset_ios = APNSDevice.objects.filter(user=self.authentication)
for device in queryset_ios:
try:
device.send_message(self.subject, badge=1, sound=self.kind.sound)
except APNSServerError as e:
errors.append(APNS_ERROR_MESSAGES[e.status])
except Exception:
pass
# ANDROID
queryset_android = GCMDevice.objects.filter(user=self.authentication)
extra = {'notification': self.pk, 'kind': self.kind.kind, 'sound': self.kind.sound}
for device in queryset_android:
try:
queryset_android.send_message(self.subject, badge=1, extra=extra)
except GCMError as e:
errors.append(str(e))
except Exception:
pass
if errors:
send_mail("Push Error",
"Push: %s \n User: %s \n\n Errors: %s" % (self.subject, self.authentication.full_name, errors),
settings.DEFAULT_FROM_EMAIL, ["my#mail.com"])
I am using https://github.com/ruby-ldap/ruby-net-ldap (net-ldap) gem to verify the authenticity of a user in my rails app. But before passing data to the ldap server, I need to verify that I am talking with the same secure server.
Is there a workaround which allows me to verify the certificate in ruby
Additional details: (things I have tried)
The certificate which is passed on to me is same as the one I see when I run
openssl s_client -showcerts -connect "<host>:<port>" </dev/null 2>/dev/null|openssl x509 -outform PEM
I used http://www.ldapsoft.com/ to connect to client's server
Unless I add the certificate file given to me in Security > Manage server certificates, I get a warning saying unknown security certificate
I tried do it manually first in plain ruby (without gem)
But i get following error
test-ssl.rb:23:in `connect': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (OpenSSL::SSL::SSLError)
Code:
cert_store = OpenSSL::X509::Store.new
cert_store.add_file "server-wildcard.crt"
io = TCPSocket.new("SECURELDAP.MYSITE.EDU","636")
ctx = OpenSSL::SSL::SSLContext.new
#ctx.cert = OpenSSL::X509::Certificate.new(File.read("server-wildcard.crt"))
#ctx.client_ca = OpenSSL::X509::Certificate.new(File.read("server-wildcard.crt"))
#ctx.ca_file = "server-wildcard.crt"
#ctx.ca_path = "./"
ctx.verify_mode = OpenSSL::SSL::VERIFY_PEER | OpenSSL::SSL::VERIFY_FAIL_IF_NO_PEER_CERT
ctx.cert_store = cert_store
conn = OpenSSL::SSL::SSLSocket.new(io, ctx)
conn.connect
I am posting my solution here for the sake of completeness.
net-ldap gem override to support certificate validation
https://gist.github.com/mintuhouse/9931865
Ideal Solution:
Maintain list of trusted root CAs on your server
(If you are lazy like me, have a cron job which will download (weekly maintained by curl) copy from http://curl.haxx.se/ca/cacert.pem)
Override Net::HTTP to always use this trusted certificate list
As of today (late 2016), ruby-net-ldap supports this upstream! However, tls_options needs to be passed with verify_mode set to a value other than the default VERIFY_NONE.
# optional: create/pass your own cert_store
cert_store = OpenSSL::X509::Store.new
cert_store.set_default_paths # or add your own CAdir, &c.
# attributes documented for OpenSSL::SSL::SSLContext are valid here
tls_options = {
verify_mode: OpenSSL::SSL::VERIFY_PEER
cert_store: cert_store
}
ldap = Net::LDAP.new(
:host => host,
:port => port,
:encryption => {
:method => :simple_tls, # could also be :start_tls
:tls_options => tls_options
}
)
I cannot access to server through RestClient.
I have this code written in cURL (it sends XML file to server for parsing)
curl -T /Users/amok/Code/1188/dragon/test.xml --cert Users/amok/Code/1188/dragon/client_1188.pem --key Users/amok/Code/1188/dragon/client_1188.key -k -u 1188:aaa https://smart.com/index.cgi
Need to make the same in RestClient. I've tried this one, but it doesnt work.
return RestClient::Resource.new(
'https://smart.com/index.cgi',
:ssl_client_cert => OpenSSL::X509::Certificate.new(File.read(File.join(Rails.root, "/cert/client_1188.pem"))),
:ssl_client_key => OpenSSL::PKey::RSA.new(File.read(File.join(Rails.root, "/cert/client_1188.key"))),
:user => '1188',
:password => 'aaa'
).post(xml, :content_type => 'application/xml')
(xml is file, stored on server)
The problem is:
OpenSSL::SSL::SSLError in BackOffice::DragonPilotController#create
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
The palm is on my face now... ((( Thank you
use PUT method instead of POST