Decrypt string with AES Cipher Block Chaining in Rails - ruby-on-rails

I am having to implement a payment gateway in Rails that I've not worked with or seen before (Westpac's Payway in Australia if anyone is interested).
Their documentation isn't bad and the system is fairly logical, so much so that it's been quite painless so far (a miracle for payment integration).
Where there is an issue is that after the payment is POSTed directly to Westpac and the payment processed they redirect back to our site with a large encrypted parameter. This is then meant to be decrypted by us to get access to the actual parameters.
Here is Westpac's guidance:
The parameters are encrypted using AES with Cipher Block Chaining, using PCKS-5
Padding. The decryption algorithm should be initialised with a 16 byte, zero-filled
initialization vector, and should use your encryption key (which can be found on the Security page of PayWay Net Shopping Cart setup).
Before decryption, the parameters passed with the redirect will appear as follows:
EncryptedParameters=QzFtdn0%2B66KJV5L8ihbr6ofdmrkEQwqMXI3ayF7UpVlRheR7r5fA6
IqBszeKFoGSyR7c7J4YsXgaOergu5SWD%2FvL%2FzPSrZER9BS7mZGckriBrhYt%2FKMAbTSS8F
XR72gWJZsul9aGyGbFripp7XxE9NQHVMWCko0NlpWe7oZ0RBIgNpIZ3JojAfX7b1j%2F5ACJ79S
VeOIK80layBwCmIPOpB%2B%2BNI6krE0wekvkkLKF7CXilj5qITvmv%2FpMqwVDchv%2FUNMfCi
4uUA4igHGhaZDQcV8U%2BcYRO8dv%2FnqVbAjkNwBqxqN3UPNFz0Tt76%2BP7H48PDpU23c61eM
7mx%2FZh%2Few5Pd0WkiCwZVkSZoov97BWdnMIw5tOAiqHvAR3%2BnfmGsx
Westpac has no Rails demos but they do have PHP. Here is the PHP demo:
function decrypt_parameters( $base64Key, $encryptedParametersText, $signatureText )
{
$key = base64_decode( $base64Key );
$iv = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0";
$td = mcrypt_module_open('rijndael-128', '', 'cbc', '');
// Decrypt the parameter text
mcrypt_generic_init($td, $key, $iv);
$parametersText = mdecrypt_generic($td, base64_decode( $encryptedParametersText ) );
$parametersText = pkcs5_unpad( $parametersText );
mcrypt_generic_deinit($td);
}
Here is what I've tried in Rails:
def Crypto.decrypt(encrypted_data, key, iv, cipher_type)
aes = OpenSSL::Cipher::Cipher.new(cipher_type)
aes.decrypt
aes.key = key
aes.iv = iv if iv != nil
aes.update(encrypted_data) + aes.final
end
iv = "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
key = Base64.decode64("mysecretkey")
data = Base64.decode64("QzFtdn0%2B66KJV5L8ihbr6ofdmrkEQwqMXI3ayF7UpVlRheR7r5fA6
IqBszeKFoGSyR7c7J4YsXgaOergu5SWD%2FvL%2FzPSrZER9BS7mZGckriBrhYt%2FKMAbTSS8F
XR72gWJZsul9aGyGbFripp7XxE9NQHVMWCko0NlpWe7oZ0RBIgNpIZ3JojAfX7b1j%2F5ACJ79S
VeOIK80layBwCmIPOpB%2B%2BNI6krE0wekvkkLKF7CXilj5qITvmv%2FpMqwVDchv%2FUNMfCi
4uUA4igHGhaZDQcV8U%2BcYRO8dv%2FnqVbAjkNwBqxqN3UPNFz0Tt76%2BP7H48PDpU23c61eM
7mx%2FZh%2Few5Pd0WkiCwZVkSZoov97BWdnMIw5tOAiqHvAR3%2BnfmGsx")
cleartext = Crypto.decrypt(data, key, iv, "AES-128-CBC")
And I simply pass in the same initialization vector as noted in the PHP, though I'm not sure this is correct for Rails.
In any event, the key is provided and easy to Base64 decode, as are the Encrypted Parameters. At the end of the day, I'm getting this error:
cipher.rb:21:in `final': wrong final block length (OpenSSL::Cipher::CipherError)
from cipher.rb:21:in `decrypt'
from cipher.rb:29:in `<main>'
Admittedly, I'm out of my depth on this Crypto stuff but am up against a wall and do not have the time (despite the interest) to learn more.

The problem was, that the input data was additionally "URI-escaped" and ruby's base64-decoder did not "care" about the invalid base64-input (% is no base64-digit), so no error was raised.
The solution was to "unescape" the URI-encoding with URI.unescape:
require 'uri'
data = Base64.decode64(
URI.unescape("QzFtdn0%2B66 ... Iw5tOAiqHvAR3%2BnfmGsx"))
Of course, if the input data is received from a GET/POST parameter, the input data is most probably already "unescaped" by your web-stack - just as a note of caution (double unescape may cause problems if a percent-sign % appears in the input data).

Related

ECIES encryption and EC key generation in Swift

Backend uses this java implementation for encrypting the data using public key(generated from iOS App in swift),
Cipher iesCipher = Cipher.getInstance("ECIESwithAES-CBC");
byte[] derivation = Hex.decode(derivationString);
byte[] encoding = Hex.decode(encodingString);
byte[] nonce = Hex.decode(nonceString);
IESParameterSpec params = new IESParameterSpec(derivation, encoding, 128, 128, nonce, true);
iesCipher.init(Cipher.ENCRYPT_MODE, publicKey, params);
byte[] ciphertext = iesCipher.doFinal(data.getBytes());
But in swift I could not find any equivalent library to decrypt it. I am using the SwiftECC for generating the EC key-pair and sending the public key to server. Then server encrypts the data using that public key as mentioned in above implementation code. But SwiftECC has no any decrypt function which accepts the arguments like derivation, encoding, nonce. And I could not find anything similar to above java implementation in swift.
ECIES is using ephemeral static-ECDH + key derivation + encryption, where the public key of the ephemeral key pair is stored with the ciphertext. So if you find ECDH + the right key derivation + AES-CBC encryption (probably using PKCS#7 padding) then you'd be in business. Apparently BC uses KDF2 with SHA-1 for key derivation. I've asked and answered what KDF2 is here.
I’ve implemented ECIES in EllipticCurveKit, feel free to use it as inspiration. Should be straightening enough to port it to SwiftECC if you don’t want to use EllipticCurveKits EC implementation.
I've hard coded the curve to be Secp256k1 (aka "the bitcoin curve"), but you could easily make it generic, by passing in a generic Curve type, just change:
func seal<Plaintext>(
_ message: Plaintext,
whitePublicKey: PublicKey<Secp256k1>,
blackPrivateKey: PrivateKey<Secp256k1>,
nonce: SealedBox.Nonce? = nil
) -> Result<SealedBox, EncryptionError> where Plaintext: DataProtocol
into (which should work in EllipticCurveKit, and should be possible in other EC libs too):
func seal<Plaintext, Curve>(
_ message: Plaintext,
whitePublicKey: PublicKey<Curve>,
blackPrivateKey: PrivateKey<Curve>,
nonce: SealedBox.Nonce? = nil
) -> Result<SealedBox, EncryptionError> where Plaintext: DataProtocol, Curve: EllipticCurve

How to generate a Tink key from a user-provided password

I want to store a keyset, and would like the file to be encrypted with key produced from a user-provided "master password". And of course, at a later point I'd like to, given the same master password, be able to load that keyset by decrypting the file.
It seems that I need an Aead, which I can generate from a KeysetHandle with AeadFactory.getPrimitive(keysetHandle). But how can I produce a KeysetHandle from a "master password"?
(And for the context of this question, getting that key from a Key Management Systems, instead of producing it "out of thin air" from a master password, isn't an option.)
An Aead can be created as follows (here done from Scala):
val password: String = "..."
val aead = {
val messageDigest = MessageDigest.getInstance("SHA-256")
messageDigest.update(password.getBytes(CharsetNames.Utf8))
val key256Bit = messageDigest.digest()
val key128Bit = key256Bit.take(16)
new AesGcmJce(key128Bit)
}
A few comments:
I'd prefer the key to be based on a 32-bit digest, but the cipher picked by Tink in this case throws an exception when provided with a 32-bit key, hence the shortening to a 16-bit key.
It seems shortening the key this way is ok from a hash weakness perspective.

How to store encrypted data?

I'm new to ruby on rails, and I'm developing an application that will have very sensitive data (api keys from other websites) and I need to store it encrypted in a db but without knowing them at any time.
Let me explain myself:
The form asks the user for his api keys
Encrypt them
Store it in the db
The main question is, how do I encrypt them in such a way that I can use them later (still without knowing them)?
Sorry if the question is silly, but I can't find a way to do it, and thanks.
I've used attr_encrypted for this. Works great.
class User
attr_encrypted :ssn, key: 'This is a key that is 256 bits!!'
end
You then work with ssn as if it were a plain field
user = User.find(1)
puts user.ssn
but it's encrypted at rest (in the database) and can't be retrieved without the key.
def encrypt text
text = text.to_s unless text.is_a? String
len = ActiveSupport::MessageEncryptor.key_len
salt = SecureRandom.hex len
key = ActiveSupport::KeyGenerator.new(Rails.application.secrets.secret_key_base).generate_key salt, len
crypt = ActiveSupport::MessageEncryptor.new key
encrypted_data = crypt.encrypt_and_sign text
"#{salt}$$#{encrypted_data}"
end
def decrypt text
salt, data = text.split "$$"
len = ActiveSupport::MessageEncryptor.key_len
key = ActiveSupport::KeyGenerator.new(Rails.application.secrets.secret_key_base).generate_key salt, len
crypt = ActiveSupport::MessageEncryptor.new key
crypt.decrypt_and_verify data
end
Pass the key to encrypt method and store the returned encrypted value in DB.
Then to decrypt pass the encrypted key to the decrypt method.
This is assuming your Secret Key Base is in Rails.application.secrets.secret_key_base
The original source for the answer is here

How can I retrieve a salt from LDAP?

The organization that I work for uses PPolicy (an OpenLDAP module) to automatically salt and hash passwords. Unfortunately, I don't have access to the machine running the OpenLDAP server, so i can't look at the config file. From what I've seen though, pretty much everything appears to be setup using the default settings.
I'd like to be able to retrieve the salt for a specific user. If I look at the user's attributes, userPassword is the SSHA password. I don't see anything about a salt for that specific user. I ended up looking at the LDAP schema and I see nothing about salts there either.
If you were to guess where the salt were being stored for each user, where would it be? I understand this is vague and probably not a lot of information, but I can't find anywhere in the OpenLDAP docs that explain where exactly the unique salts are stored. Perhaps someone who has configured an OpenLDAP server before would know where the default location is.
Thank you.
With SSHA, normally the salt is appended to the SHA1 hash and then the whole thing is Base64 encoded (I've never seen an LDAP that didn't do SSHA this way). You should be able to tell this by looking at the userPassword attribute. If it's 28 character long with a = at the end, it's only the hash.
If the Base64 value is 32 character long or greater, it contains both the hash and the salt. Base64 decode the value and strip off the first 20 bytes, this is the SHA1 hash. The remaining bytes are the salt.
Example:
Base64 encoded hash with salt
userPassword: {SSHA}MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0
Base64 decoded value
SHA1 Hash Salt
--------------------++++
123456789012345678901234
Edit: After double checking, it seems that variable length salts are sometimes supported. Corrected the encoding description to account for this.
The post of Syon did help me a lot, thanks for that! I thought a working test would be a nice extra for someone else struggling with this topic ;).
public class SshaPasswordVerifyTest {
private final static int SIZE_SHA1_HASH = 20;
#Test
public void itShouldVerifyPassword() throws Exception{
String password = "YouNeverGuess!";
String encodedPasswordWithSSHA = "{SSHA}M6HeeJAbwUCzuLwXbq00Fc3n3XcxFI8KjQkqeg==";
Assert.assertEquals(encodedPasswordWithSSHA, getSshaDigestFor(password, getSalt(encodedPasswordWithSSHA)));
}
// The salt is the remaining part after the SHA1_hash
private byte[] getSalt(String encodedPasswordWithSSHA){
byte[] data = Base64.getMimeDecoder().decode(encodedPasswordWithSSHA.substring(6));
return Arrays.copyOfRange(data, SIZE_SHA1_HASH, data.length);
}
private String getSshaDigestFor(String password, byte[] salt) throws Exception{
// create a SHA1 digest of the password + salt
MessageDigest crypt = MessageDigest.getInstance("SHA-1");
crypt.reset();
crypt.update(password.getBytes(Charset.forName("UTF-8")));
crypt.update(salt);
byte[] hash = crypt.digest();
// concatenate the hash with the salt
byte[] hashPlusSalt = new byte[hash.length + salt.length];
System.arraycopy(hash, 0, hashPlusSalt, 0, hash.length);
System.arraycopy(salt, 0, hashPlusSalt, hash.length, salt.length);
// prepend the SSHA tag + base64 encode the result
return "{SSHA}" + Base64.getEncoder().encodeToString(hashPlusSalt);
}
}
In PHP, this compares a plain text password (usually entered by a user) to a given ssha hash (usually stored in your db):
private function checkSshaPassword($encrypted_password, $password)
{
// get hash and salt from encrypted_password
$base_64_hash_with_salt = substr($encrypted_password, 6);
$hash_with_salt = base64_decode($base_64_hash_with_salt);
$hash = substr($hash_with_salt, 0, 20);
$salt = substr($hash_with_salt, 20);
// hash given password
$hash_given = sha1($password . $salt, true);
return ($hash == $hash_given);
}

QuickBooks Online querying with filter returns 401 everytime

I've had success creating objects with POST and Content-Type application/xml
I've also had success querying using Content-Type application/x-www-form-urlencoded with a blank request body which returns all of the object type depending on which URI I specify.
I can also get the same to work with something like PageNum=1&ResultsPerPage=1 in the request body and I have figured out how to incorporate that into the signature so I get a valid response.
However no matter how I format it, I cannot get anything other than a 401 response when I try to use a filter (something basic like Filter=FAMILYNAME :EQUALS: Doe). I've read over the OAuth Core 1.0 Revision A specifications on how all parameter names and values are escaped using the [RFC3986] percent-encoding. However I feel like I'm missing a step or formatting incorrectly. I've seen inconsistent information in my searching through Intuit's forums on what exactly is the proper format.
Any help on this would be greatly appreciated. I've been struggling with this for a good week now.
The response I get when trying to use a filter is:
HTTP Status 401 - message=Exception authenticating OAuth; errorCode=003200; statusCode=401
----Update----
I'm am seeing the same error when I try to use filters with the New IPP Developer Tools - IPP API Explorer. I'm using the IDS V2 QBO API Explorer. I'm able to use that tool to do a retrieve all Post and the response shows all of my customers, but when I try to use a filter I get :
Server Error
401 - Unauthorized: Access is denied due to invalid credentials.
You do not have permission to view this directory or page using the credentials that you supplied.
Any Ideas? If I'm getting the same error from the API Explorer tool, it makes me think the problem is something else entirely.
----Final Update----
I have finally had success with filters and I believe I have figure out what my problem was. I was always suspicious that I was able to get queries with pagination like "PageNum=1&ResultsPerPage=1" to work, but could not get something like "Filter=FAMILYNAME :EQUALS: Doe". I suspected there problem was with the white space in the filter format. What threw me off tracking this down earlier was that I could not get the filters to work in the IDS V2 QBO API Explorer. That made me suspect there was something else going on. I decided to ignore the API Explorer all together and focus on why I could get it to work the one way but no the other.
I believe my problem came down to improper encoding of the Filter's value in the signature. That explains the 401 invalid signature errors I was getting.
"Filter=Name :EQUALS: Doe" becomes "Filter=Name%20%3AEQUALS%20%3ADoe" after normalization.
Percent-Encoding that should give "Filter%3DName%2520%253AEQUALS%2520%253ADoe".
In essence you have to "double" encode the blank space and the colons, but not the equal sign. I tried many permutations of doing the encoding, but believe my mistake was that I was either not "double" encoding, or when I was double encoding I was including the '=' sign. Either way breaks your signature. Thanks for everyone's input.
I believe my problem came down to improper encoding of the Filter's value in the signature. That explains the 401 invalid signature errors I was getting.
I used an online tool to take me through the steps in properly signing an Oauth request. While going through those steps I realized my problem was with the steps where you normalize the request parameters and then percent-encode them. I was including the '=' of the filter in the normalization step, which breaks your signature. The tool I used can be found at:
http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/
Thanks for everyone's input.
Do you get a 401 with the same request in the API Explorer?
http://ippblog.intuit.com/blog/2013/01/new-ipp-developer-tool-api-explorer.html
Also, are you using the static base URL or retrieving it at runtime?
https://ipp.developer.intuit.com/0010_Intuit_Partner_Platform/0050_Data_Services/0400_QuickBooks_Online/0100_Calling_Data_Services/0010_Getting_the_Base_URL
If you are using the static base URL, try switching to the runtime base URL to see if you still get the error.
peterl answered one of my questions on here that may also answer yours. I had been trying to put the Filters in the body when they should have gone into the header. Here was peterl's code sample for getting all unpaid invoices (open balance greater than 0.00) for a particular customer.
http://pastebin.com/raw.php?i=7VUB6whp
public List<Intuit.Ipp.Data.Qbo.Invoice> GetQboUnpaidInvoices(DataServices dataServices, int startPage, int resultsPerPage, IdType CustomerId)
{
StringBuilder requestXML = new StringBuilder();
StringBuilder responseXML = new StringBuilder();
var requestBody = String.Format("PageNum={0}&ResultsPerPage={1}&Filter=OpenBalance :GreaterThan: 0.00 :AND: CustomerId :EQUALS: {2}", startPage, resultsPerPage, CustomerId.Value);
HttpWebRequest httpWebRequest = WebRequest.Create(dataServices.ServiceContext.BaseUrl + "invoices/v2/" + dataServices.ServiceContext.RealmId) as HttpWebRequest;
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded";
httpWebRequest.Headers.Add("Authorization", GetDevDefinedOAuthHeader(httpWebRequest, requestBody));
requestXML.Append(requestBody);
UTF8Encoding encoding = new UTF8Encoding();
byte[] content = encoding.GetBytes(requestXML.ToString());
using (var stream = httpWebRequest.GetRequestStream())
{
stream.Write(content, 0, content.Length);
}
HttpWebResponse httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse;
using (Stream data = httpWebResponse.GetResponseStream())
{
Intuit.Ipp.Data.Qbo.SearchResults searchResults = (Intuit.Ipp.Data.Qbo.SearchResults)dataServices.ServiceContext.Serializer.Deserialize<Intuit.Ipp.Data.Qbo.SearchResults>(new StreamReader(data).ReadToEnd());
return ((Intuit.Ipp.Data.Qbo.Invoices)searchResults.CdmCollections).Invoice.ToList();
}
}
protected string GetDevDefinedOAuthHeader(HttpWebRequest webRequest, string requestBody)
{
OAuthConsumerContext consumerContext = new OAuthConsumerContext
{
ConsumerKey = consumerKey,
ConsumerSecret = consumerSecret,
SignatureMethod = SignatureMethod.HmacSha1,
UseHeaderForOAuthParameters = true
};
consumerContext.UseHeaderForOAuthParameters = true;
//URIs not used - we already have Oauth tokens
OAuthSession oSession = new OAuthSession(consumerContext, "https://www.example.com",
"https://www.example.com",
"https://www.example.com");
oSession.AccessToken = new TokenBase
{
Token = accessToken,
ConsumerKey = consumerKey,
TokenSecret = accessTokenSecret
};
IConsumerRequest consumerRequest = oSession.Request();
consumerRequest = ConsumerRequestExtensions.ForMethod(consumerRequest, webRequest.Method);
consumerRequest = ConsumerRequestExtensions.ForUri(consumerRequest, webRequest.RequestUri);
if (webRequest.Headers.Count > 0)
{
ConsumerRequestExtensions.AlterContext(consumerRequest, context => context.Headers = webRequest.Headers);
if (webRequest.Headers[HttpRequestHeader.ContentType] == "application/x-www-form-urlencoded")
{
Dictionary<string, string> formParameters = new Dictionary<string, string>();
foreach (string formParameter in requestBody.Split('&'))
{
formParameters.Add(formParameter.Split('=')[0], formParameter.Split('=')[1]);
}
consumerRequest = consumerRequest.WithFormParameters(formParameters);
}
}
consumerRequest = consumerRequest.SignWithToken();
return consumerRequest.Context.GenerateOAuthParametersForHeader();
}
You can also see my original Question Here on StackOverflow: Query for All Invoices With Open Balances using QuickBooks Online (QBO) Intuit Partner Platform (IPP) DevKit.

Resources