rails openssl different results of encryption from code and terminal - ruby-on-rails

when I try to encrypt data from terminal like
echo -n "TestData" | openssl enc -aes-256-cbc -a -K C81E728D9D4C2F636F067F89CC14862C65990ABE58735B91B6B8798E8CE45F22 -iv D342F9C6310F6B21E97AB38595BD8CAA
than the Base64 encoded result I receive is
VJwJBTtVntJvRGkD24S4wg==
But when I try same thing with rails using exactly same key and initialization vector
cipher = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
cipher.encrypt
cipher.key = "C81E728D9D4C2F636F067F89CC14862C65990ABE58735B91B6B8798E8CE45F22"
cipher.iv = "D342F9C6310F6B21E97AB38595BD8CAA"
encrypted_data = cipher.update("TestData")
encrypted_data << cipher.final
Base64.strict_encode64(encrypted_data)
than I receive entirely different Base64 encoded result
qavpNrU7llgauAyyEZz/bw==
can someone point that what I missed?

You provide the key and iv attributes as hex strings, but the expected format is raw bytes. Converting them to binary yields the expected result, with the following script:
require 'openssl'
require 'base64'
def hex_to_bin(s)
s.scan(/../).map { |x| x.hex.chr }.join
end
cipher = OpenSSL::Cipher::Cipher.new('aes-256-cbc')
cipher.encrypt
cipher.key = hex_to_bin("C81E728D9D4C2F636F067F89CC14862C65990ABE58735B91B6B8798E8CE45F22")
cipher.iv = hex_to_bin("D342F9C6310F6B21E97AB38595BD8CAA")
encrypted_data = cipher.update("TestData")
encrypted_data << cipher.final
puts Base64.strict_encode64(encrypted_data)
(Source for the hex_to_bin function: To Hex and Back (With Ruby)).
Calling it encrypt.rb, this is the result of running it:
$ ruby encrypt.rb
encrypt.rb:8: warning: constant OpenSSL::Cipher::Cipher is deprecated
VJwJBTtVntJvRGkD24S4wg==
To get rid of the "is deprecated" warning I had to replace the deprecated class OpenSSL::Cipher::Cipher with OpenSSL::Cipher.

The key is to short, 98304A2480DDC0FA354278936DAC2A0D7D9074650AD6 is an invalid key size, AES keys are 128, 192 or 256 bits in length (16, 24 or 32 bytes). Since it appears the key should be 256-bits (32-bytes) the missing key bytes will be either garbage or possibly nulls, key extension is undefined. Thus different results.
Assuming null padding and PKCS#7 padding for the first case the result is correct:
AESCALC
The second example is filling out the key in some other manor.
The solution is to use a full length key.

Related

0x85 windows 1252 breaks line if file opened with utf-8 encoding

I have a file with an old format from the 70s used in Companies House (UK company registry).
I inherited a parser written 6 years ago which goes line by line and according to a set of conditions extracts the information from the line and inserts them into a dictionary.
There is a weird character that is breaking a line.
I copied this line to a new file awk '{if(NR==33411) print $0}' PROD216_1950_ew_1.dat > broken and opend broken in vim.
Turns out that weird character is read by vim a <85>.
The result is that everything after MAYFIELD is read as a new line.
Below the line in question:
000376702103032986930001 1993010119941024 193709 0105<BARRY ALEXANDER<GROSVENOR<<<<MAYFIELD 3<41 PLANTATION ROAD<THE PEAK<<HONG KONG<BANK EXECUTIVE<BRITISH<<
in vim becomes
000376702103032986930001 1993010119941024 193709 0105<BARRY ALEXANDER<GROSVENOR<<<<MAYFIELD <85>3<41 PLANTATION ROAD<THE PEAK<<HONG KONG<BANK EXECUTIVE<BRITISH<<
I am using codecs to read this file with a context manager, which I thought was the way of going about it -
Is there anything I am missing? What is that <85>?
with codecs.open(filepath, 'r', 'utf-8') as fh:
for line in fh:
linetype = determine_line_type(line)
if linetype == 'header':
continue
elif linetype == 'company':
do stuff...
elif linetype == 'officer':
do stuff...
vim shows <85> to indicate a hex 85 byte that is invalid in the current encoding (i.e., the encoding it's using to decode the file).
My guess is that the file's encoding is Windows-1252, in which hex 85 denotes the ellipsis character.
So the solution for your parser might be as simple as changing 'utf-8' to 'cp1252' in the codecs.open call.
After going around for some time here and here I came up with this solution, which works.
with open(filepath, encoding='utf-8') as fh:
for line in fh:
byteline = bytearray(line, encoding='utf-8').replace(b'\xc2\x85', b'')
line_clean = byteline.decode(encoding='utf-8')
# do stuff with clean line.
Knowing that the byte sequence that breaks the string is b'\xc2\x85' (it is interpreted as an ... ellipsis character.
First encode the string to an array of bytes with bytearray, then use replace method of the bytearray class, finally, decode the clean line using the decode method, which will return the string without the weird character from before the transformation.

Ruby Gem randomly returns Encoding Error

So I forked this gem on GitHub, thinking that I may be able to fix and update some of the issues with it for use in a Rails project. I basically get this output:
irb(main):020:0> query = Query::simpleQuery('xx.xxx.xxx.xx', 25565)
=> [false, #<Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT>]
irb(main):021:0> query = Query::simpleQuery('xx.xxx.xxx.xx', 25565)
=> {:motd=>"Craftnet", :gametype=>"SMP", :map=>"world", :numplayers=>"0", :maxplayers=>"48"}
The first response is the example of the Encoding error, and the second is the wanted output (IP's taken out). Basically this is querying a Minecraft server for information on it.
I tried using
Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8
But that just gave the same response, randomly spitting encoding errors and not.
Here is the relevant GitHub repo with all the code: RubyMinecraft
Any help would be greatly appreciated.
In the Query class there is this line:
#key = Array(key).pack('N')
This creates a String with an associated encoding of ASCII-8BIT (i.e. it’s a binary string).
Later #key gets used in this line:
query = #sock.send("\xFE\xFD\x00\x01\x02\x03\x04" + #key, 0)
In Ruby 2.0 the default encoding of String literals is UTF-8, so this is combining a UTF-8 string with a binary one.
When Ruby tries to do this it first checks to see if the binary string only contains 7-bit values (i.e. all bytes are less than or equal to 127, with the top byte being 0), and if it does it considers it compatible with UTF-8 and so combines them without further issue. If it doesn’t, (i.e. if it contains bytes greater than 127) then the two strings are not compatible and an Encoding::CompatibilityError is raised.
Whether an error is raised depends on the contents of #key, which is initialized from a response from the server. Sometimes this value happens to contain only 7-bit values, so no error is raised, at other times there is a byte with the high bit set, so it generates an error. This is why the errors appear to be “random”.
To fix it you can specify that the string literal in the line where the two strings are combined should be treated as binary. The simplest way would be to use force_encoding like this:
query = #sock.send("\xFE\xFD\x00\x01\x02\x03\x04".force_encoding(Encoding::ASCII_8BIT) + #key, 0)

Inconsistent IO character reading when converting encoding

In Ruby 1.9.3-429, I am trying to parse plain text files with various encodings that will ultimately be converted to UTF-8 strings. Non-ascii characters work fine with a file encoded as UTF-8, but problems come up with non-UTF-8 files.
Simplified example:
File.open(file) do |io|
io.set_encoding("#{charset.upcase}:#{Encoding::UTF_8}")
line, char = "", nil
until io.eof? || char == ?\n || char == ?\r
char = io.readchar
puts "Character #{char} has #{char.each_codepoint.count} codepoints"
puts "SLICE FAIL" unless char == char.slice(0,1)
line << char
end
line
end
Both files are just a single string áÁð encoded appropriately. I have checked that the files have been encoded correctly via $ file -i <file_name>
With a UTF-8 file, I get back:
Character á has 1 codepoints
Character Á has 1 codepoints
Character ð has 1 codepoints
With an ISO-8859-1 file:
Character á has 2 codepoints
SLICE FAIL
Character Á has 2 codepoints
SLICE FAIL
Character ð has 2 codepoints
SLICE FAIL
The way I am interpreting this is readchar is returning an incorrectly converted encoding which is causing slice to return incorrectly.
Is this behavior correct? Or am I specifying the file external encoding incorrectly? I would rather not rewrite this process so I am hoping I am making a mistake somewhere. There are reasons why I am parsing files this way, but I don't think those are relevant to my question. Specifying the internal and external encoding as an option in File.open yielded the same results.
This behavior is a bug. See http://bugs.ruby-lang.org/issues/8516 for details.

How do I convert this "array of bytes" to PDF in ROR (Ruby)?

Over the web service, I am returned an array of bytes. Part of which looks like the following... How do I get this back to a file? (It started as a pdf)
rg1uje94ppbarWm6azwlDCJeHFFJuXlMN532v46qiyi2u/WNVHCgl10DFe64oZVSFKHN7pZ6qaulNHULZJjix33PWhzPLBVwcbptx5Husx+a7Y4q3T76KBu7pfjvXeav1emibcBSG2mMFakTv0Ho7LvYsVf57hzUq8ptL752worpSKa3L0s9IJ6Z6qIlFzDaXW4ml+3WCWvaHhUW2H+6xfFSuhjHzL8pKmd5t3aI8vsun16YY1VwLw9ivAGX+GUPRVBOTYpVqgLikJhKB7Fkpn5SJSATFAQGoviYGsw7A+B2hA0dpVlisUf0mvC2LjYwfEhcUPGmvwG3sRpGJkUPtzXWx+5a2UaTOtytnLR9qwFbXKf8s2DxS9dR/p+/rwjb9mr24p7E2e8e/ZWNP7dpX7V7xJWpLAxu67lOYhixHFPRZff6063L5q8yGXtOc/J/YP5sSev6l8trGk+c+WNXSa5+b7PfpqY/WJbkefxp4Xe5RfaHqx6oqU/o9ObBdjn3MDm3MzkvFmrvWaXfPavC9s6/8gZZdMeI3cPyp8n/nBSnpjXYUwelZlyKm+ek7Pl8YfhXM4c6uTwxhPyJvZscfRnzaSd7cwWLTs3zj8ucXWe7TGzR+NGXumfk7HqVXCAkrJVS/T+uNDXKHSh5viMpPuTzW+vXu7vIj7eOXLT47XX1vYynzBdcaGx1qo0qrEijTL81UcSZRrFwS7Zv72L/paRvgswpPVdNKe/Qq9hT2R/XQXC8De/HaGVqkC1rkqFIxCto1vzFn1+1xGpOgu+fG/I7P7NBiqm+Ri823b7edVvMEvoIuVLjvjJ7Mv3nTRcV2ZKn+CeR06xqGtHnfN6XVCyyiRx8d2DdxbM0Whz19Imd928mSGz9KpLbXZ0NZhaNX7e08BjbR4fsO+fcdZ7fnhMz0FN2rEnplApbV+aLRt/zHFc15fDpt3/6Kz77vjM+aGNgjJ/eaCpseryirwPdcHuovZPLr3sVRnp2XZwpwH5hwrK0u3vB
Ive tried a few things, the closest is as follows (though I am unsure by the output if it is correct):
File.open(pdf_filename, 'w' ) do |output|
byteArray.each_byte do | byte |
output.print byte
puts byte
end
end
which returns in the console the following but does not create a valid file ( I assume these numbers are the bytes in Integer (base10)form or something?) :
77
52
79
89
57
etc..
I am no expert.. I am learning ruby myself at the moment (looking at questions on SO to vary techniques a bit ;-)
but have you tried:
File.open(pdf_filename, 'wb' ) do |output|
byteArray.each_byte do | byte |
output.print byte
puts byte
end
end
or maybe even (I really don't know if that will work) I don't have Ruby installed here to test:
File.open(pdf_filename, 'wb') { |output|
output << byteArray
}
I got this info from here (among other places):
http://strugglingwithruby.blogspot.com/2008/11/ruby-file-access.html
Binaries files are just the same; you just add a b to the second parameter of the open method.
Depending on your byte array format, you may need to use the unpack method.
File.open(pdf_filename, 'wb' ) do |output|
output << byteArray.unpack("m")
end
See the following for possible parameters in the unpack method:
http://www.codeweblog.com/ruby-string-pack-unpack-detailed-usage/

Decrypt Lua byte code?

I'm encrypting my Lua code with this script.
local script = string.dump(
function()
local function h4x(strtbl)
buffer=""
for v in strtbl do
buffer=buffer..strtbl[v]
end
return buffer
end
print("encrypted")
end
)
buff=""
for v=1,string.len(script) do --Convert our string into a hex string.
buff=buff..'\\'..string.byte(script,v)
end
file=io.open('encrypted.txt','w') --Output our bytecode into ascii format to encrypted.txt
file:write(buff)
file:flush()
file:close()
The output of encrypted.txt is like "00/12/46/4/2/6/4/62/". How do I decrypt bytecode?
This text is not encrypted. It's just Lua bytecode in hexadecimal.
Discussion of means of disassembling this bytecode into human-readable opcodes is in another question: Lua equivalent to Python dis()?
Obviously its printing out each BYTE as a value (which is decimal, even though its stated its converted to hex) delimited by a '/'.
All you need to do then is fill an array using the bytes you pull from the string, using tonumber to convert them back to their byte value. this will help with parsing the formatted output

Resources