I want to generate a tar from a buch of files.
out_file = File.new('some.tar', 'w')
tar = Gem::Package::TarWriter.new out_file
attachments = #Array of attachment objects
attachments.each{|a|
file = Attachment.new(a).read_file #returns a String
file.force_encoding('UTF-8')
tar.add_file_simple( a[:filename], 777, file.length) do |io|
io.write(file)
end
}
Gem::Package::TarWriter::FileOverflow - You tried to feed more data
than fits in the file.
Has anyone an idea why this happens and how to fix it?
String#length returns the number of characters in the String. Since a UTF-8 character can be represented by more than a single byte, the bytesize of a string is usually larger.
The TarWriter now expects the file size to be given in bytes. Thus, if you use anything else than plain ascii characters in your file, it will overflow.
To solve this, you should thus pass file.bytesize to the add_file_simple method instead of file.size.
I'm sending an .AAC file from an iPhone to an API in RoR. What I'm doing is read the file byte by byte in the iPhone, convert the byte[] to a Base64 String, send the string to an API and then decode the String to the array and save that byte[] to a file.
The problem is the file being created on the serve side is different from the one sent, even though I checked and the byte[] is the same on the server side, when I save byte by byte I end up with a different filesize and it's also unplayable.
This is the code I'm using
File.open("test.aac", 'wb' ) do |output|
plain.each_byte do | byte |
output.print byte
puts byte
i=i+1
end
puts "_______"
puts i
puts "_______"
end
I've literally tried everything but I have no idea why it doesn't work.
This is the code that receives
mail=params["mail"]
archivo=params["byte"]
puts mail
puts archivo
plain=Base64.decode64(archivo);
variable plain has exactly the same bytes as the byte I sent from the iPhone.
This is in Xamarin:
byte[] info = File.ReadAllBytes (audioFilePath.Path.ToString ());
String bytesTo64 = Convert.ToBase64String (info);
Okay, so the problem was writing byte by byte. I changed the code with
Base64 encoded string to file(Ruby on Rails)
and it worked flawlessly. This is my final code.
File.open('Now.aac', 'wb') { |f|
f.write(Base64.decode64(archivo))
}
I am building an iPhone app which gets a decrypted string via JSON from a PHP backend.
In PHP I am encrypting the string like this:
$encrypt_method = "AES-256-CBC";
$secret_key = 'This is my secret key';
$secret_iv = 'This is my secret iv';
// hash
$key = hash('sha256', $secret_key);
// iv - encrypt method AES-256-CBC expects 16 bytes - else you will get a warning
$iv = substr(hash('sha256', $secret_iv), 0, 16);
if( $action == 'encrypt' ) {
$output = openssl_encrypt($string, $encrypt_method, $key, 0, $iv);
$output = base64_encode($output);
}
In Objective C I tried to decrypt this string with BBEAS: https://github.com/benoitsan/BBAES
This is the code I have got in Objective C:
NSData* salt = [BBAES IVFromString:#"This is my secret iv"];
NSData *key = [BBAES keyBySaltingPassword:#"This is my secret key" salt:salt keySize:BBAESKeySize256 numberOfIterations:BBAESPBKDF2DefaultIterationsCount];
NSData *decryptedMessage = [BBAES decryptedDataFromString:#"RlVnd01XOE5teTNseDFGQ3JScVhkQT09" IV:salt key:key];
NSLog(#"Decrypted message: %#", decryptedMessage);
The log only shows a null object now.
I have found a duplicate post for C#: How to decrypt an AES-256-CBC encrypted string
EDIT:
Lets say that i can adjust the encoding in PHP. How should I encrypt the string in PHP to be decrypted in Objective C?
You are not doing the same thing in PHP as in iOS. I am not familiar with this BBAES framework, but what you seem to have is a password from which you are generating a 256 bit AES key using PBKDF key derivation, and using that to decrypt the data.
However, in PHP you are hashing your password and using it to encrypt your data, so you are probably using different AES keys for encryption and decryption. And I am not sure that IVs match either.
What you should do is:
In PHP, generate a random 16 byte IV for every encryption you do and use PBKDF key derivation to generate the 256 bit AES key from your password. Keep in mind that the salt and the number of iterations have to be the same in both PHP and iOS. After the encryption, append the IV to the encrypted data and send it.
In iOS, extract the IV from the received ciphertext (the last 16 bytes), generate the AES key from your password the same way you did before using the same salt and number of iterations, and decrypt the data (without the 16 byte IV at the end)
Edit:
As #Zaph pointed out, I forgot to mention that you should use also the same type of padding. BBAES seem to use PKCS7 padding.
To decrypt in Objective C you can use Apples's version of the CommonCrypto C library. It has a man page and there are already several posts that show decryption examples on Stack Overflow for example:
Determine if key is incorrect with CCCrypt kCCOptionPKCS7Padding-Objective C
which comes from the tutorial here:
http://robnapier.net/aes-commoncrypto
This also really helped me:
CCCrypt decrypting in AES CBC works even without IV
If you have trouble getting it working post some code.
Dwolla permits an app to solicit and store a user's PIN as a form of pre-authorization, but requires that it be encrypted. From the TOS:
PIN(s) must be encrypted in transit and at rest (this includes any and
all backup mediums) using FIPS 140-2 standards (at a minimum)
Normally, I'd use Bcrypt to encrypt (actually, make a secure hash. Neil Slater, thanks for the correction) something (using bcrypt-ruby gem), such as a password. But if I encrypt with Bcrypt, then I'd have to transmit the hash, and of course that won't match what Dwolla is expecting and the PIN will be rejected.
How do you encrypt the PIN and unencrypt it for secure transmittal?
UPDATE:
One of the answers in the question that Andrew links to below referenced OpenSSL:Cipher, and using that I can encrypt the PIN with the below code. But remaining questions then are:
How should I store the key, iv (initialization vector), and cipher? Is it secure to save as environment variables, or would it be better to put in a database table in a secure hash?
Does the below code make sense as a way to encrypt the PIN?
Since I don't have a public key from Dwolla, what's the best way to transmit it?
pin = "1111" # this is what needs to be encrypted
#encryption:
cipher = OpenSSL::Cipher.new('AES-128-CBC') #=> #<OpenSSL::Cipher:0x00000100ef09d8>
cipher.encrypt
key = cipher.random_key #=> odd characters...
iv = cipher.random_iv #=> odd characters...
encrypted = cipher.update(pin) + cipher.final #=> odd characters...
#dcryption:
decipher = OpenSSL::Cipher::AES.new(128, :CBC)
decipher.decrypt
decipher.key = key
decipher.iv = iv
plain = decipher.update(encrypted) + decipher.final
puts plain == pin #=> true
So this is what I've found out. In Rails, generate the key just once and store as an environment variable (and when you deploy encrypt it). Generate a new iv (initialization vector) for each pin. Store the iv and the encrypted pin in the database.
You may want to convert the encrypted PIN and the IV to UTF8 in order to successfully save without changing how you set up your database. (Be default, they'll be generated as ASCII 8-bit).
Here is one way to do it inside your User model, but you may want to refactor since these are large methods:
def dwolla_pin # => this is to decrypt the PIN in order to use it
unless encrypted_dwolla_pin.nil?
decipher = OpenSSL::Cipher::AES.new(128, :CBC)
decipher.decrypt
decipher.key = ENV["ENCRYPT_KEY"]
# Convert IV from UTF8 (as stored) back to ASCII-8bit (for OpenSSL)
utf8_iv = self.iv_for_pin
decipher.iv = Base64.decode64(utf8_iv.encode('ascii-8bit'))
# Convert PIN from UTF8 (as stored) back to ASCII-8bit (for OpenSSL)
utf8_pin = self.encrypted_dwolla_pin
ascii_pin = Base64.decode64(utf8_pin.encode('ascii-8bit'))
dwolla_pin ||= decipher.update(ascii_pin) + decipher.final
end
end
def dwolla_pin=(new_pin) # => this is to encrypt the PIN in order to store it
return false unless valid_pin?(new_pin)
cipher = OpenSSL::Cipher.new('AES-128-CBC')
cipher.encrypt
cipher.key = ENV["ENCRYPT_KEY"]
# Create IV and convert to UTF-8 for storage in database
iv = cipher.random_iv
utf8_iv = Base64.encode64(iv).encode('utf-8')
self.update_attribute(:iv_for_pin, utf8_iv)
# Encrypt PIN and convert to UTF-8 for storage in database
encrypted_pin = cipher.update(new_pin) + cipher.final
utf8_pin = Base64.encode64(encrypted_pin).encode('utf-8')
self.update_attribute(:encrypted_dwolla_pin, utf8_pin)
end
def valid_pin?(pin) # => Here I'm just checking to make sure the PIN is basically in the right format
pin.match(/^\d{4}/) && pin.length == 4
end
"Secure transit" means SSL for usage and SSH for deployment. If deploying to Heroku then already using SSH, but for SSL you will need to buy from your DNS host wildcard cert and the ssl endpoint on Heroku.
Does anyone have anything to add to this?
I'd use public/private key encryption in a case like this. Not an expert on Ruby, but this link might help:
Ruby: file encryption/decryption with private/public keys
If your pin is being sent externally then you'd need the end-users public key to encrypt. If this isn't possible then you could use a mixture of asynmmetric (public/private) and symmetric algorithms - basically what SSH does.
http://en.wikipedia.org/wiki/Secure_Shell
I'm having serious issues trying to parse some CSV in rails right now.
Basically my app gets a user to upload a CSV file. The app then converts the file to ensure it is in UTF-8 format, then attempts to parse it and process it. Whenever the app attempts to parse it however, I get the MalformedCSVError stating "Illegal quoting on line 1"
Now what I don't get, is if I copy the original file into a new document and save it, then I can parse it on a rails console without a problem.
If I attempt to parse the original file, it complains about an invalid character for UTF-8 encoding (the file isn't in UTF-8 hence the app converts it)
If I attempt to parse the file which the app has converted to UTF-8 and changed the line endings to LF, it fails to parse.
If I do a file diff between the version the app has produced, and the copy/paste version that I have made (which works) there are 0 differences so I really can't figure out why one is parsable, and one is not.
Any suggestions? My app is processing the file as follows :
def create
#survey = Survey.new(params[:survey])
# Now we need to try and convert this to UTF-8 if it isn't already
encoded = File.read(#survey.survey_data.current_path)
encoding = CharlockHolmes::EncodingDetector.detect(encoded)
# We've got a guess at the encoding,
# so we can try and convert it but it
# may still fail so we need to handle
# that
begin
re_encoded = CharlockHolmes::Converter.convert(encoded, encoding[:encoding], 'UTF-8')
re_encoded = re_encoded.gsub(/\r\n?/, "\n")
# Now replace the uploaded file
File.open(#survey.survey_data.current_path, 'w') { |f|
f.write(re_encoded)
}
rescue ArgumentError
puts "UH OH!!!!!"
end
puts "#{#survey.survey_data.current_path}"
#parsed = CSV.read(#survey.survey_data.current_path)
end
The file uploading gem is CarrierWave if that makes any difference.
Please can someone help me as this is driving me insane!
Edit
The error says it's on line 1. Line 1 (assuming it doesn't index from 0) is
"Survey","RD","GarrysMDs","NigelsMDs","PaulsMDs","StephensMDs","BrinleyJ","CarolineP","DaveL","GrantR","GregS","Kent","NeilC","NicolaP","AndyC","DarrenS","DeanB","KarenF","PaulR","RichardF","SteveG","BrianG","GordonA","NickD","NickR","NickT","RayL","SimonH","EdmondH","JasonF","MikeS","SamanthaN","TimB","TravisF","AlanS","Q1","Q2","Q3","Q4","Q5","Q6","Q7","Q8PM","Q8N","Q9","Q10","Q11","Q12","Q13","Q14","Q15","Q16PM","Q16N","Q17PM","Q17N","Q18PM","Q18N","Q19","Q20","Q21","Q22","comment","Q23.1","Q23.2","Q23.3","TQ23.1","TQ23.2","VPM","VN","VQ1","VQ2","VQ3","VQ4","VQ5","VQ6","VQ7","VQ8N","VQ8PM","VQ9","VQ10","VQ11","VQ12","VQ13","VQ14","VQ15","VQ16","VQ16N","VQ16PM","VQ17","VQ17N","VQ17PM","VQ18","VQ18N","VQ18PM","VQ19","VQ20","VQ21","VQ22","VQ23.1","VQ23.2","VQ23.3","VRD","XQ16","XQ17","XQ18"
Well that was irritating!
Turns out the file had a BOM which was causing the CSV parser to break. Loading the file with
CSV.open("path/to/file.csv", "rb:bom|encoding")
allowed it to parse it perfectly! So annoyed how long it took to track down but it's now working and with no need to convert to UTF-8 now either!