My application accepts an uploaded file from the user and parses it, making use of seek and rewind methods quite heavily to parse blocks from the file (lines can begin with 'start' or 'end' to enclose a section of data, etc).
A new requirement allows the user to upload encrypted files. I've implemented decryption of the content of the file and return the content string to the existing method. I can parse the string as a CSV but lose the file controls.
Storing an unencrypted version of the file is not an option for business reasons.
I'm using FasterCSV but not averse to using something else if I can keep the seek/rewind behaviour.
Current code:
FasterCSV.open(path, 'rb') do |csv| # Can I open a string as if it were a file?
unless csv.eof? # Catch empty files
# Read, store position, seek, rewind all used during parsing
position = csv.pos
row = csv.readline
csv.seek(pos)
After some digging and experimentation I've found that it was possible to retain the IO methods by using the StringIO class like so:
csv = StringIO.new(decrypted_content)
unless csv.nil?
unless csv.eof? # Catch empty files
position = csv.pos
row = csv.readline.chomp.split(',')
csv.seek(pos)
Only change is needing to manually split the line to be able to use it like a csv row, not much extra work.
You don't need the CSV gem anymore but if you prefer the seek/rewind behaviour you can roll your own for strings. Something like this might work for your scenario:
array_of_lines=unecrypted_file_string.split('\n')
array_of_lines.each_with_index do |line,index|
position=index
row=line
seek=line[10]
end
Related
I'm managing to record audio using the recorderjs. It creates the audio. I'm using ruby rails. I save it. It sends the file correctly to my server. When I go to test and see if the audio is the same. It doesn't play and has a play time of 0 seconds.
I want to record an audio file that is 2-5 seconds long. then save that file on the server and it will be linked to a word.
File.open('public/mp3s/' + filename, 'wb') do |f|
f.write(Base64.decode64(params[:mp3data]))
end
Is there something wrong with my write method here?
It is likely that you have an encoding error, or a typo generating an empty file. The param could be a binary octet stream instead of base64 for instance.
Or it could be coming in as a different key, in which case you'd make an empty file.
If you're on mac or linux, try using the file command on the written file. The command should look at the magic byte, and if it is an MP3 you should see something along those lines.
Likewise, if you cat, hexdump, or notepad the file that rails saved, and the generated one you should be able to see any differences.
params[:mp3data] may also be nil
in development mode, the rails server should list the incoming params.
thank you. it was coming back as nil. so it brought me to check the file and if it had any data in it. so i created a size for it along with params. I figured it out and it is now working! '
metadata_size = "data:audio/mp3;base64,".length data = params[:mp3data][metadata_size, params[:mp3data].length] File.open('public/mp3s/' + filename, 'wb') do |f| f.write(Base64.decode64(data)) end
#EnabrenTane
I have the following code:
sftp = Net::SFTP.start('ftp.test.com','luigi', :password => 'pass_code')
data = sftp.download!("luigi/List.csv")
This converts data to a string like so:
2015,This is, one value, test, 9820\r\n4003, This is also one value, test, 0393
I want to separate each row in my CSV file into an array value. This is my attempt:
data.split(/\r\n/)
data.each do |record|
record.split(/,/)
end
This works for most cases, but it obviously gets screwed up for the first row in my csv file since the additional comma creates another value in my array:
2015,This is, one value, test, 9820
Because there is an additional comma that is not escaped, my array looks like this:
["2015","This is", "one value","test","9820"]
I want the above to look like this:
["2015","This is, one value","test","9820"]
or even
["2015","This is one value","test","9820"]
Is it possible, using sftp, to download my file into memory (not locally, I have nowhere to store it) and loop through each row in the file without the additional commas throwing things off? It seems that the root of the issue is I can't do something like:
CSV.foreach(...) do |csv_row|
But instead I have to use sftp.download! which only returns one string, not an array of "csv rows".
I am using paperclip to attach an excel file to a mode.
The purpose is to import the data from the excel file to the database.
Model: Import
has_attached_file: spreadsheet
For the import process, I want to access the file in my model as following.
path = "#{Rails.root}/public/#{spreadsheet.url}"
This doesn't work. I guess because the url has the timestamp at the end.
In general what is the best way to access the attachments on server side ?
I think you're looking for the to_file method. You should be able to do something like this:
excel_file = self.spreadsheet.to_file
which will either return the uploaded file from the server (if you're using s3 or remote storage), or if it has been assigned to the model but not actually stored yet (if you haven't called model.save since it was uploaded), it returns the temp file stored on disk.
From there you should be able to use an excel gem or library to parse the contents.
Alternatively, you can use spreadsheet.url(nil, false) - the second parameter denotes whether or not to append a timestamp.
#spreadsheet.path
http://rdoc.info/gems/paperclip/2.3.8/Paperclip/Attachment#path-instance_method
I have generated a html table from my web application and save the table into .xls format(in a single word i am generating a .xls sheet from my web application ).
What other setting I have to show it in table form.
You are not producing an XLS file, you are producing a mal-formed HTML file with a name that ends in .xls.
Indeed, you aren't even doing that since there aren't files on the web (there are streams that may or may not end up in files).
Different versions of Open Office, with different settings, will differ in terms of how they deal with stuff that is wrong. The version on one of the machines you are doing is saying "eh, this isn't XLS, oh! it's HTML with a table, I know what to do", while the other is getting as far as "eh, this isn't XLS, it's a bunch of text with strange less-than and greater-than characters all over the place, what do I do".
What you want to do is to produce an actual stream that Open Office and other spreadsheets can deal with. XLS is possible, but pretty hard. Go for CSV instead.
If your table was going to be:
<table>
<tr>
<th>1 heading</th><th>2 & last heading</th>
</tr>
<tr>
<td>1st cell</td><td>This is the "ultimate" cell</td>
</tr>
</table>
Then it sould become:
"1 heading","2 & last heading"
"1st cell","This is the ""ultimate"" cell"
In otherwords newlines to indicate rows, commas to indicate cells, no HTML encoding, quotes around everything and quotes in your actual content doubled-up. (You don't need to always have quotes on your content, but it's never wrong so that's simpler than working out when you do need them).
Now, make your content type "text/csv".
You are now outputting a CSV stream that can be saved as a CSV file. Your spreadsheet software will have a much better idea about what to do with this (it may still ask about character ecodings on opening, but the preview will show you a spreadsheet of data, not a bunch of HTML source all over the place.
It's not really saving as a .xls file -- it appears to be saving as the HTML, but with a .xls extension. How are you generating the .xls? On the server-side, you can provide a button to generate .xls directly (different methods depending on your server platform -- using perl there is the Spreadsheet::WriteExcel module that writes .xls directly, using Java there is JExcel (http://jexcelapi.sourceforge.net/ and POI (http://poi.apache.org/)), other platforms will have their methods.
Okay Subodh, If you want to generate .xls or .csv files, You can't just change the extension of the file and have it open up correctly in that program.
2 Options you have at this point, both involve creating the file with the data on the server and then sending it to the user to download it.
.csv
CSV files are easier to generate from the server side. In a very basic way you can think of them as regular text files with commas(not necessarily only commas) separating individual cells that can be read by spreadsheet programs. For PHP there is an article Here that explains how to generate CSV files.
.xls
xls files are not as simple as simple to generate as CSV files. On the server-side you will need a solution to generate these. For PHP there is a resource Here.
Using xls over CSV has obvious advantage that you can specify formatting and can control visual representation of your data.
Edit :
Upon closely looking at the image you posted, I can see what you are trying to do. If you just want to get that file to open correctly in a spreadsheet program, then don't save it either as CSV or xls
hello.html
<table>
<tr><td>Hi</td><td>Hi</td><td>Hi</td><td>Hi</td></tr>
<tr><td>2</td><td>2</td><td>131</td><td>11312</td></tr>
</table>
Saved as an HTML file will open up correctly(as a proper table) in any spreadsheet program.
To narrow down the problem:
1) Are you opening the same .xls file on both machines?
- what version of OpenOffice is on Machine 1?
- what version of OpenOffice is on Machine 2?
2) How are you creating your .xls file?
- are you just using the response object to change the content-type, or some proprietary software?
- can you include a code sample?
3) Have you tried a pure HTML format?
In my Rails application Admin can export the user data into csv format. Every user in my application has their profile photo.... my client wants to include the user photo into the CSV file .. I have not how to do this . can any one please help me....
i am using fastercsv gem and here is some my controller code for reference
In my Controller :
require 'fastercsv'
def getcsv
entries = User.find(:all)
csv_string = FasterCSV.generate do |csv|
csv << ["first_name","last_name","username","address","photo" ]
entries.each do |e|
csv << [e.first_name,e.last_name,e.username,e.address,e.photo.url]
end
end
send_data csv_string,:type=>'text/csv;charset=iso-8859-1; header=present', :filename=>"entries.csv",:disposition => 'attachment'
end
Saving the actual photo in a CSV file is technically possible, but very ill-advised. CSV is simply not intended for that sort of job. You obviously cannot simply embed the binary image data into the ASCII text-based CSV file. It would be possible to use Base-64 encoding to convert the 8-bit binary data into 7-bit text, and then store this in one of the fields in your CSV file. (This will also expand the storage required by about 20%).
But then what software could decode this? You would have to write some other software to convert the images back on the other end, which would defeat the purpose. Furthermore, each line of the CSV file would be massive, and probably cause problems on importing.
You would be much better off exporting the profile photos as a set of PNGs, and save the filename in the CSV file against each record.
CSV is plain text. There's no way to include graphic data unless both the generator and the reader agree on a format, such as base64-encoded PNG.
You could try adding
links to image files
single line base64 encoded string
CSV (comma separated values) is not a format suitable for binary data, the general rule of thumb though for saving binary data to a text file is to convert it to a format that does suit text files, something like Base64, or a textual representation of the raw data in Hex.