Carrierwave & Zipfiles: Using an extracted file as a version - ruby-on-rails

Something I'm not getting about the version process...
I have a zip file with a file inside, and I want to upload the file as a "version" of the zip:
Uploader:
version :specificFile do
process :extract_file
end
def extract_file
file = nil
Zip::ZipFile.open(current_path) do |zip_file|
file = zip_file.select{|f| f.name.match(/specificFile/)}.first
zip_file.extract(file, "tmp/" + file.name.gsub("/", "-")){ true }
end
File.open("tmp/" + file.name.gsub("/", "-"))
end
Usage:
=link_to "Specific File", instance.uploader.specificFile.url
Only this just nets me two copies of the zip. Clearly, there's something I'm missing about how version / process works, and I haven't been able to find documentation that actually explains the magic.
So how do I do this, and what am I missing?

This provided the "why", although it took a bit to understand:
How do you create a new file in a CarrierWave process?
To rephrase, when you go to create a version, carrierwave makes a copy of the file and then passes the process the file path. When the process exits, carrierwave will upload the contents of that path - not the file the process returns, which is what I thought was going on.
Working code:
version :specificFile do
process :extract_file
def full_filename (for_file = model.logo.file)
"SpecificFile.ext"
end
end
def extract_plist
file = nil
Zip::ZipFile.open(current_path) do |zip_file|
file = zip_file.select{|f| f.name.match(/specificFile/)}.first
zip_file.extract(file, "tmp/" + file.name.gsub("/", "-")){ true }
end
File.delete(current_path)
FileUtils.cp("tmp/" + file.name.gsub("/", "-"), current_path)
end
So, to make what I want to happen, happen, I:
Tell carrierwave to use a particular filename. I'm using a hardcoded value but you should be able to use whatever you want.
Overwrite the contents of current_path with the contents you want under the version name. In my case, I can't just overwrite the zip while I'm "in it" (I think), so I make a copy of the file I care about and overwrite the zip via File and FileUtils.
PS - It would be nice to avoid the duplication of the zip, but it doesn't look like you can tell carrierwave to skip the duplication.

Related

Rails FTP OPEN CSV

I have the following code to connect my rails app to my FTP. This works great. However, I want to use open-uri to open the csv file so I can parse it. Any ideas how to do this? I think it's an easy thing to do but I'm missing something.
require 'net/ftp'
ftp = Net::FTP.new
ftp.connect("xxx.xxx.xx.xxx",21)
ftp.login("xxxxx","xxxx")
ftp.chdir("/")
ftp.passive = true
puts ftp.list("TEST.csv")
You'll need to use #gettextfile.
A) Get the file to a local temporary file and read its content
# Creating a tmp file can be done differently as well.
# It may also be omitted, in which case `gettextfile`
# will create a file in the current directory.
Dir::Tmpname.create(['TEST', ['.csv']) do |file_name|
ftp.gettextfile('TEST.csv', file_name)
content = File.read(file_name)
end
B) Pass a block to gettextfile and get the content one line at a time
content = ''
ftp.gettextfile('TEST.csv') do |line|
content << line
end

Using Tempfile to create a zip file in rails

I want to create a temporary .zip in rails. For creating zip file I am using rubyzip gem.
Currently I am doing this:
zfname = Tempfile.new(['somename','.zip'], Rails.root.to_s + '/tmp/')
Zip::ZipFile.open(zfname.path, Zip::ZipFile::CREATE) do |zipfile|
zipfile.add(file, basepath + file)
end
This generates following error:
Zip::ZipError: Zip end of central directory signature not found
Is it possible to use Tempfile for zip? If yes, what is wrong here?
In my rails app when I needed to send zip files to user, I just stored them in a buffer and used 'send data' method in controller.
I was trying to use 'Tempfile' initially but it had an added task of removing zip files after sending it to user which is a pain.
Is this something you are looking for?
Zip::OutputStream.write_buffer do |stream|
file_paths.each_with_index do |file_path, index|
# rename the pdf
stream.put_next_entry("#{name}-#{index + 1}.pdf")
# add pdf to zip
stream.write IO.read(file_path)
end
end
So it doesn't actually look like you need or want a Tempfile. You really want a random path in the Rails.root/tmp directory. Try something like this:
zfpath = Rails.root.join('tmp', "somename-#{SecureRandom.hex(8)}.zip"
Zip::ZipFile.open(zfpath, Zip::ZipFile::CREATE) do |zipfile|
zipfile.add(file, basepath + file)
end
Update:
While it's far more complex, you can find a discussion of how to do this with a Tempfile here - http://thinkingeek.com/2013/11/15/create-temporary-zip-file-send-response-rails/ .

Not able to create a file for writing

I am uploading txt files using carrierwave. The files are not small (80 MB - 500 MB) and I want to remove some of the lines to reduce this size (about 80% of the file size is going to be reduced).
I have created a model method in order to clear these lines:
require 'fileutils'
def clear_unnecessary_lines
old_file_path = Rails.root.join('public').to_s + log_file.to_s
new_file_path = old_file_path.sub! '.txt', '_temp.txt'
File.open(old_file_path, 'r') do |old_file|
File.open(new_file_path, 'w') do |new_file|
old_file.each_line do |line|
new_file.write(line) unless line.grep(/test/).size > 0
end
end
end
FileUtils.mv new_file_path, old_file_path
end
but I am getting error when I am trying to open the new file saying there is no such file. As I have read opening a file with the w option should create an empty file for writing. Then why I am getting such error?
Also, since log_file column is holding the path to the original file, and I am changing it, could you tell how to rename the new file with the old name? As I have checked I should specify only old and new names, not paths.
Errno::ENOENT: No such file or directory - /home/gotqn/Rails/LogFilesAnalyser/LogFilesAnalyser/public/uploads/log_file/log_file/3/log_debug_temp.txt
It is strange that If I execute the following command in rails console, it is not throwing an error and the file is created.
File.open('/home/gotqn/Rails/LogFilesAnalyser/LogFilesAnalyser/public/uploads/log_file/log_file/3/log_debug_temp.txt','w')
Ah, i see your problem now. When you do this
new_file_path = old_file_path.sub! '.txt', '_temp.txt'
you call the "self-altering" version of sub, ie sub!. This will actually change the value of old_file_path as a side effect. Then, in the next line, you try to open this file, which hasn't been created yet. Take out the exclamation mark and you should be fine.

Rubyzip: Export zip file directly to S3 without writing tmpfile to disk?

I have this code, which writes a zip file to disk, reads it back, uploads to s3, then deletes the file:
compressed_file = some_temp_path
Zip::ZipOutputStream.open(compressed_file) do |zos|
some_file_list.each do |file|
zos.put_next_entry(file.some_title)
zos.print IO.read(file.path)
end
end # Write zip file
s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
bucket.put("#{BUCKET_PATH}/archive.zip", IO.read(compressed_file), {}, 'authenticated-read')
File.delete(compressed_file)
This code works already but what I want is to not create the zip file anymore, to save a few steps. I was wondering if there is a way to export the zipfile data directly to s3 without having to first create a tmpfile, read it back, then delete it?
I think I just found the answer to my question.
It's Zip::ZipOutputStream.write_buffer. I'll check this out and update this answer when I get it working.
Update
It does work. My code is like this now:
compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
some_file_list.each do |file|
zos.put_next_entry(file.some_title)
zos.print IO.read(file.path)
end
end # Outputs zipfile as StringIO
s3 = Aws::S3.new(S3_KEY, S3_SECRET)
bucket = Aws::S3::Bucket.create(s3, S3_BUCKET)
compressed_filestream.rewind
bucket.put("#{BUCKET_PATH}/archive.zip", compressed_filestream.read, {}, 'authenticated-read')
The write_buffer returns a StringIO and needs to rewind the stream first before reading it. Now I don't need to create and delete the tmpfile.
I'm just wondering now if write_buffer would be more memory extensive or heavier than open? Or is it the other way around?

Rails how to delete a file without failing on error

I'm using JPEGCAM to allow users to take a profile pic with their web cam. This uploads a temporary file as so:
def ajax_photo_upload
File.open(upload_path, 'w:ASCII-8BIT') do |f|
f.write request.raw_post
end
# #user.photo = File.open(upload_path)
#user.assign_attributes(
:photo => File.open(upload_path),
:orig_filename => "#{current_user.full_name}.jpg"
)
if #user.save
respond_to do |format|
.....
private
def upload_path # is used in upload and create
file_name = session[:session_id].to_s + '.jpg'
File.join(::Rails.root.to_s, 'public', 'temp', file_name)
end
What's the best way to go about deleting this temporary file safely? Thanks
When you know that you are done with the file:
File.delete(path_to_file) if File.exist?(path_to_file)
Another thing: make sure that you always close files that you have opened, an operating system can only handle a certain number of open files/file descriptors and you'll may run into strange bugs when you pass that limit... So when you want to open files in Ruby always either use the block form:
File.open(path) do |f|
# ...
end
and Ruby will close the file automatically for you. If the block form is not usable, you have to close files by yourself:
f = File.open(path)
# ...
f.close
So make sure to close the file that you pass to #user.assign_attributes(...)...
If you are sure you are done with it, why not just use FileUtils.rm or FileUtils.rm_f?
FileUtils.rm_f(upload_path)
http://www.ruby-doc.org/stdlib-1.9.3/libdoc/fileutils/rdoc/FileUtils.html#method-c-rm_f
You could also ignore this in Rails, and have a cron that wakes up and deletes files older than a day from the temp directory that match these temp files. That has the benefit of some margin for error if a file fails to be reprocessed - you don't rm it immediately - and the file operation is not done on the request/response loop for Rails, which will then respond a bit faster.

Resources