I am trying to download a zip file, extract the zip and read the files. Below is my code snippet:
url = "http://localhost/my.zip"
response = RestClient::Request.execute({:url => url, :method => :get, :content_type => 'application/zip'})
zipfile = Tempfile.new("downloaded")
zipfile.binmode #someone suggested to use binary for tempfile
zipfile.write(response)
Zip::ZipFile.open(zipfile.path) do |file|
file.each do |content|
data = file.read(content)
end
end
When I run this script, I see below error:
zip_central_directory.rb:97:in `get_e_o_c_d': Zip end of central directory signature not found (Zip::ZipError)
I am not able to understand what this error is for ? I can download and view the zip from the zip file url.
Couldn't get the download to work with Restclient so I used net/http instead, tested and works. Working with tempfiles and Zip gave me trouble in the past so I rather use a normal file. You can delete it afterwards.
require 'net/http'
require 'uri'
require 'zip/zip'
url = "http://localhost/my.zip"
uri = URI.parse(url)
req = Net::HTTP::Get.new(uri.path)
filename = './test.zip'
# download the zip
File.open(filename,"wb") do |file|
Net::HTTP::Proxy(proxy_host, proxy_port, proxy_user, proxy_pass).start(uri.host, uri.port) do |http|
http.get(uri.path) do |str|
file.write str
end
end
end
# and show it's contents
Zip::ZipFile.open(filename) do |zip|
# zip.each { |entry| p entry.get_input_stream.read } # show contents
zip.each { |entry| p entry.name } # show the name of the files inside
end
I suspect you have a corrupted zip.
Unzip cannot find the line of code that mark the end of the archive, so either:
The archive is corrupt.
It is not a .zip archive.
There are more than one parts to the archive.
Related
I'm trying to create a function that would download a zip that contains files. I followed this tutorial, that says that my code should look like the following if you want to download a CSV (I translated the comments) :
def exporter_zip
# CSV's header
csv = "Title;Content;Publishing date;File name;\n"
# Loop over articles saved in database
Article.all.each do |article|
# Creating the CSV with datas
csv += "#{article.title};#{article.content};#{article.publishing_date.strftime("%Y-%m-%d") if article.publishing_date };#{article.file_name}\n"
end
# Creating the zip file inside the folder
zip_tmp = File.new("#{Rails.root}/db/mon_fichier.zip", "w+")
# opening the file in writing mode
Zip::File.open(zip_tmp.path, Zip::File::CREATE) {
|zipfile|
# Inserting the csv variable's content inside the article.csv file, which is inserted into the zip file
zipfile.get_output_stream("articles.csv") { |f| f.puts csv }
}
# Sending the created file
send_file "#{Rails.root}/db/mon_fichier.zip"
end
Here is how i adapted the code:
class DownloadController < ApplicationController
require 'zip'
def zip
# Creating zip file
zip_tmp = File.new("#{Rails.root}/public/zip-#{Time.now.strftime('%d-%m-%Y')}.zip", 'w+')
FileDetail.all.each do |fichier|
Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
}
end
send_file "#{Rails.root}/public/zip-#{Date.today.to_time}.zip"
end
end
However, while I'm not even sure I'm doing this correctly, I get the following error: cannot open entry for reading while its open for writing, targetting the following line: Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
Can anyone can tell me what's going on? I never did this before so I don't know what went wrong..
Thank you in advance
FileDetail.all.each do |fichier|
Zip::File.open(zip_tmp.path, Zip::File::CREATE) { |zipfile|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
}
end
Doing it this way will attempt to create the zip file for each member of your FileDetail. You should code it to only open and create once:
Zip::File.open(zip_tmp.path, Zip::File::CREATE) do |zipfile|
FileDetail.all.each do |fichier|
# fichier.filename => file.mp3
# fichier.path => path/to/file.mp3
zipfile.get_output_stream(fichier.filename, fichier.path)
end
end
I'm exporting some data from my server to the client.
It's an zip archive but when the amount of data is to big : TimeOut !
#On my controller
def export
filename = 'my_archive.zip'
temp_file = Tempfile.new(filename)
begin
Zip::OutputStream.open(temp_file) { |zos| }
Zip::File.open(temp_file.path, Zip::File::CREATE) do |zip|
#videos.each do |v|
video_file_name = v.title + '.mp4'
zip.add(video_file_name, v.source.file.path(:original))
end
end
zip_data = File.read(temp_file.path)
send_data(zip_data, :type => 'application/zip', :filename => filename)
ensure
temp_file.close
temp_file.unlink
end
end
I'm using PaperClip to attach my video on my app.
Is there any way to create and upload the zip (with a stream?) without a too long wait?
You could try the zipline gem. It claims to be "Hacks on Hacks on Hacks" so heads up! Looks very easy to use though, worth a shot.
I am having an issue with my rails action that takes binary (blob) files from my database and packages them up into a nice zip file and then finally sends it out for download. The problem occurs when I try to unzip the file, it says something like "Unable to expand; Error 1 - Operation not permitted." I believe that means that the file is corrupted but I don't know what I am doing wrong. I've included my code below, any help would be greatly appreciated. Thanks!
require 'zip/zip'
require 'zip/zipfilesystem'
def export
#layers = Layer.where('group_id > 1')
temp = Tempfile.new("layers-zip-export")
Zip::ZipOutputStream.open(temp.path) do |zipfile|
#layers.each do |layer|
zipfile.put_next_entry(layer.name)
file = Tempfile.new("temp-" + layer.id.to_s)
file.binmode
file << layer.file
file.rewind
zipfile.write IO.binread(file.path)
file.close
file.unlink
end
end
send_file temp.path, :type => 'application/zip', :filename => "layer-export.zip"
temp.close
end
I am using a supplier's api and the response they send to our server includes a url to a file, upon trying to save this file locally I fail miserably.
def self.create_file_new(filename, ext, url)
require 'open-uri'
file = Tempfile.new(filename + ext)
file.binmode
# data = open(url).read
# data = open(url, :http_basic_authentication => [username, password])
file << open(url, :http_basic_authentication => [username, password]).read
# file.write CGI::unescape(data)
file.close
file = File.open(file.path)
return file
end
I was originally getting a OpenURI::HTTPError (401 Unauthorised): but I have since created a file named bypass_ssl_verification_for_open_uri in app/initializers containing the following:
# Make open-uri work with https
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE
which I found whilst Googling on how to fix it.
I then started to get this error message: NoMethodError (undefined method 'tr' for #<StringIO:0xb5b728c4>):, I tried creating another file (cgi_escape_fix.rb in app/initializers) containing this:
require 'cgi'
class << CGI
alias_method :orig_escape, :escape
def escape(str)
orig_escape(str.to_str)
end
end
Which I also found on my Google travels but that doesn't seem to have solved anything, so I commented out the file.write CGI::unescape(data) to try a different way but still no joy.
Now in the log I am just getting a plain 500 Internal Server Error with no useful information.
The file I'm attempting to save will always be a pdf.
Ruby 1.8.7
Rails 2.3.14
Got it to work with the following (two new initializer scripts removed):
file = Tempfile.new(filename + ext)
file.binmode
file << open(url, :http_basic_authentication => [username, password]).read
file.close
file = File.open(file.path)
return file
Should also mention that this is being passed to the attachment_fu plugin incase anyone else has problems with it.
I have a small Rails 3.2.1 app that uses CarrierWave 0.5.8 for file uploads to S3 (using Fog)
I want users to be able to select some images that they'd like to download, then zip them up and send them a zip. Here is what I've come up with:
def generate_zip
#A collection of Photo objects. The Photo object has a PhotoUploader mounted.
photos = Photo.all
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
file_to_add = photo.photo.file
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
#do the rest.. like send zip or upload file and e-mail link
end
This doesn't work because photo.photo.file returns an instance of CarrierWave::Storage::Fog::File instead of a regular file.
EDIT: The error this leads to:
Errno::ENOENT: No such file or directory - uploads/photos/name.jpg
I also tried the following:
tmp_filename = "#{Rails.root}/tmp/" << Time.now.strftime('%Y-%m-%d-%H%M%S-%N').to_s << ".zip"
zip = Zip::ZipFile.open(tmp_filename, Zip::ZipFile::CREATE)
zip.close
photos.each do |photo|
processed_uri = URI.parse(URI.escape(URI.unescape(photo.photo.file.authenticated_url)).gsub("[", "%5B").gsub("]", "%5D"))
file_to_add = CarrierWave::Uploader::Download::RemoteFile.new(processed_uri)
zip = Zip::ZipFile.open(tmp_filename)
zip.add("tmp/", file_to_add.path)
zip.close
end
But this gives me a 403. Some help would be greatly appreciated.. It probably is not that hard I'm just Doing it Wrong™
I've managed to solve the problem with help from #ffoeg
The solution offered by #ffoeg didn't work quite so well for me since I was dealing with zip files > 500 MB which caused me problems on Heroku. I've therefor moved the zipping to a background process using resque:
app/workers/photo_zipper.rb:
require 'zip/zip'
require 'zip/zipfilesystem'
require 'open-uri'
class PhotoZipper
#queue = :photozip_queue
#I pass
def self.perform(id_of_object_with_images, id_of_user_to_be_notified)
user_mail = User.where(:id => id_of_user_to_be_notified).pluck(:email)
export = PhotoZipper.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
Notifications.zip_ready(export.archive_url, user_mail).deliver
end
# Zipfile generator
def self.generate_zip(id_of_object_with_images, id_of_user_to_be_notified)
object = ObjectWithImages.find(id_of_object_with_images)
photos = object.images
# base temp dir
temp_dir = Dir.mktmpdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
# This assumes the ObjectWithImages object has an attribute title which is a string.
zip_path = File.join(temp_dir, "#{object.title}_#{Date.today.to_s}.zip")
Zip::ZipOutputStream.open(zip_path) do |zos|
photos.each do |photo|
path = photo.photo.path
zos.put_next_entry(path)
zos.write photo.photo.file.read
end
end
#Find the user that made the request
user = User.find(id_of_user_to_be_notified)
#Create an export object associated to the user
export = user.exports.build
#Associate the created zip to the export
export.archive = File.open(zip_path)
#Upload the archive
export.save!
#return the export object
export
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end
end
app/controllers/photos_controller.rb:
format.zip do
#pick the last ObjectWithImages.. ofcourse you should include your own logic here
id_of_object_with_images = ObjectWithImages.last.id
#enqueue the Photozipper task
Resque.enqueue(PhotoZipper, id_of_object_with_images, current_user.id)
#don't keep the user waiting and flash a message with information about what's happening behind the scenes
redirect_to some_path, :notice => "Your zip is being created, you will receive an e-mail once this process is complete"
end
Many thanks to #ffoeg for helping me out. If your zips are smaller you could try #ffoeg's solution.
Here is my take. There could be typos but I think this is the gist of it :)
# action method, stream the zip
def download_photos_as_zip # silly name but you get the idea
generate_zip do |zipname, zip_path|
File.open(zip_path, 'rb') do |zf|
# you may need to set these to get the file to stream (if you care about that)
# self.last_modified
# self.etag
# self.response.headers['Content-Length']
self.response.headers['Content-Type'] = "application/zip"
self.response.headers['Content-Disposition'] = "attachment; filename=#{zipname}"
self.response.body = Enumerator.new do |out| # Enumerator is ruby 1.9
while !zf.eof? do
out << zf.read(4096)
end
end
end
end
end
# Zipfile generator
def generate_zip(&block)
photos = Photo.all
# base temp dir
temp_dir = Dir.mktempdir
# path for zip we are about to create, I find that ruby zip needs to write to a real file
zip_path = File.join(temp_dir, 'export.zip')
Zip::ZipFile::open(zip_path, true) do |zipfile|
photos.each do |photo|
zipfile.get_output_stream(photo.photo.identifier) do |io|
io.write photo.photo.file.read
end
end
end
# yield the zipfile to the action
block.call 'export.zip', zip_path
ensure
# clean up the tempdir now!
FileUtils.rm_rf temp_dir if temp_dir
end