I am uploading txt files using carrierwave. The files are not small (80 MB - 500 MB) and I want to remove some of the lines to reduce this size (about 80% of the file size is going to be reduced).
I have created a model method in order to clear these lines:
require 'fileutils'
def clear_unnecessary_lines
old_file_path = Rails.root.join('public').to_s + log_file.to_s
new_file_path = old_file_path.sub! '.txt', '_temp.txt'
File.open(old_file_path, 'r') do |old_file|
File.open(new_file_path, 'w') do |new_file|
old_file.each_line do |line|
new_file.write(line) unless line.grep(/test/).size > 0
end
end
end
FileUtils.mv new_file_path, old_file_path
end
but I am getting error when I am trying to open the new file saying there is no such file. As I have read opening a file with the w option should create an empty file for writing. Then why I am getting such error?
Also, since log_file column is holding the path to the original file, and I am changing it, could you tell how to rename the new file with the old name? As I have checked I should specify only old and new names, not paths.
Errno::ENOENT: No such file or directory - /home/gotqn/Rails/LogFilesAnalyser/LogFilesAnalyser/public/uploads/log_file/log_file/3/log_debug_temp.txt
It is strange that If I execute the following command in rails console, it is not throwing an error and the file is created.
File.open('/home/gotqn/Rails/LogFilesAnalyser/LogFilesAnalyser/public/uploads/log_file/log_file/3/log_debug_temp.txt','w')
Ah, i see your problem now. When you do this
new_file_path = old_file_path.sub! '.txt', '_temp.txt'
you call the "self-altering" version of sub, ie sub!. This will actually change the value of old_file_path as a side effect. Then, in the next line, you try to open this file, which hasn't been created yet. Take out the exclamation mark and you should be fine.
Related
I'm trying to work with zip file. I have a document I want to extract, add content to it, replace it and remove the one I extracted.
Whenever I just extract the file, add content, and then delete it - every thing works. But when I try to use replace -
#zipfile.replace(document_entry, document_path)
or add and then delete, I get different errors.
If I just add the file without deleting it - it works ok.
If I add the file and delete it or use replace function, it doesn't work and raises an error.
for example, when I use this code, I get:
No such file or directory # rb_sysopen - /home/shiran/Desktop/pop/app/documentfirst.xml (Errno::ENOENT)
Any help would be appreciated, I'm stuck and have no idea what else to look for.
document_entry = "word/document.xml" #this is the path in my zip I opened before
document_path = "/home/shiran/Desktop/pop/app/documentfirst.xml"
zipfile.extract(document_entry, document_path)
fd = File.open(document_path)
file = Nokogiri::XML(fd)
#this is where I add content to my file which works ok
wbody = file.xpath("//w:r")
wtstart = Nokogiri::XML::Node.new "t", file
wtstart.content = "I am adding a new content after the header!"
wbody.children.before(wtstart)
File.write(document_path, file.to_xml)
zipfile.remove(document_entry)
zipfile.add(document_entry, document_path)
fd.close
file_exist = File.exist?(document_path)
if file_exist
#raises an error here -
File.delete(document_path)
puts "file exists" #(it does puts file exists)
end
I have the following code to connect my rails app to my FTP. This works great. However, I want to use open-uri to open the csv file so I can parse it. Any ideas how to do this? I think it's an easy thing to do but I'm missing something.
require 'net/ftp'
ftp = Net::FTP.new
ftp.connect("xxx.xxx.xx.xxx",21)
ftp.login("xxxxx","xxxx")
ftp.chdir("/")
ftp.passive = true
puts ftp.list("TEST.csv")
You'll need to use #gettextfile.
A) Get the file to a local temporary file and read its content
# Creating a tmp file can be done differently as well.
# It may also be omitted, in which case `gettextfile`
# will create a file in the current directory.
Dir::Tmpname.create(['TEST', ['.csv']) do |file_name|
ftp.gettextfile('TEST.csv', file_name)
content = File.read(file_name)
end
B) Pass a block to gettextfile and get the content one line at a time
content = ''
ftp.gettextfile('TEST.csv') do |line|
content << line
end
I am having some trouble writing a file to a specific path taking the file name from excel. Here is the code which I am using
out_file = File.new (#temp_path/ "#{obj_info[3].to_s}","w")
"#{obj_info[3].to_s}" = sample.txt
The value sample.txt comes from Excel during run time
#temp_path = "C:/Users/Somefolder/"
The error displayed is:
NoMethodError: undefined method `[]' for nil:NilClass
However, if the code is:
out_file = File.new ("#{obj_info[3].to_s}","w")
it successfully creates a file called sample.txt in the default directory. However, I want it to be stored in a specific directory and the file name needs to be passed from Excel.
Any help is much appreciated.
I believe your problem is because there a space between / and "
#temp_path/ "#{obj_info[3].to_s}
and I guess you want to build a path.
My advice is that you use File.join
f_path = File.join(#temp_path,obj_info[3].to_s)
out_file = File.new (f_path,"w")
Let me know if that solved the problem
You have 2 problems:
obj_info is nil so you make an error reading the value from excel, the error you get indicates it is on an array, in the code you published the only thing that's an array is what you read from excel.
Print the contents with p obj_info right before your code to check.
#temp_path and {obj_info[3].to_s} need to be concatenated to make a path.
You can do that with File.join like Mauricio suggests or like this
out_file = File.new ("#{#temp_path}/#{obj_info[3]}","w")
You can drop the to_s in this case.
It would be better if you publish the whole of your script that is meaningful.
Something I'm not getting about the version process...
I have a zip file with a file inside, and I want to upload the file as a "version" of the zip:
Uploader:
version :specificFile do
process :extract_file
end
def extract_file
file = nil
Zip::ZipFile.open(current_path) do |zip_file|
file = zip_file.select{|f| f.name.match(/specificFile/)}.first
zip_file.extract(file, "tmp/" + file.name.gsub("/", "-")){ true }
end
File.open("tmp/" + file.name.gsub("/", "-"))
end
Usage:
=link_to "Specific File", instance.uploader.specificFile.url
Only this just nets me two copies of the zip. Clearly, there's something I'm missing about how version / process works, and I haven't been able to find documentation that actually explains the magic.
So how do I do this, and what am I missing?
This provided the "why", although it took a bit to understand:
How do you create a new file in a CarrierWave process?
To rephrase, when you go to create a version, carrierwave makes a copy of the file and then passes the process the file path. When the process exits, carrierwave will upload the contents of that path - not the file the process returns, which is what I thought was going on.
Working code:
version :specificFile do
process :extract_file
def full_filename (for_file = model.logo.file)
"SpecificFile.ext"
end
end
def extract_plist
file = nil
Zip::ZipFile.open(current_path) do |zip_file|
file = zip_file.select{|f| f.name.match(/specificFile/)}.first
zip_file.extract(file, "tmp/" + file.name.gsub("/", "-")){ true }
end
File.delete(current_path)
FileUtils.cp("tmp/" + file.name.gsub("/", "-"), current_path)
end
So, to make what I want to happen, happen, I:
Tell carrierwave to use a particular filename. I'm using a hardcoded value but you should be able to use whatever you want.
Overwrite the contents of current_path with the contents you want under the version name. In my case, I can't just overwrite the zip while I'm "in it" (I think), so I make a copy of the file I care about and overwrite the zip via File and FileUtils.
PS - It would be nice to avoid the duplication of the zip, but it doesn't look like you can tell carrierwave to skip the duplication.
On a RoR app that i've inherited, a test is failing that involves a file upload. The assertion that fails looks like so:
assert_equal File.size("#{RAILS_ROOT}/test/fixtures/#{filename}"), #candidate.picture.length
It fails with (the test file is 69 bytes):
<69> expected but was <5>.
This is after a post using:
fixture_file_upload(filename, content_type, :binary)
In the candidate model, the uploaded file is assigned to a property that is then saved to a mediumblob in MySQL. It looks to me like the uploaded file is 69 bytes, but after it is assigned to the model property (using UploadedFile.read), the length is showing as only 5 bytes.
So this code:
puts "file.length=" + file.length.to_s
self.picture = file.read
puts "self.picture.length=" + self.picture.length.to_s
results in this output:
file.length=69
self.picture.length=5
I'm at a bit of a loss as to why this is, any ideas?
This came down to a Windows/Ruby idiosyncrasy, where reading the file appeared to be happening in text mode. There is an extension in this app in test_helper, something like:
class ActionController::TestUploadedFile
# Akward but neccessary for testing since an ActionController::UploadedFile subtype is expected
include ActionController::UploadedFile
def read
tempfile = File.new(self.path)
tempfile.read
end
end
And apparently, on Windows, there is a specific IO method that can be called to force the file into binary mode. Calling this method on the tempfile, like so:
tempfile.binmode
caused everything to work as expected, with the read from the UploadedFile matching the size of the fixture file on disk.