I am trying to upload a file in rails (using paperclip), and I want to process some of the file data before letting paperclip send it off to s3 storage. In my controller, I just grab the file parameter (which does give me a file) and then I try to read the lines into an array
csv_file = params[:activity][:data]
array = IO.readlines(csv_file.path)
The problem is, I'm only getting the last line of the file. I tried using .rewind, but still get just the last line.
I dislike readlines and I always use regular expressions. Try this.
End of line - \n
Handy block structure to ensure that the file handle is closed:
File.open(csv_file.path) do |f|
a = f.readlines
process a...
end
Reading a whole file into memory might not be a good idea depending on the size of the files.
Related
I am trying to figure out how to take a string that is the file contents of a mp4 file and write a properly formatted mp4 file. Currently I am just throwing the string into a file and slapping a .mp4 extension on it, but this resulting file cannot be played by any video players (I am assuming because of all the missing meta data).
def write_mp4(mp4_string)
file = File.new('movie.mp4', 'w')
file.puts(mp4_string)
file.close
end
For context, I am doing this in a Ruby on Rails application, not sure if that changes anything. Please help thanks.
Use "wb" mode, which will suppress EOL conversions and set the encoding properly
Use write, not puts, as the latter inserts an extra EOL
You could use a shortcut: File.write('movie.mp4', mp4_string, mode: 'wb') - or even File.binwrite('movie.mp4', mp4_string).
Of course, make sure the string actually contains a correct file before - for example, if mp4_string.encoding doesn't return #<Encoding:ASCII-8BIT>, you probably done goofed somewhere before the writing step, too :)
I'm building a feature that parses a PDF format CV. I have a method that is called on :before_save which handles parsing. I'm able to access the PDF file within this method, before it saves using...
file = cv.queued_for_write[:original]
But then I need to pass the file to PDF::Reader, however, it seems like pdf-reader only accepts paths or URLs to files, not the actual file itself. This approach...
reader = PDF::Reader.new(file)
Throws this error:
ArgumentError (input must be an IO-like object or a filename):
Do I need to save the file to a tmp folder or something and then pass the path to the pdf-reader to parse it? I'm hoping to parse the PDF as quickly as possible, so that doesn't seem ideal. Any advice is appreciated!
I figured out that the "queued_for_write" object has a path attribute.
file = cv.queued_for_write[:original]
So I can just access it like this:
reader = PDF::Reader.new(file.path)
In a model, I want to get a file's contents without rendering it. Say the file is a .erb file. I want to store its contents in a database, and then later on, I'll evaluate the string so that it replaces the variables in the .erb file with actual values.
Is there a method like render_to_string that doesn't actually evaluate the .erb part?
I'll ignore any reason you would want to do something like this. Ok so, you're looking to read files, this is done with plain Ruby:
File.read 'path/to/file'
That's how you read any file in Ruby. For a view in Rails you'd have to specify the path:
File.read Rails.root.join('app/views/some_view_dir/your_view_file.erb')
Just replace some_view_dir/your_view_file.erb with your actual view.
Reading files this way gets just the raw content, and you can do this with any file type.
In my Rails app, I have a form that allows users to upload images. My app is supposed to resize the images with the following controller method. (POST to this method, params[:file] contains the ActionDispatch::Http::UploadedFile that was uploaded:
def resize_and_store
file = params[:file]
# resize image
Magick::Image.read(file.tempfile).first
newimg = image.resize(100,100)
#etc... Store newimg
end
I get the following error, on the line that says Image.read:
Magick::ImageMagickError (no decode delegate for this image format `0xb9f6052c>' # error/constitute.c/ReadImage/544):
Testing this with an uploaded PNG file, it seems RMagick doesn't pick up that the temporary file is a PNG file. The code above does work if I read a locally stored PNG file, so it can't be that I'm missing the PNG decoder. How can I fix this and why does this happen?
You can do from_blob on a ActionDispatch::Http::UploadedFile param (this is how a file comes in):
images = Magick::Image.from_blob(params[:file].read)
Storing the file temporarily will solve the problem:
open('temp.png', 'wb') do |file|
file << uploaded.tempfile.read
end
images=Magick::Image.read('temp.png')
Probably wise to check input size as well.
Alternatively, parse the image from a blob.
Using the answer by #joost (or similar approach) really helped to point me in the right direction but it didn't work on the second attempt with the same temp file - my use case was creating multiple image types from the tempfile source. This is what I've used, wrapping in a File.open block so we don't leak the file descriptor:
File.open(tempfile, "rb") do |f|
img = Magick::Image::from_blob(f.read).first
resized = img.resize_to_fit(w, h)
resized.write(dest)
resized.destroy!
img.destroy!
end
Maybe there's something wrong with the form? You can consult with Rails Guide here:
Rails Guides: Uploading Files
I think that you may have multipart: true missing in your form declaration.
Also, I would strongly advise to use Carrierwave to handle file uploads. Among several things, it will help you to organize your file transformations (putting logic out of the controllers). Here's a railscast about it:
RailsCasts: CarrierWave File Uploads.
Good luck!
I have a script which takes a file, manipulates some data, and writes an output .csv file. The .csv file should be available for the user to view and download. This is a rails app on heroku with S3.
Right now the script writes a hard coded local filesystem output file "line by line". When I integrate this script with rails, heroku, & amazon S3 do I have to restructure the script to build an array line by line in the controller and write it once as a whole to S3? or do I continue writing to S3 line by line as I do locally?
It appears like I need to build an array in the controller and post to S3? Then a controller 'show' action would reference the file for instance variables used in the view. Almost makes me wonder if the user can just make the csv on the client side and never have to make a file to store on S3? Is this a job for AJAX?
I'm looking at the aws-sdk now to access the file as I would any other file on my local system.
rough example of the as is, write per line code:
file_in.each_line do |line|
#some line manipulation
file_out << output
end
Easy to switch this code to build an array and then write once... I originally wrote it line by line so I don't have the whole file in a large array...
file_in.each_line do |line|
#some line manipulation
#array.push(output)
end
file_out << #array
S3 is not a local filesystem - you need to build the file locally and then send it to S3 (there is software that will make s3 look like a filesystem although I don't know whether you can get that to run on heroku).
If your file is large you can do multipart uploads, but each part (other than the last) must be at least 5MB.