Struggling to get a CSV Report to load when requested - ruby-on-rails

I'm trying to make it possible to view a report from a webpage and I am struggling. This task uses Sidekiq.
When I click on the link that should take me there I get the error: "The action 'churn_risk_report' could not be found for Admin::ReportsController."
In a show.html.erb file, in the Reports view, I've added <li><%= link_to 'Churn Risk Report', churn_risk_report_admin_reports_path %></li> beneath a number of similar lines with links to other reports.
I've added get 'churn_risk_report' in the correct place in my Routes file.
In my Workers directory, which I believe is my system's version of a Scripts directory used for Sidekiq jobs, I have a file called churn_risk_report.rb and in this file I have the following code:
class ChurnRiskReport
include Sidekiq::Worker
def perform
csv_temp = Tempfile.new
puts csv_temp.path
CSV.open(csv_temp.path, 'wb') do |csv|
csv << ["AAA", "BBB", "CCC", "DDD", "EEE", "FFF"]
AccountChurnRisk.all.each do |acr|
report_data = [acr.aaa,
acr.bbb,
"#{acr.ccc} #{acr.ccc}",
acr.ddd,
acr.eee,
acr.fff]
csv << report_data
end
end
report_name = "churn_risk_report"
file_name = "gggggggggg-#{report_name}-#{DateTime.now.to_date.strftime("%b-%Y")}.csv".downcase
bucket_name = 'hhh-reports'
s3 = AWS::S3.new
key = "reports/churn_reports/" + File.basename(file_name)
x = s3.buckets[bucket_name].objects[key].write(:file => csv_temp.path)
end
end
Consecutive letters = code that I think should be kept private. This code worked in a rails console when I ran it, so I know the code works.
I'm struggling with the final part of the task. In the Reports controller I've defined churn_risk_report and just copied in the same code as is in the Worker file. I know this is incorrect but I'm not sure what should go in there. I think there should be some code in here that temporarily creates a churn_risk_report file in the Workers directory, however this might not be the case. I also think that 'async' should be involved somewhere.
Thanks in advance for any help!

At the bottom of this answer, is the gist I keep specifically for basic CSV export logic handling works for most cases that don't require high performance. Typically I use it as a way for the user to download a CSV copy of the data they are currently looking at (so caching in S3 isn't overly practical): https://gist.github.com/MyklClason/f6ac68ca4ce1faa5d655abfb0abe788b
Depending on how large the report is, saving it in S3 may be excessive, but could work as a way to cache it (or store it for long term). In which case before rendering the download link (to the S3 file itself), you check to see if the report already exists, if it does, render a link directly to AWS, if it doesn't, render a "generate report" link, then once the report is generated, render a "Download report" link.
Even better if you can auto-generate the reports, which seems to be what you are doing in the original code. Though I think a "Generate report" button could queue up the job to run the code you have though likely the reports are monthly or something so you know exactly when you need to create them, in which case the "Generate report" button probably isn't needed and you can just provide an S3 download link, if the report has been generated.
Gist content:
# Good solution for exporting to CSV in rails.
# Source: https://www.codementor.io/victorhazbun/export-records-to-csv-files-ruby-on-rails-vda8323q0
class UsersController < ApplicationController
def index
#users = User.all
respond_to do |format|
format.html
format.csv { send_data #users.to_csv, filename: "users-#{Date.today}.csv" }
end
end
end
class User < ActiveRecord::Base
def self.to_csv
attributes = %w{id email name}
CSV.generate(headers: true) do |csv|
csv << attributes
all.each do |user|
csv << attributes.map{ |attr| user.send(attr) }
end
end
end
end
<%= link_to 'Export CSV', user_path(format: :csv) %>

Related

Creating multiple csv-files and download all in one zip-archive using rails

I am looking for a way to create multiple csv files and download them as one zip archive within one request in my rails application.
To build the archive I use rubyzip gem - to download it just the rails built-in function send_data. The problem I have is that rubyzip's add-function requires a pathname to load files from. But there is no path as my csv files are created within the same request.
Some Code:
# controller action to download zip
def download_zip
zip = #company.download_all
send_data zip, filename: "abc.zip", type: 'application/zip'
end
# method to create zip
def download_all
Zip::File.open('def.zip', Zip::File::CREATE) do |zipfile|
self.users.each do |user|
#some magic to combine zipfile.add() and user.to_csv
end
end
end
# method to create csv
def to_csv
CSV.generate do |csv|
#build awesome csv
end
end
Is there a way to save my csv files temporarely at some directory, that I can pass a pathname to zipfile.add()?
Nice weekend everybody and happy coding!
You could either write your CSV output into a temporary file and call zipfile.add() on that, but there is a cleaner solution:
zipfile.get_output_stream("#{user.name}.csv") { |f| f.puts(user.to_csv) }
See http://rdoc.info/github/rubyzip/rubyzip/master/Zip/File#get_output_stream-instance_method for more details on get_output_stream - you can also pass additional parameters to specify attributes for the file to be created.
get_output_stream doesn't work for me. However, the updated method Zip::OutputStream.write_buffer helps
https://gist.github.com/aquajach/7fde54aa9bc1ac03740feb154e53eb7d
The example adds password protection to the file as well.

Ruby parse CSV file to print out the rows

I have a file upload in my Rails application and I want to parse the CSV file assuming the upload went okay. You can see the comment below that indicates where I would like to read the rows of the CSV file. How can I do this? I used carrierwave for the file upload.
I mounted it as such
mount_uploader :file, LCFileUploader
Here is the code I currently have
require 'CSV'
class LCFilesController < ApplicationController
def new
authorize! :create, :lc_file
#lc_file = LCFile.new
end
def create
authorize! :create, :lc_file
puts params
#lc_file = LCFile.new(params[:lc_file])
#lc_file.user_id = current_user.id
if #lc_file.save
#PARSE CSV HERE TO PRINT OUT THE ROWS OF THE CSV FILE
CSV.foreach(#lc_file.file.path) do |row|
puts row
end
redirect_to lc_path, :notice => 'New lc created!'
else
render :new
end
end
end
and I get this error:
undefined method `find_all_by_team_id' for #<Class:0x007fe14c40d848>
You can use the CSV class:
puts CSV.read(#lc_file.file.path)
or one row at a time:
CSV.foreach(#lc_file.file.path) do |row|
puts row
end
Besides CSV generation there are a few more issues:
the redirect will not work after you send send some output. But even if it did, the output would not be seen, since you're redirecting.
the path you are redirecting to is incorrect (I believe that's why you get that error). I suppose you want something like lcfiles_path or lcfile_path(#lc_file). Run rake routes (the same way you ran rails console) to see a list of all available routes.
Now if you still have issues, I suggest posting another question, as this one was mainly about CSV generation and that should be solved using the code I posted at the start of this answer.

How to save a raw_data photo using paperclip

I'm using jpegcam to allow a user to take a webcam photo to set as their profile photo. This library ends up posting the raw data to the sever which I get in my rails controller like so:
def ajax_photo_upload
# Rails.logger.info request.raw_post
#user = User.find(current_user.id)
#user.picture = File.new(request.raw_post)
This does not work and paperclip/rails fails when you try to save request.raw_post.
Errno::ENOENT (No such file or directory - ????JFIF???
I've seen solutions that make a temporary file but I'd be curious to know if there is a way to get Paperclip to automatically save the request.raw_post w/o having to make a tempfile. Any elegant ideas or solutions out there?
UGLY SOLUTION (Requires a temp file)
class ApiV1::UsersController < ApiV1::APIController
def create
File.open(upload_path, 'w:ASCII-8BIT') do |f|
f.write request.raw_post
end
current_user.photo = File.open(upload_path)
end
private
def upload_path # is used in upload and create
file_name = 'temp.jpg'
File.join(::Rails.root.to_s, 'public', 'temp', file_name)
end
end
This is ugly as it requires a temporary file to be saved on the server. Tips on how to make this happen w/o the temporary file needing to be saved? Can StringIO be used?
The problem with my previous solution was that the temp file was already closed and therefore could not be used by Paperclip anymore. The solution below works for me. It's IMO the cleanest way and (as per documentation) ensures your tempfiles are deleted after use.
Add the following method to your User model:
def set_picture(data)
temp_file = Tempfile.new(['temp', '.jpg'], :encoding => 'ascii-8bit')
begin
temp_file.write(data)
self.picture = temp_file # assumes has_attached_file :picture
ensure
temp_file.close
temp_file.unlink
end
end
Controller:
current_user.set_picture(request.raw_post)
current_user.save
Don't forget to add require 'tempfile' at the top of your User model file.

how can I upload and parse an Excel file in Rails?

I want to be able to upload an Excel file that contains contact information. I then went to be able to parse it and create records for my Contact model.
My application is a Rails application.
I am using the paperclip gem on heroku, I've been able to parse vim cards into the Contact model, and am looking for something similar, but will go through all lines of the Excel file.
Gems that simplify the task and sample code to parse would be helpful!
Spreadsheet is the best Excel parser that I have found so far. It offers quite a lot of functionality.
You say you use Paperclip for attachments which is good. However, if you store the attachments in S3 (which I assume since you use Heroku) the syntax for passing the file to spreadsheet is a little different but not difficult.
Here is an example of the pure syntax that can be used and not placed in any classes or modules since I don't know how you intend to start the parsing of contacts.
# load the gem
require 'spreadsheet'
# In this example the model MyFile has_attached_file :attachment
#workbook = Spreadsheet.open(MyFile.first.attachment.to_file)
# Get the first worksheet in the Excel file
#worksheet = #workbook.worksheet(0)
# It can be a little tricky looping through the rows since the variable
# #worksheet.rows often seem to be empty, but this will work:
0.upto #worksheet.last_row_index do |index|
# .row(index) will return the row which is a subclass of Array
row = #worksheet.row(index)
#contact = Contact.new
#row[0] is the first cell in the current row, row[1] is the second cell, etc...
#contact.first_name = row[0]
#contact.last_name = row[1]
#contact.save
end
I had a similar requirement in one of my Rails 2.1.0 application. I solved it in the following manner:
In the 'lib' folder I wrote a module like this:
require 'spreadsheet'
module DataReader
def read_bata(path_to_file)
begin
sheet = book.worksheet 0
sheet.each 2 do |row|
unless row[0].blank?
# Create model and save it to DB
...
end
end
rescue Exception => e
puts e
end
end
end
Had a model Upload:
class Upload < AR::Base
has_attached_file :doc,
:url => "datafiles/:id",
:path => ":rails_root/uploads/:id/:style/:basename.:extension"
# validations, if any
end
Generated an UploadsController which would handle the file upload and save it to appropriate location. I used Paperclip for file upload.
class UploadsController < AC
include DataReader
def new
#upload = Upload.new
end
def create
#upload = Upload.new(params[:upload])
#upload.save
file_path = "uploads/#{#upload.id}/original/#{#upload.doc_file_name}"
#upload.read = DataReader.read_data(file_path)
# respond_to block
end
end
Read about 'spreadsheet' library here and here. You can make appropriate improvements and make the technique work in Rails 3. Hope this helps.
I made a gem to achieve this easily. I called it Parxer and...
It's built on to of roo gem.
It allows you to parse xls, xlsx and csv files.
Has a DSL to handle:
Column mapping.
File, row, and column/cell validation.
Column/cell formatting.

How do I download a remote image from another site to a file_column in Ruby on Rails?

first question, hopefully I don't mess it up :)
A bit of a Ruby on Rails newbie (also Ruby newbie) and have stumbled upon a problem with the intended behavior of the application.
I have a file_column :image in model picture that belongs to model product, which can have many pictures.
The file_column works just fine when used as I think it's meant to be used and that's for uploading image using <%= file_column_field "picture", "image" %> etc. That part works just fine.
The problem comes with the intention of having a text field where user can enter a css -selector for an image tag on their site (they've registered the site and the path to the page where the image should be). I haven't been able to figure out how to properly download the image from that other site "under the hood".
Using these two methods both result in Do not know how to handle a string with value 'GIF89ad..... followed by loads of "binary".
Method 1:
url = URI.parse(picture_www.external_url)
Net::HTTP.start(url.host, url.port) {|http|
resp = http.get(url.path)
picture_www.image = resp.body unless resp.nil?
}
Method 2:
res = open(picture_www.external_url)
picture_www.image = res.read unless res.nil?
The external_url contains the correct url and the download goes ok, so the problem seems to be in the way I'm trying to assign the image to the file_column field. Naturally the problem could be the way I'm downloading the image, I have no idea TBH where the problem actually lies... :)
Anyone able to help me please?
Update:
Trying to use a tempfile "causes undefined method 'original_filename' for" etc
Net::HTTP.start(url.host, url.port) {|http|
resp = http.get(url.path)
tempfile = Tempfile.new('test.jpg')
File.open(tempfile.path, 'wb') do |f|
f.write resp.body
end
picture_www.image = tempfile unless resp.nil?
}
Update2:
Debugging shows me that an uploaded file has attributes #content_type ("image/jpeg" for instance) and #original_path (file name without path) under #_dc_obj and #tmpfile when the tempfile I created does not. Setting these properly would perhaps make this work? How do I set those properly? And if setting those values properly, would the file downloading be done "properly"? After ofcourse re-structuring the code once I get a working solution.
Update3:
From Minver's answer I got the solution for "original_filename" issue and this code seems to work:
io = open(picture_www.external_url)
def io.original_filename; base_uri.path.split('/').last; end
io.original_filename.blank? ? nil : io
picture_www.image = io
No idea though, if this is the "proper" way to do this or not, but this is what I'll be using for now unless some "clearly the right way to do it" solution appears :)
-Pkauko
The UrlUpload method by Joe Martinez is a good solution but the code is missing a key method. If you over-ride the method_missing, you should always also over-ride the respond_to? method as well. In this case it is especially important since some software uses respond_to? when deciding whether to do a multipart-post.
For example, the Faraday gem does this:
def has_multipart?(body)
body.values.each do |v|
if v.respond_to?(:content_type)
return true
elsif v.respond_to?(:values)
return true if has_multipart?(v)
end
end
false
end
So, if you are going to use the UrlUpload code above, I suggest you add the following method:
def respond_to?(symbol)
attachment_data.respond_to?(symbol) || super
end
Then Faraday and other related gems will be able to use an instance of this class to generate a proper multipart-post.
I don't know but maybe this is what you are looking for. When you save the image you provide a css_selector and gets a image file in return.
This is the view:
<%= form_for(#image) do |f| %>
<div class="field">
<%= f.label :css_selector %><br />
<%= f.text_field :css_selector %>
</div>
<div class="actions">
<%= f.submit %>
</div>
<% end %>
and this is the model:
class Picture < ActiveRecord::Base
require 'open-uri' # Required to download the photo
require 'mechanize' # Good gem to parse html pages
belongs_to :product
# Define the css_selector (not required as a filed in the database)
attr_accessor :css_selector
# Before we save the image, we download the photo if image has a css_selector value
before_save :download_remote_photo, :if => :css_selector_provided?
private
# Check if the attribute is provided
def css_selector_provided?
!self.css_selector.blank?
end
# This method opens the page where the photo is
# and grab the url to the image using a css-selector
def fetch_photo_url
agent = Mechanize::new
page = agent.get(HERE_IS_THE_URL_TO_THE_PAGE_YOU_WANNA_SCRAPE)
doc = Nokogiri::HTML(page.body)
image_element = doc.at_css(self.css_selector) # Get the image on that page using the css selector
image_url = image_element[:src]
end
def download_remote_photo
self.image = do_download_remote_photo(fetch_photo_url)
end
def do_download_remote_photo(photo_url)
io = open(URI.parse(URI.escape(photo_url)))
def io.original_filename; base_uri.path.split('/').last; end
io.original_filename.blank? ? nil : io
rescue # catch url errors with validations instead of exceptions (Errno::ENOENT, OpenURI::HTTPError, etc...)
end
end
Haven't tested the code but I hope you get the idea!
Here ya go
require 'open-uri'
class UrlUpload
EXTENSIONS = {
"image/jpeg" => ["jpg", "jpeg", "jpe"],
"image/gif" => ["gif"],
"image/png" => ["png"]
}
attr_reader :original_filename, :attachment_data
def initialize(url)
#attachment_data = open(url)
#original_filename = determine_filename
end
# Pass things like size, content_type, path on to the downloaded file
def method_missing(symbol, *args)
if self.attachment_data.respond_to? symbol
self.attachment_data.send symbol, *args
else
super
end
end
private
def determine_filename
# Grab the path - even though it could be a script and not an actual file
path = self.attachment_data.base_uri.path
# Get the filename from the path, make it lowercase to handle those
# crazy Win32 servers with all-caps extensions
filename = File.basename(path).downcase
# If the file extension doesn't match the content type, add it to the end, changing any existing .'s to _
filename = [filename.gsub(/\./, "_"), EXTENSIONS[self.content_type].first].join(".") unless EXTENSIONS[self.content_type].any? {|ext| filename.ends_with?("." + ext) }
# Return the result
filename
end
end
# Make it always write to tempfiles, never StringIO
OpenURI::Buffer.module_eval {
remove_const :StringMax
const_set :StringMax, 0
}

Resources