Streaming CSV Download from Rails 3.2 app - ruby-on-rails

I am desperate to get a streaming CSV download working in my Rails 3.2.2 app.
I have tried the 'csv_builder' gem (https://github.com/dasil003/csv_builder), which advertises support for this feature, however it seems that there were some changes in Rails 3.2 that are keeping it from working (it produces an 'uninitialized constant ActionView::Template::Handler' error on app startup).
Any other ideas or solutions out there? Thanks!
EDIT: To clarify, I am needing to export all entries of a model as a CSV file. There are so many rows, that it is timing out... therefore the need for streaming. I have used the Comma gem (https://github.com/crafterm/comma) for this in the past, but it doesn't support streaming either, at the moment.

OK, after a bit more research I hacked together the following in my controller. It'll stream if response_body is given something enumeratable (is that a word?). Also, the server needs to be able to stream (I am using Unicorn on Heroku). I'd like very much to not have all this stuff in the controller, so my next step is to extract it out somehow.
format.csv {
#entries = Entry.all
#columns = ["First Name", "Last Name"].to_csv
#filename = "entries-#{Date.today.to_s(:db)}"
self.response.headers["Content-Type"] ||= 'text/csv'
self.response.headers["Content-Disposition"] = "attachment; filename=#{#filename}"
self.response.headers["Content-Transfer-Encoding"] = "binary"
self.response_body = Enumerator.new do |y|
#entries.each_with_index do |entry, i|
if i == 0
y << #columns
end
y << [entry.first_name, entry.last_name].to_csv
end
end
}

The approach that I took with a Rails 2.3.8 app was to spawn a new thread to handle the csv parsing and then use an AJAX call to check the server to see if the file was ready (I relied on File.mtime).
Just ripped it out of the app to post here so I've removed alot of the csv parsing code, and haven't included all of the views
sorry end of the day rush :D
controllers/exports_controller.rb
class ExportsController < ApplicationController
require 'fastercsv'
require 'generic_agent'
require 'generic_record'
def listing
#this_filepath = "../html/whatever/" << Time.now.strftime("%I:%M:%S_%d:%m:%y") << ".csv"
#spawn_id = spawn(:nice => 1) do
FasterCSV.open(#this_filepath, "w") do |csv|
csv << [ "outbreak_id"]
end
end
render :update do |page|
page.replace_html 'export_status', :partial => 'export_status_partial'
end
end
def send_export
#this_filepath = params[:with]
csv_file = File.open(#this_filepath.to_s, 'r')
csv_string = ""
csv_file.each_line do |line|
csv_string << line
end
send_data csv_string, :filename => "export.csv",
:type => 'text/csv; charset=iso-8859-1; header=present',
:disposition => "attachment; filename=export.csv"
#send_file #this_filepath.to_s, :stream => false, :type=>"text/csv", :x_sendfile=>true
#send_data csv_string, :filename => export.csv
#File.delete(#this_filepath.to_s)
end
def export_checker
filename_array = params['filename'].split(/\//)
#file_found = 0
#file_ready = 0
#file_size = File.size(params['filename'])
#this_filepath = params['filename']
if File.exists?(params['filename'])
release_time = Time.now - 5.seconds
if File.mtime(params['filename']).utc < release_time.utc
#file_found = 1
#file_ready = 1
#file_access_time = File.mtime(params['filename'])
#file_release_time = release_time
#file_size = File.size(params['filename'])
else
#file_found = 1
#file_ready = 0
#file_size = File.size(params['filename'])
end
else
#file_found = 0
#file_ready = 0
#file_size = File.size(params['filename'])
end
render :action => "export_checker"
end
end
views/exports/export_checker.rjs
if #file_found == 1 && #file_ready == 1 && #file_size > 0
page.replace_html 'link_to_file', :partial => "export_ready"
if #file_release_time
page.replace_html 'notice', "<div>Completed #{#file_release_time.strftime("%I:%M:%S %A %d %B %Y")} :: file size #{#file_size.to_s}</div>"
end
page.visual_effect :highlight, 'link_to_file', :endcolor => '#D3EDAB'
elsif #file_found == 1
page.replace_html 'link_to_file', "<div> File found, but still being constructed.</div><div>#{#this_filepath.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF9900'
else
page.replace_html 'link_to_file', "<div> File not found #file_found #{#file_found.to_s} #file_ready #{#file_ready.to_s}</div>"
page.visual_effect :highlight, 'link_to_file', :endcolor => '#FF0000'
end

Related

How to automatically download app builds from TestFlight

I have a bunch of apps uploaded to the Testflight. How to automatically download all the builds to my computer?
You can use Ruby + Mechanize (https://github.com/sparklemotion/mechanize):
require 'mechanize'
#agent = Mechanize.new
def process_login_page page
puts 'Login...'
login_form = page.forms.first # form has no action or name so just take the first one
login_field = login_form.field_with(:name => 'username')
password_field = login_form.field_with(:name => 'password')
login_field.value = 'username'
password_field.value = 'password'
login_form.submit
app_link_pattern = /\/dashboard\/applications\/(.*?)\/token\//
puts 'Dashboard...'
#agent.get("https://testflightapp.com/dashboard/applications/") do |dashboard_page|
dashboard_page.links.each do |link|
link.href =~ app_link_pattern
if $1 != nil
puts "Builds page for #{$1}..."
#agent.get "https://testflightapp.com/dashboard/applications/#{$1}/builds/" do |builds_page|
process_builds_page builds_page
end
end
end
end
end
def process_builds_page page
body = page.body
build_pages = body.scan /<tr class="goversion pointer" id="\/dashboard\/builds\/report\/(.*?)\/">/
build_pages.each do |build_id|
#agent.get "https://testflightapp.com/dashboard/builds/complete/#{build_id.first}/" do |build_page|
process_build_page build_page
end
end
end
def process_build_page page
build_link = page.links_with(:dom_class => 'bitly').first
#agent.get("https://www.testflightapp.com#{build_link.href}") { |install_page| process_install_page install_page}
end
def process_install_page page
# we need to figure out what kind of build is that
ipa_link = page.link_with(:text => "download the IPA.")
if (ipa_link != nil)
download_build ipa_link, "ipa"
else
apk_link = page.link_with(:text => "download the APK.")
if (apk_link != nil)
download_build apk_link, "apk"
end
end
end
def download_build link, file_ext
link.href =~ /\/dashboard\/ipa\/(.*?)\//
filename = "#{$1}.#{file_ext}"
file_url = "https://www.testflightapp.com#{link.href}"
puts "Downloading #{file_url}..."
#agent.get(file_url).save("out/#{filename}")
end
FileUtils.rm_rf "out"
Dir.mkdir "out"
login_page_url = "https://testflightapp.com/login/"
#agent.get(login_page_url) { |page| process_login_page page }
Disclaimer: I'm not a Ruby developer and this code is way far from being well designed or safe. Just a quick and dirty solution.

How do I create one CSV file containing two models with comma gem?

I would like to create one CSV file containing two models with comma gem in my ruby 3.2.8 application. Maybe the answer to the question is trivial, but this is the first time I use this gem. I know how create the file it based on a model, but I don' t know how make matches of two.
I have a views\participants\index with :
<%= link_to 'Download CSV', '/participants.csv' %>
the controller :
def index
#participants = Participant.all
respond_to do |format|
format.html # index.html.erb
format.json { render json: #participants }
format.csv { send_data #participants.to_comma }
end
end
participant Model:
require 'comma'
class Participant < ActiveRecord::Base
comma do
id
token
end
end
and field Model:
require 'comma'
class Field < ActiveRecord::Base
comma do
name
value
id_participant
end
end
in the db i have:
Participant1 = ["id" => 1 , "token" => "a"]
Participant2 = ["id" => 2 , "token" => "b"]
Field1= ["id_participant" => 1, "name" => "p1_exams1", "value" =>5]
Field2= ["id_participant" => 1, "name" => "p1_exams2", "value" =>3]
Field3= ["id_participant" => 2, "name" => "p2_exams1", "value" =>2]
Field4= ["id_participant" => 2, "name" => "p2_exams2", "value" =>3]
I would like to have a file like this:
id token
1 a
id_p name value
1 p1_c1_exams1 5
1 p1_c1_exams2 3
id token
2 b
id_p name value
2 p1_c1_exams1 2
2 p1_c1_exams2 3
I tried with this in controller:
def index
#participants = Participant.all
#fields = Field.all
require 'csv'
csv_string = CSV.generate do |csv|
#participants.each do |p|
csv << ["id","token","last_ip_address","start_date","last_transition_date","completion_date","completed","total_time_survey","created_at"]
csv << [ p.id, p.token , p.last_ip_address, p.start_date, p.last_transition_date, p.completion_date, p.completed, p.total_time_survey, p.created_at]
#fields.each do |f|
if String(f.id_participant) == String(p.id)
csv << ["id","name","value","id_participant","id_survey","created_at"]
csv << [f.id,f.name, f.insert_value, f.id_participant, f.id_survey, f.created_at]
end
end
end
end
respond_to do |format|
format.html # index.html.erb
format.json { render json: #participants }
format.csv { send_data csv_string,
:type => "text/csv; charset=iso-8859-1; header=present",
:disposition => "attachment; filename=Database.csv" }
end
end
You can also use the fastercsv for this
i think this will help u what i am understanding that u have has many relationship between Participant and Field regarding this i have write some piece of code u can customize it as ur need
#participants = Participant.all
csv_string = FasterCSV.generate do |csv|
#participants.each do |i|
csv << ["id","token"]
csv << [ i.id, i.token ]
i.fields.each do |j|
csv << ["id_p","name", "value"]
csv << [i.id,j.name, j.value]
end
end
end
send_data csv_string,
:type => "text/csv; charset=iso-8859-1; header=present",
:disposition => "attachment; filename=anyName.csv"

call a class from a collection_action in ActiveAdmin (RoR)

I'm new in ruby on rails. I'm trying to call a class from a collection_action in ActiveAdmin. Here is the code(app/admin/models):
collection_action :status_race, :method => :post do
#Do some import work..
redirect_to :class => :import_route
end
And this is the code of the class I want to call(app/lib/route):
class ImportRoute
def initialize
#seperator = " "
#time_format = "%d-%m-%y"
end
def run(filename)
puts "Running route import file"
raise "File" + filename + "doesn't not exist" unless File.exist(filename)
ri = RouteImporter.find(:name => self.class.name)
if(ri.nil?)
puts "Error, file doesn't exists"
end
CSV.foreach(filename, {:col_sep => #seperator}) do |row|
if row.lenght >5
ri.country_name = row[0] + " " + row[1]
ri.type = row[2]
ri.company = row [3]
else
ri.country_name = row[0]
ri.type = row[1]
ri.company = row[2]
ri.date = row[4].gsub(";", " ")
end
end
end
end
I was using redirect_to to call the class but is not working, and I don't have any clue about how to do it. Any idea? Thanks!
This code is taken from http://activeadmin.info/docs/8-custom-actions.html#collection_actions
ActiveAdmin.register Post do
collection_action :import_csv, :method => :post do
# Do some CSV importing work here...
redirect_to {:action => :index}, :notice => "CSV imported successfully!"
end
end
This collection action will generate a route at
“/admin/posts/import_csv” pointing to the
Admin::PostsController#import_csv controller action.
So it means you have to add a method import_csv in app/controllers/admin/posts_controller.rb. Inside this method, you can instantiate your model:
def import_csv
import_route = ImportRoute.new
# do stuff on this object
end
You can easily adapt this to your code

How I can dynamically generate url ( for generating xls report )?

Hello!
I have this trouble: I'm searching reports by date and in html view everything is alright, BUT when I'm rendering xls view error appear, because it didn't receive params, so I need to pass them in URL for xls link_to generator.
My controller:
def show
#website = Website.find(params[:id])
if params[:report] && params[:report][:start_date] && params[:report][:end_date]
#search_by_created_at
#performance_reports = #website.performance_reports.where("created_at between ? and ?", params[:report][:start_date].to_date, params[:report][:end_date].to_date)
else
#performance_reports = #website.performance_reports
end
respond_to do |format|
format.html # index.html.erb
format.xls
format.xml { render :xml => #performance_reports }
end
end
and my generated url looks like:
http://127.0.0.1:3000/websites/25/performance_reports/show?utf8=%E2%9C%93&report[end_date]=07%2F09%2F2012&report[start_date]=04%2F09%2F2012&commit=Run+Report
mine xls url is generated like this:
<%= link_to url_for(:format => 'xls') do%>
<%= image_tag("excel.png", :id => "analytics",:size => '21x23')%> <b>Export</b>
<% end %>
result:
http://127.0.0.1:3000/websites/25/performance_reports/show
Any help will be appreciated.
xls in not available by default.
Add this:
gem "spreadsheet"
gem "to_xls", :git => "https://github.com/dblock/to_xls.git", :branch => "to-xls-on-models"
Register the Excel MIME type in config/initializers/mime_types.rb by adding this:
Mime::Type.register "application/vnd.ms-excel", :xls
Add an as_xls method to model that you want to export for the fields you want.
For example for a User model you might have:
def as_xls(options = {})
{
"Id" => id.to_s,
"Name" => name,
"E-Mail" => email,
"Joined" => created_at,
"Last Signed In" => last_sign_in_at,
"Sign In Count" => sign_in_count
}
end
Add code to the controller:
def index
#users = User.all
respond_to do |format|
format.html
format.xls { send_data #users.to_xls, content_type: 'application/vnd.ms-excel', filename: 'users.xls' }
end
end
Provide a link:
= link_to 'Export', users_path(request.parameters.merge({:format => :xls}))
All code should have a test. You could do something like this:
describe "GET index.xls" do
it "creates an Excel spreadsheet with all users" do
user = Fabricate :user
get :index, :format => :xls
response.headers['Content-Type'].should == "application/vnd.ms-excel"
s = Spreadsheet.open(StringIO.new(response.body))
s.worksheets.count.should == 1
w = s.worksheet(0)
w.should_not be_nil
w.row(0)[0].should == "Id"
w.row(1)[0].should == user.id.to_s
w.row(0)[1].should == "Name"
w.row(1)[1].should == user.name
end
end

Export CSV with Mongoid and FasterCSV

Class UserController
def export_users
users = User.all
stream_csv do |csv|
csv << ["Name","Email","Gender"]
users.each do |i|
csv << [i.name,i.email,i.gender]
end
end
end
def stream_csv
require 'fastercsv'
filename = params[:action] + ".csv"
#this is required if you want this to work with IE
if request.env['HTTP_USER_AGENT'] =~ /msie/i
headers['Pragma'] = 'public'
headers["Content-type"] = "text/plain"
headers['Cache-Control'] = 'no-cache, must-revalidate, post-check=0, pre-check=0'
headers['Content-Disposition'] = "attachment; filename=\"#{filename}\""
headers['Expires'] = "0"
else
headers["Content-Type"] ||= 'text/csv'
headers["Content-Disposition"] = "attachment; filename=\"#{filename}\""
controller.response.headers["Content-Transfer-Encoding"] = "binary"
end
render :text => Proc.new { |response, output|
csv = FasterCSV.new(output, :row_sep => "\r\n")
yield csv
}
end
end
Err: "#Proc:0x9382539#/sites/app/controllers/export_controller.rb:56"
Using Ruby 1.8 and Rails 3.0.9
So I think the problem here is that I'm not using "Proc" right. Or it's not supposed to act like just another block...
I thought about programming a new logic into the class so that reads better. But if somebody could explain to me why my code is wrong or at least point me in a new direction than I might be able to learn something new here. Thanks
Note: Found a better way:
def export_inverts
require 'fastercsv'
inverts = Invert.all
filename = params[:action] + ".csv"
#this is required if you want this to work with IE
if request.env['HTTP_USER_AGENT'] =~ /msie/i
headers['Pragma'] = 'public'
headers["Content-type"] = "text/plain"
headers['Cache-Control'] = 'no-cache, must-revalidate, post-check=0, pre-check=0'
headers['Content-Disposition'] = "attachment; filename=\"#{filename}\""
headers['Expires'] = "0"
else
headers["Content-Type"] ||= 'text/csv'
headers["Content-Disposition"] = "attachment; filename=\"#{filename}\""
headers["Content-Transfer-Encoding"] = "binary"
end
csv_string = FasterCSV.generate do |csv|
csv << ["Genus","Species","Common Name","Pet Name","Gender"]
inverts.each do |i|
csv << [i.scientific_name,i.scientific_name,i.common_name,i.pet_name,i.gender]
end
end
render :text => csv_string
end
Yield can only be used inside a function or a block. Yield is used in a function that takes a block to say, yield some value into the block. Actually it says yield this value into the proc that the block has been converted into with the ampersand operator (in most cases). However, you could pass a Proc to a function that was expecting it.
Here, you just want to return the value from the proc and "yield" isn't needed.

Resources