I have 1 field delivery_time It is in an array
include :
DELIVERY_TIME = [
I18n.t("activerecord.attributes.order.none_delivery_time"),
"09:00～12:00",
"12:00～14:00",
"14:00～16:00",
"16:00～18:00",
"18:00～20:00",
"19:00～21:00",
"20:00～21:00",
].freeze
when I downloaded the csv directory it was in the form
"09:00～12:00"
but i want now when I download it will take the form :
"0912"
how to customize it?
my code:
def perform
CSV.generate(headers: true) do |csv|
csv << attributes
orders.each do |order|
csv << create_row(order)
end
end
end
def create_row(order)
row << order.delivery_time
end
AFAIU, you need to modify DELIVERY_TIME to fit your format. CSV is absolutely out of scope here. So to transform values, one should split by ～ and take the hour from the result.
DELIVERY_TIME = [
"09:00～12:00",
"12:00～14:00",
"14:00～16:00",
"16:00～18:00",
"18:00～20:00",
"19:00～21:00",
"20:00～21:00",
].freeze
DELIVERY_TIME.map { |s| s.split('～').map { |s| s[0...2] }.join }
#⇒ ["0912", "1214", "1416", "1618", "1820", "1921", "2021"]
A safer method would be to use DateTime#parse for this
require 'time'
DELIVERY_TIME.map do |s|
s.split('～').map { |s| DateTime.parse(s).strftime("%H") }.join
end
#⇒ ["0912", "1214", "1416", "1618", "1820", "1921", "2021"]
It's not real clear what you're asking, but I'd probably start with something like this:
"09:00～12:00".scan(/\d{2}/).values_at(0, 2).join # => "0912"
Using that in some code:
"09:00～12:00".scan(/\d{2}/).values_at(0, 2).join # => "0912"
DELIVERY_TIME = [
'blah',
"09:00～12:00",
"12:00～14:00",
"14:00～16:00",
"16:00～18:00",
"18:00～20:00",
"19:00～21:00",
"20:00～21:00",
].freeze
ary = [] << DELIVERY_TIME.first
ary += DELIVERY_TIME[1..-1].map { |i|
i.scan(/\d{2}/).values_at(0, 2).join
}
# => ["blah", "0912", "1214", "1416", "1618", "1820", "1921", "2021"]
Related
I have no idea where to start with testing this rake task. Do I need stubs? If so, how to use them? Any help would be appreciated. Thanks!
desc "Import CSV file"
task :import => [:environment] do
data = "db/data.csv"
headers = CSV.open(data, 'r') { |csv| csv.first }
cs = headers[2..-1].map { |c| Model1.where(name: c).first_or_create }
ls = Model2.find_ls
csv_contents = CSV.read(photos)
csv_contents.shift
csv_contents.each do |row|
p = Model2.where(id: row[0], f_name: row[1]).first_or_create
p_d = FastImage.size(p.file.url(:small))
p.update_attributes(dimensions: p_d)
row[2..-1].each_with_index do |ls, i|
unless ls.nil?
ls.split(',').each { |l|
cl = Model3.where(name: l.strip, model_1_id: cs[i].id).first_or_create
Model4.where(p_id: p.id, model_3_id: cl.id).first_or_create
}
end
end
end
end
Here's how I'd do it:
1) Happy-path test(s)
Rake tasks as such are a pain to test. Extract the body of the rake task into a class:
whatever.rake
desc "Import CSV file"
task :import => [:environment] do
CSVImporter.new.import "db/data.csv"
end
end
lib/csv_importer.rb
class CsvImporter
def import(data)
headers = CSV.open(data, 'r') { |csv| csv.first }
cs = headers[2..-1].map { |c| Model1.where(name: c).first_or_create }
ls = Model2.find_ls
csv_contents = CSV.read(photos)
csv_contents.shift
csv_contents.each do |row|
p = Model2.where(id: row[0], f_name: row[1]).first_or_create
p_d = FastImage.size(p.file.url(:small))
p.update_attributes(dimensions: p_d)
row[2..-1].each_with_index do |ls, i|
unless ls.nil?
ls.split(',').each { |l|
cl = Model3.where(name: l.strip, model_1_id: cs[i].id).first_or_create
Model4.where(p_id: p.id, model_3_id: cl.id).first_or_create
}
end
end
end
end
Now it's easy to write a test that calls CSVImporter.new.import on a test file (that's why import takes the file as a parameter instead of hardcoding it) and expects the results. If it's reasonable to import db/data.csv in the test environment, you can do that in a test if you want. You probably only need one test like this. No stubs are required.
2) Edge and error cases
There is a lot of logic here which, for simplicity and speed, you'll want to test without creating actual model objects. That is, yes, you'll want to stub. Model2.find_ls and FastImage.size are already easy to stub. Let's extract a method to make the other model calls easy to stub:
class CsvImporter
def import(data)
headers = CSV.open(data, 'r') { |csv| csv.first }
cs = headers[2..-1].map { |c| Model1.first_or_create_with(name: c) }
ls = Model2.find_ls
csv_contents = CSV.read(photos)
csv_contents.shift
csv_contents.each do |row|
p = Model2.first_or_create_with(id: row[0], f_name: row[1])
p_d = FastImage.size(p.file.url(:small))
p.update_attributes(dimensions: p_d)
row[2..-1].each_with_index do |ls, i|
unless ls.nil?
ls.split(',').each { |l|
cl = Model3.first_or_create_with(name: l.strip, model_1_id: cs[i].id)
Model4.first_or_create_with(p_id: p.id, model_3_id: cl.id)
}
end
end
end
end
app/models/concerns/active_record_extensions.rb
module ActiveRecordExtensions
def first_or_create_with(attributes)
where(attributes).first_or_create
end
end
and include the module in all of the models that need it.
Now it's easy to stub all of the model methods so you can write tests that simulate any database situation you like.
I like to export a dataset from my rails application as a csv file using the builtin csv library of rails. Usually a csv file is written row wise like in my example below which comes from my datasets_controller.rb:
require 'csv'
dataset = Dataset.find(6)
dataset_headers = dataset.datacolumns.collect { |dc| dc.columnheader }
csv_file = CSV.generate do |csv|
csv << dataset_headers
end
And now my question is if I could also write my csv files column wise like this?
require 'csv'
dataset_columns = Datacolumn.all(:conditions => ["dataset_id = ?", 6], :order => "columnnr ASC").uniq
csv_file = CSV.generate do |csv|
csv << "here put one after another all my data columns"
end
EDIT:
Based on Douglas suggestion I came up with the colde below.
data_columns=Datacolumn.all(:conditions => ["dataset_id = ?", dataset.id], :order => "columnnr ASC").uniq
CSV.generate do |csv|
value=Array.new
data_columns.each do |dc|
value << dc.columnheader
dc.sheetcells.each do |sc|
if sc.datatype && sc.datatype.is_category? && sc.category
value << sc.category.short
elsif sc.datatype && sc.datatype.name.match(/^date/) && sc.accepted_value
value << sc.accepted_value.to_date.to_s
elsif sc.accepted_value
value << sc.accepted_value
else
value << sc.import_value
end
end
csv << value
value = Array.new
end
end
The output is not transposed for this case and looks like this:
height,10,2,<1,na
fullauthor,Fortune,(Siebold & Zucc.) Kuntze,Fortune,(Siebold & Zucc.) Kuntze
Year,1850,1891,1850,1891
fullname,Mahonia bealei,Toxicodendron sylvestre,Mahonia bealei,Toxicodendron sylvestre
But when I change the line which writes the csv to
csv << value.transpose
I get an error which tells me that it could not convert a string to array to do that.
Anybody an Idea how to fix this?
Any help with this would be appreciated.
Best Claas
You could use Array#transpose, which will flip your rows to columns. A simple example:
> a = [['name', 'charles', 'dave'],['age', 24, 36],['height', 165, 193]]
=> [["name", "charles", "dave"], ["age", 24, 36], ["height", 165, 193]]
> a.transpose
=> [["name", "age", "height"], ["charles", 24, 165], ["dave", 36, 193]]
Thus, assuming dataset_columns is an array:
require 'csv'
dataset_columns = Datacolumn.all(:conditions => ["dataset_id = ?", 6], :order => "columnnr ASC").uniq
csv_file = CSV.generate do |csv|
csv << dataset_columns.transpose
end
Now I'm fetching data from another url...
Here is my code:
require 'rubygems'
require 'nokogiri'
html = page.body
doc = Nokogiri::HTML(html)
doc.encoding = 'utf-8'
rows = doc.search('//table[#id = "MainContent_GridView1"]//tr')
#details = rows.collect do |row|
detail = {}
[
[:car, 'td[1]/text()'],
[:article, 'td[2]/text()'],
[:group, 'td[3]/text()'],
[:price, 'td[4]/text()'],
].each do |name, xpath|
detail[name] = row.at_xpath(xpath).to_s.strip
end
detail
end
#details
I tried to do it via array, not a hash. But I get a lot of errors...
Are there any ideas?
I need it for another method...
also i set data (this result hash) to another car here:
oem_art = []
#constr_num.each do |o|
as_oem = get_from_as_oem(o.ARL_SEARCH_NUMBER)
if as_oem.present?
oem_art << as_oem
end
end
#oem_art = oem_art.to_a.uniq
Do you just want to change a hash into an array? If so, just use the to_a method on your hash.
hash = {:a => "something", :b => "something else"}
array = hash.to_a
array.inspect #=> [[:a, "something"], [:b, "something else"]]
It looks like you're looking for something like hash['key'] to hash.key in Ruby
The Hash Class doesn't support .key notation by default, OpenStruct creates an Object from the Hash so you can use dot notation to access the properties. Overall it's basically just syntactic sugar with overhead.
Suggested code (from linked answer)
>> require 'ostruct'
=> []
>> foo = {'bar'=>'baz'}
=> {"bar"=>"baz"}
>> foo_obj = OpenStruct.new foo
=> #<OpenStruct bar="baz">
>> foo_obj.bar
=> "baz"
So in your example, you could do:
# Initialised somewhere
require 'ostruct'
DETAIL_INDICES = {
:car => 1,
:article => 2,
:group => 3,
:price => 4,
}
# ** SNIP **
#details = rows.map do |row|
DETAIL_INDICES.inject({}) do |h,(k,v)|
h.merge(k => row.at_xpath("td[#{v}]/text()").to_s.strip)
end
end.collect { |hash| OpenStruct.new hash }
#details.each do |item|
puts item.car
end
Of course if performance is a concern you can merge your map&collect (They are the same), but this is just a minor separation for basic semantic differences, although I usually only use map for consistency, so feel free to choose yourself :)
EDIT -- Additional code from your edit simplified
#oem_art = #constr_num.select do |item|
as_oem = get_from_as_oem(item.ARL_SEARCH_NUMBER)
as_oem.present?
end
puts #oem_art.uniq
I have a Xpath query which accepts array elements for output using Axslx, I need to tidy up my ouput for certain conditions one of which is the 'Software included'
My xpath scrapes the following URL http://h10010.www1.hp.com/wwpc/ie/en/ho/WF06b/321957-321957-3329742-89318-89318-5186820-5231694.html?dnr=1
A sample of my code is below:
clues = Array.new
clues << 'Optical drive'
clues << 'Pointing device'
clues << 'Software included'
selector = "//td[text()='%s']/following-sibling::td"
data = clues.map do |clue|
xpath = selector % clue
[clue, doc.at(xpath).text.strip]
end
Axlsx::Package.new do |p|
p.workbook.add_worksheet do |sheet|
data.each { |datum| sheet.add_row datum }
end
p.serialize 'output.xlsx'
end
My Current output formatting
My Desired output formatting
If you can rely on the data always using ';' for separators, have a go at this:
data = []
clues.each do |clue|
xpath = selector % clue
details = doc.at(xpath).text.strip.split(';')
data << [clue, details.pop]
details.each { |detail| data << ['', detail] }
end
to generate the data before the Axlsx::Package.new block
In answer to you comment/question: You do it with something like this ;)
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'axlsx'
class Scraper
def initialize(url, selector)
#url = url
#selector = selector
end
def hooks
#hooks ||= {}
end
def add_hook(clue, p_roc)
hooks[clue] = p_roc
end
def export(file_name)
Scraper.clues.each do |clue|
if detail = parse_clue(clue)
output << [clue, detail.pop]
detail.each { |datum| output << ['', datum] }
end
end
serialize(file_name)
end
private
def self.clues
#clues ||= ['Operating system', 'Processors', 'Chipset', 'Memory type', 'Hard drive', 'Graphics',
'Ports', 'Webcam', 'Pointing device', 'Keyboard', 'Network interface', 'Chipset', 'Wireless',
'Power supply type', 'Energy efficiency', 'Weight', 'Minimum dimensions (W x D x H)',
'Warranty', 'Software included', 'Product color']
end
def doc
#doc ||= begin
Nokogiri::HTML(open(#url))
rescue
raise ArgumentError, 'Invalid URL - Nothing to parse'
end
end
def output
#output ||= []
end
def selector_for_clue(clue)
#selector % clue
end
def parse_clue(clue)
if element = doc.at(selector_for_clue(clue))
call_hook(clue, element) || element.inner_html.split('<br>').each(&:strip)
end
end
def call_hook(clue, element)
if hooks[clue].is_a? Proc
value = hooks[clue].call(element)
value.is_a?(Array) ? value : [value]
end
end
def package
#package ||= Axlsx::Package.new
end
def serialize(file_name)
package.workbook.add_worksheet do |sheet|
output.each { |datum| sheet.add_row datum }
end
package.serialize(file_name)
end
end
scraper = Scraper.new("http://h10010.www1.hp.com/wwpc/ie/en/ho/WF06b/321957-321957-3329742-89318-89318-5186820-5231694.html?dnr=1", "//td[text()='%s']/following-sibling::td")
# define a custom action to take against any elements found.
os_parse = Proc.new do |element|
element.inner_html.split('<br>').each(&:strip!).each(&:upcase!)
end
scraper.add_hook('Operating system', os_parse)
scraper.export('foo.xlsx')
And the FINAL answer is... a gem.
http://rubydoc.info/gems/ninja2k/0.0.2/frames
the following code:
[1,3,5].to_csv
=> "1,3,5\n" # this is good
[[1,3,5], [2,4,6]].to_csv
=> "135,246\n" # why doesn't it just do it for array of array?
but require this instead:
data = [[1,3,5], [2,4,6]]
csv_string = FasterCSV.generate do |csv|
data.each {|a| csv << a}
end
=> "1,3,5\n2,4,6\n"
or shorter:
data = [[1,3,5], [2,4,6]]
csv_string = FasterCSV.generate {|csv| data.each {|a| csv << a}}
=> "1,3,5\n2,4,6\n"
The question is, when given an array of array, why is to_csv not designed to handle it automatically, so that in Rails, we can do
respond_to do |format|
format.csv { render :text => data.to_csv }
[[1,3,5], [2,4,6]].each{ |line| puts line.to_csv } isn't so bad. You could always override Array#to_csv if you wanted.
I suspect FasterCSV's decision to not implement that was because it is hard to be absolutely certain that's what the programmer will want. What if the input happens to be
[[1], 2, 3, 4] ? Just looking at the first element of the outer array would make you think that it may be an array of arrays...