I am using a rake task to take data from one csv, call the shopify api using that data, and save the response to another CSV. The problem is I have no error handler in place so that if the shopify api cannot find the resource I provided, the whole task gets aborted. What is the best way to handle the error so that if the resource is not found in Shopify, simply skip it and proceed to the next row?
The line calling the shopify API in the code below is:
variant = ShopifyAPI::Variant.find(vid)
namespace :replace do
desc "replace variant id with variant sku"
task :sku => :environment do
file="db/master-list-3-28.csv"
newFile = Rails.root.join('lib/assets', 'newFile.csv')
CSV.open(newFile, "a+") do |csv|
CSV.foreach(file) do |row|
msku, namespace, key, valueType, value = row
valueArray = value.split('|')
newValueString = ""
valueArray.each_with_index do |v, index|
recArray = v.split('*')
handle = recArray[0]
vid = recArray[1]
newValueString << handle
newValueString << "*"
# use api call to retrieve variant sku using handle and vid
#replace vid with sku and save to csv
variant = ShopifyAPI::Variant.find(vid)
sleep 1
# puts variant.sku
newValueString << variant.sku
if index < 2
newValueString << "|"
end
end
#end of value save the newvaluestring to new csv
csv << [newValueString]
end
end
end
end
Here's a simple way to get it done:
begin
variant = ShopifyAPI::Variant.find(vid)
rescue
next
end
If an exception is raised the stuff in rescue happens.
Related
I have an import working correctly from a Spreadsheet using Roo gem.
The problem is every time I call the rake task, new records are created.
I want to update_attributes of the records in case the record exists.
Is there any way to approach this? I've tried this with no luck:
namespace :import do
desc "Import data from spreadsheet" # update this line
task data: :environment do
data = Roo::Spreadsheet.open('lib/t3.xlsx') # open spreadsheet
headers = data.row(1) # get header row
data.each_with_index do |row, idx|
next if idx == 0 # skip header
# create hash from headers and cells
product_data = Hash[[headers, row].transpose]
product = Product.new(product_data)
puts "Guardando Producto #{product.name}"
if product?
product.update_attributes
else
product.save!
end
rescue ActiveRecord::RecordInvalid => invalid
puts invalid.record.errors
end
end
end
if product? will never return false. You're testing whether the variable contains a falsy value (nil/false) or any other value. After calling product = Product.new, the value stored in product can never be nil or false.
What you want is to first find, and if not found, new, and then update_attributes on the resulting object:
product = Product.find_by(product_data.name) || Product.new
product.update_attributes(product_data)
I'm working on a code which displays the images from the AWS server. But I'm facing trouble in looping the code.
It works fine for the 1st display but it is not going further (I've to display upto 6 images)
code for this -
def get_image_urls(user)
user_identifications = user.user_identifications.where(current_flag: true).order(:id_dl)
urls = []
keys = []
if !user_identifications.empty? && !user_identifications.nil?
user_identifications.each_with_index do |each_id, index|
obj = S3_BUCKET.object(each_id.aws_key)
urls << {each_id.id_dl=> obj.presigned_url(:get)}
keys << {each_id.id_dl=> each_id.aws_key}
end
end
return urls, keys
end
How to increment the loop based on checking the id and user.identifications value?
reject all empty values and then iterate users_identifications.
Like:
sanitized_identifications = users_identifications.reject(&:blank?)
sanitized_identifications.each_with_index do |identification, _index|
# Now if you want to skip an iteration based on some condition, try `next`, like:
# next if some_condition
# in you case
obj = S3_BUCKET.object(each_id.aws_key)
next if obj.blank?
urls << {each_id.id_dl=> obj.presigned_url(:get)}
keys << {each_id.id_dl=> each_id.aws_key}
end
UPDATE
# ...
user_identifications.each_with_index do |each_id, index|
begin
obj = S3_BUCKET.object(each_id.aws_key)
urls << {each_id.id_dl=> obj.presigned_url(:get)}
keys << {each_id.id_dl=> each_id.aws_key}
rescue => e
next
end
end
#...
Cheers!
I am using a rake task and the csv module to loop through one csv, extract and alter the data I need and then append each new row of data to a second csv. However each row seems to be overwriting/replacing the previous row in the new csv instead of appending it as a new row after it. I've looked at the documentation and googled but can't find any examples of appending rows to the csv differently.
require 'csv'
namespace :replace do
desc "replace variant id with variant sku"
task :sku => :environment do
file="db/master-list-3-28.csv"
CSV.foreach(file) do |row|
msku, namespace, key, valueType, value = row
valueArray = value.split('|')
newValueString = ""
valueArray.each_with_index do |v, index|
recArray = v.split('*')
handle = recArray[0]
vid = recArray[1]
newValueString << handle
newValueString << "*"
variant = ShopifyAPI::Variant.find(vid)
newValueString << variant.sku
end
#end of value save the newvaluestring to new csv
newFile = Rails.root.join('lib/assets', 'newFile.csv')
CSV.open(newFile, "wb") do |csv|
csv << [newValueString]
end
end
end
end
Your mode when opneing the file is wrong and should be a+. See details in the docs: http://ruby-doc.org/core-2.2.4/IO.html#method-c-new
Also, you might want to open that file just once and not with every line.
I am trying to scrape list of restaurants for my zip code from Deliveroo.co.uk
I need to add a way to figure out whether a restaurant is open or closed... from the website its very clear, but I just need to update my code to reflect this.
How do I go about doing this? I need to create something like a 'status' variable and then set each restaurant to 'open' or 'closed'.
Here is the website I'm trying to scrape from: https://deliveroo.co.uk/restaurants/london/maida-vale?postcode=W92DE&time=1800&day=today
And my code is below.
thanks.
require 'open-uri'
require 'nokogiri'
require 'csv'
# Store URL to be scraped
url = "https://deliveroo.co.uk/restaurants/london/maida-vale?postcode=W92DE"
# Parse the page with Nokogiri
page = Nokogiri::HTML(open(url))
# Display output onto the screen
name =[]
page.css('span.list-item-title.restaurant-name').each do |line|
name << line.text
end
category = []
page.css('span.restaurant-detail.detail-cat').each do |line|
category << line.text
end
delivery_time = []
page.css('span.restaurant-detail.detail-time').each do |line|
delivery_time << line.text
end
distance = []
page.css('span.restaurant-detail.detail-distance').each do |line|
distance << line.text
end
status = []
# Write data to CSV file
CSV.open("deliveroo.csv", "w") do |file|
file << ["Name", "Category", "Delivery Time", "Distance", "Status"]
name.length.times do |i|
file << [name[i], category[i], delivery_time[i], distance[i]]
end
end
end
We need to check li.restaurant--details have / have not class unavailable for close / open restaurant.
status = []
page.css('li.restaurant--details').each do |line|
if line.attr("class").include? "unavailable"
sts = "closed"
else
sts = "open"
end
status << sts
end
Btw, you should remove white space when get restaurant_name, etc ...
page.css('span.list-item-title.restaurant-name').each do |line|
name << line.text.strip
end
You can refer my code at here: https://gist.github.com/vinhnglx/4eaeb2e8511dd1454f42
I'm running a rake task to import some file attributes and I'm receiving an error that would lead me to believe that the string created for each line contains some sort of new-line character (e.g. /n).
EDIT - New-line character has been confirmed to be the issue.
Here is a sample of what my CSV file might look like:
1|type1,type2|category1
2|type2|category1,category2,category3
3|type2,type4|category3,category8
And here is my code to deal with it:
namespace :data do
desc "import"
task :import => :environment do
file = File.open(Rails.root.join('lib/assets/data.csv'), 'r')
file.each do |line|
attrs = line.split("|")
foo = Model.find(attrs[0])
attrs[1].split(",").each do |type|
foo.add_type!(ModelType.find_by_name(type))
end
attrs[2].split(",").each do |category|
foo.categorize!(ModelCategory.find_by_name(category))
end
end
end
end
ModelType and ModelCategory are both seperate models with a :through relationship to Model that is built with the function Model.add_type! and Model.categorize!.
When I run rake data:import, everything works fine up until the final category is reached at the end of the first line. It doesn't matter which category it is, nor how many categories are present in attrs[2] - it only fails on the last one. This is the error I receive:
Called id for nil, which would mistakenly be 4 -- if you really wanted the id of nil, use object_id
Any thoughts on how to fix this or avoid this error?
You can use chomp:
attrs = line.chomp.split("|")
attrs = line.split("|")
if attrs.length > 0
foo = Model.find(attrs[0])
...
end
You probably have an empty line at the end of your CSV
UPDATE
file = File.open(Rails.root.join('lib/assets/data.csv'), 'r')
file.split("\r\n").each do |line|
or
file = File.open(Rails.root.join('lib/assets/data.csv'), 'r')
file.split("\r").each do |line|
or
file = File.open(Rails.root.join('lib/assets/data.csv'), 'r')
file.split("\n").each do |line|
depending on how the CSV was originally generated!
Use String.encode(universal_newline: true) instead gsub.
It converting CRLF and CR to LF # Always break lines with \n