My problem is that i have to export an excel sheet save some rows to the database without duplication or redundancy
so i started it with importing CSV instead of XLS then when i finish i might be able to parse the xls
this is my model code:
require 'csv'
class Machine < ActiveRecord::Base
def self.assign_row(row)
a, b, c, d = row
#c = c.slice(1,4)
Machine.create(name: c, mid: #c)
end
def self.import(file)
CSV.foreach(file.path) do |row|
machine = Machine.assign_row(row)
end
end
end
Import method in machines_controller
def import
count = Machine.import params[:file]
redirect_to machines_path, notice: "file imported successfully!"
end
Migration code
def change
create_table :machines do |t|
t.string :name
t.string :mid
t.timestamps null: false
end
add_index :machines, :name, :unique => true
end
and the view code
<%= form_tag import_machines_path, multipart: true do %>
<%= file_field_tag :file %>
<%= submit_tag "upload" %>
<% end %>
routes
Rails.application.routes.draw do
resources :errors
resources :machines do
collection do
post :import
end
end
root "machines#index
end
any thoughts on how to skip duplicated records from saving into database would be appreciated
thanks
Unique Identifier:
To avoid duplicate records saving to database you should maintain a unique identifier other than primary key. This helps you to identify if the record already available in DB, if it is available you can skip that record from saving again.
I guess you can use name in this case, which should be unique for each record in database. write a uniqueness validation in model to implement this.
After changes:
validates_uniqueness_of :name
def self.assign_row(row)
a, b, c, d = row
#c = c.slice(1,4)
machine = Machine.find_by(name: c)
Machine.create(name: c, mid: #c) if machine.blank?
end
Hope it helps!!
Thank you.
Related
I want to generate forms for a resource that has a postgres jsonb column :data, and I want the schema for these forms to be stored in a table in the database. After a lot of research I am 90% there but my method fails in ActiveAdmin forms upon create (not update). Can anyone explain this?
Sorry for the long code snippets. This is a fairly elaborate setup but I think it would be of some interest since if this works one could build arbitrary new schemas dynamically without hard-coding.
I am following along this previous discussion with Rails 6 and ActiveAdmin 2.6.1 and ruby 2.6.5.
I want to store Json Schemas in a table SampleActionSchema that belong_to SampleAction (using the json-schema gem for validation)
class SampleActionSchema < ApplicationRecord
validates :category, uniqueness: { case_sensitive: false }, allow_nil: false, allow_blank: true
validate :schema_is_json_schema
private
def schema_is_json_schema
metaschema = JSON::Validator.validator_for_name("draft4").metaschema
unless JSON::Validator.validate(metaschema, schema)
errors.add :schema, 'not a compliant json schema'
end
end
end
class SampleAction < ActiveRecord::Base
belongs_to :sample
validate :is_sample_action
validates :name, uniqueness: { case_sensitive: false }
after_initialize :add_field_accessors
before_create :add_field_accessors
before_update :add_field_accessors
def add_store_accessor field_name
singleton_class.class_eval {store_accessor :data, field_name.to_sym}
end
def add_field_accessors
num_fields = schema_properties.try(:keys).try(:count) || 0
schema_properties.keys.each {|field_name| add_store_accessor field_name} if num_fields > 0
end
def schema_properties
schema_arr=SampleActionSchema.where(category: category)
if schema_arr.size>0
sc=schema_arr[0]
if !sc.schema.empty?
props=sc.schema["properties"]
else
props=[]
end
else
[]
end
end
private
def is_sample_action
sa=SampleActionSchema.where(category: category)
errors.add :category, 'not a known sample action' unless (sa.size>0)
errors.add :base, 'incorrect json format' unless (sa.size>0) && JSON::Validator.validate(sa[0].schema, data)
end
end
This all works correctly; For example, for a simple schema called category: "cleave", where :data looks like data: {quality: "good"}, I can create a resource as follows in the rails console:
sa=SampleAction.new(sample_id: 6, name: "test0", data: {}, category: "cleave" )
=> #<SampleAction id: nil, name: "test0", category: "cleave", data: {}, created_at: nil, updated_at: nil, sample_id: 6>
sa.quality = "good" => true
sa.save => true
To make this system work in AA forms, I call the normal path (new or edit)_admix_sample_action_form with params: {category: "cleave"} and then I generate permit_params dynamically:
ActiveAdmin.register SampleAction, namespace: :admix do
permit_params do
prms=[:name, :category, :data, :sample_id, :created_at, :updated_at]
#the first case is creating a new record (gets parameter from admix/sample_actions/new?category="xxx"
#the second case is updating an existing record
#falls back to blank (no extra parameters)
categ = #_params[:category] || (#_params[:sample_action][:category] if #_params[:sample_action]) || nil
cat=SampleActionSchema.where(category: categ)
if cat.size>0 && !cat[0].schema.empty?
cat[0].schema["properties"].each do |key, value|
prms+=[key.to_sym]
end
end
prms
end
form do |f|
f.semantic_errors
new=f.object.new_record?
cat=params[:category] || f.object.category
f.object.category=cat if cat && new
f.object.add_field_accessors if new
sas=SampleActionSchema.where(category: cat)
is_schema=(sas.size>0) && !sas[0].schema.empty?
if session[:active_sample]
f.object.sample_id=session[:active_sample]
end
f.inputs "Sample Action" do
f.input :sample_id
f.input :name
f.input :category
if !is_schema
f.input :data, as: :jsonb
else
f.object.schema_properties.each do |key, value|
f.input key.to_sym, as: :string
end
end
end
f.actions
end
Everything works fine if I am editing an existing resource (as created in the console above). The form is displayed and all the dynamic fields are updated upon submit. But when creating a new resource where e.g. :data is of the form data: {quality: "good"} I get
ActiveModel::UnknownAttributeError in Admix::SampleActionsController#create
unknown attribute 'quality' for SampleAction.
I have tried to both add_accessors in the form and to override the new command to add the accessors after initialize (these should not be needed because the ActiveRecord callback appears to do the job at the right time).
def new
build_resource
resource.add_field_accessors
new!
end
Somehow when the resource is created in the AA controller, it seems impossible to get the accessors stored even though it works fine in the console. Does anyone have a strategy to initialize the resource correctly?
SOLUTION:
I traced what AA was doing to figure out the minimum number of commands needed. It was necessary to add code to build_new_resource to ensure that any new resource AA built had the correct :category field, and once doing so, make the call to dynamically add the store_accessor keys to the newly built instance.
Now users can create their own original schemas and records that use them, without any further programming! I hope others find this useful, I certainly will.
There are a couple ugly solutions here, one is that adding the parameters to the active admin new route call is not expected by AA, but it still works. I guess this parameter could be passed in some other way, but quick and dirty does the job. The other is that I had to have the form generate a session variable to store what kind of schema was used, in order for the post-form-submission build to know, since pressing the "Create Move" button clears the params from the url.
The operations are as follows: for a model called Move with field :data that should be dynamically serialized into fields according to the json schema tables, both
admin/moves/new?category="cleave" and admin/moves/#/edit find the "cleave" schema from the schema table, and correctly create and populate a form with the serialized parameters. And, direct writes to the db
m=Move.new(category: "cleave") ==> true
m.update(name: "t2", quality: "fine") ==> true
work as expected. The schema table is defined as:
require "json-schema"
class SampleActionSchema < ApplicationRecord
validates :category, uniqueness: { case_sensitive: false }, allow_nil: false, allow_blank: true
validate :schema_is_json_schema
def self.schema_keys(categ)
sas=SampleActionSchema.find_by(category: categ)
schema_keys= sas.nil? ? [] : sas[:schema]["properties"].keys.map{|k| k.to_sym}
end
private
def schema_is_json_schema
metaschema = JSON::Validator.validator_for_name("draft4").metaschema
unless JSON::Validator.validate(metaschema, schema)
errors.add :schema, 'not a compliant json schema'
end
end
end
The Move table that employs this schema is:
class Move < ApplicationRecord
after_initialize :add_field_accessors
def add_field_accessors
if category!=""
keys=SampleActionSchema.schema_keys(category)
keys.each {|k| singleton_class.class_eval{store_accessor :data, k}}
end
end
end
Finally, the working controller:
ActiveAdmin.register Move do
permit_params do
#choice 1 is for new records, choice 2 is for editing existing
categ = #_params[:category] || (#_params[:move][:category] if #_params[:move]) || ""
keys=SampleActionSchema.schema_keys(categ)
prms = [:name, :data] + keys
end
form do |f|
new=f.object.new_record?
f.object.category=params[:category] if new
if new
session[:current_category]=params[:category]
f.object.add_field_accessors
else
session[:current_category] = ""
end
keys=SampleActionSchema.schema_keys(f.object.category)
f.inputs do
f.input :name
f.input :category
keys.each {|k| f.input k}
end
f.actions
end
controller do
def build_new_resource
r=super
r.assign_attributes(category: session[:current_category])
r.add_field_accessors
r
end
end
end
I created a web scraper.
I'm struggling to save the results into the model's columns.
How do I push the results of the scrapped results into the columns?
Do you map it? Would like to understand it so I can eventually post to index perhaps ... or show previous saved results, etc ...
Schema
ActiveRecord::Schema.define(version: 20170308223314) do
# These are extensions that must be enabled in order to support this database
enable_extension "plpgsql"
create_table "links", force: :cascade do |t|
t.string "link_info"
t.string "date"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
end
The two columns that I'm trying to save to are the ones you see in the schema above: link_info and date ...
Current Controller
class LinksController < ApplicationController
def craigslist_scrape
require 'open-uri'
url = "https://losangeles.craigslist.org/search/web"
page = Nokogiri::HTML(open(url))
#craigslist_info = page.css("ul.rows")
#link_info = page.css("li.result-row p.result-info a.result-title.hdrlnk")
#date = page.css("li.result-row p.result-info time.result-date")
end
end
craiglist_scrape.html.erb
<% #link_info.each_with_index do |link, index| %>
<h2><%= "Title of the job: #{link.text}" %></h2>
<p><%= "Date: #{#date[index].text}" %></p>
<% end %>
routes
Rails.application.routes.draw do
root 'links#craigslist_scrape'
end
model
class Link < ApplicationRecord
end
In the controller you can add:
def craigslist_scrape
require 'open-uri'
url = "https://losangeles.craigslist.org/search/web"
page = Nokogiri::HTML(open(url))
#craigslist_info = page.css("ul.rows")
#link_info = page.css("li.result-row p.result-info a.result-title.hdrlnk")
#date = page.css("li.result-row p.result-info time.result-date")
#link_info.each_with_index do |link, index|
Link.new(:link => link.text, :date => #date[index].text).save
end
end
I'm rather new to Ruby and don't know much about Rails but what I always do when I'm trying to save something into a "database" is I create a hash. For example:
database = {}
puts "What is your name?"
input = gets.chomp
puts "What's your age?"
input_2 = Integer(gets.chomp)
database[input.to_sym] = input_2
In the example above I create a hash called "database" and I ask the user for their name and age. Storing the user's name as the string value for database and their age as an integer value. I don't know if this helps at all, like I said I'm fairly new to Ruby.
here is my code:
Perk not save on multiple select,when multiple true/false. perk save and habtm working.
class Perk < ActiveRecord::Base
has_and_belongs_to_many :companies
end
class Company < ActiveRecord::Base
has_and_belongs_to_many :perks
end
view perk/new.html.erb
<%= select_tag "company_id", options_from_collection_for_select(Company.all, 'id', 'name',#perk.companies.map{ |j| j.id }), :multiple => true %>
<%= f.text_field :name %>
Controller's code:
def new
#perk = Perk.new
respond_with(#perk)
end
def create
#perk = Perk.new(perk_params)
#companies = Company.where(:id => params[:company_id])
#perk << #companies
respond_with(#perk)
end
Your select_tag should return an array of company_ids:
<%= select_tag "company_ids[]", options_from_collection_for_select(Company.all, 'id', 'name',#perk.companies.map{ |j| j.id }), :multiple => true %>
http://apidock.com/rails/ActionView/Helpers/FormTagHelper/select_tag#691-sending-an-array-of-multiple-options
Then, in your controller, reference the company_ids param:
#companies = Company.where(:id => params[:company_ids])
(I assume that you've intentionally left out the #perk.save call in your create action... Otherwise, that should be included as well. Model.new doesn't store the record.)
It sounds like you may not have included company_id in the perk_params method in your controller. Rails four uses strong pramas this means you need to state the params you are allowing to be set.However it is difficult to say for sure without seeing more of the code.
In your controller you should see a method like this (there may be more options that just :name):
def perk_params
params.require(:perk).permit(:name)
end
You should try adding :company_id to it so it looks something like this:
def perk_params
params.require(:perk).permit(:name, :company_id)
end
if there are other params int your method leave them in and just added :company_id
EDIT to original answer
The above will only work on a one-to-many or one-to-one because you are using has_and_belongs_to_many you will need to add companies: [] to the end of your params list like this
def perk_params
params.require(:perk).permit(:name, companies: [] )
end
or like this
def perk_params
params.require(:perk).permit(:name, companies_ids: [] )
end
See these links for more details:
http://edgeapi.rubyonrails.org/classes/ActionController/StrongParameters.html
http://edgeguides.rubyonrails.org/action_controller_overview.html#strong-parameters
I need to upload a text file without saving it in the database. My goal is to upload this file and automatically have to take your content and save it in my database.
my file: data.txt
name age occupation
julio 19 student
ramon 20 student
my database:
class CreateStudents < ActiveRecord::Migration
def change
create_table: students do |t|
t.string "name"
t.integer "age"
t.string "occupation"
t.timestamps
end
end
end
Does anyone have any idea how this can be done? I searched on the internet, but found no solution to my case. I need help.
= form_tag url, {multipart: true} do
= file_field_tag :file
....
in controller
if params[:file]
lines = params[:file].tempfile.readlines.map(&:chomp) #readlines from file & removes newline symbol
lines.shift #remove first line
lines.each do |l|
m = l.match(/(\S+)\s(\d+)\s(\S+)/) #parse line
Student.create {name: m[1],age: m[2], occupation: m[3]}
end
end
I have used globalize2 to add i18n to an old site. There is already a lot of content in spanish, however it isn't stored in globalize2 tables. Is there a way to convert this content to globalize2 with a migration in rails?
The problem is I can't access the stored content:
>> Panel.first
=> #<Panel id: 1, name: "RT", description: "asd", proje....
>> Panel.first.name
=> nil
>> I18n.locale = nil
=> nil
>> Panel.first.name
=> nil
Any ideas?
I'm sure you solved this one way or another but here goes. You should be able to use the read_attribute method to dig out what you're looking for.
I just used the following to migrate content from the main table into a globalize2 translations table.
Add the appropriate translates line to your model.
Place the following in config/initializers/globalize2_data_migration.rb:
require 'globalize'
module Globalize
module ActiveRecord
module Migration
def move_data_to_translation_table
klass = self.class_name.constantize
return unless klass.count > 0
translated_attribute_columns = klass.first.translated_attributes.keys
klass.all.each do |p|
attribs = {}
translated_attribute_columns.each { |c| attribs[c] = p.read_attribute(c) }
p.update_attributes(attribs)
end
end
def move_data_to_model_table
# Find all of the translated attributes for all records in the model.
klass = self.class_name.constantize
return unless klass.count > 0
all_translated_attributes = klass.all.collect{|m| m.attributes}
all_translated_attributes.each do |translated_record|
# Create a hash containing the translated column names and their values.
translated_attribute_names.inject(fields_to_update={}) do |f, name|
f.update({name.to_sym => translated_record[name.to_s]})
end
# Now, update the actual model's record with the hash.
klass.update_all(fields_to_update, {:id => translated_record['id']})
end
end
end
end
end
Created a migration with the following:
class TranslateAndMigratePages < ActiveRecord::Migration
def self.up
Page.create_translation_table!({
:title => :string,
:custom_title => :string,
:meta_keywords => :string,
:meta_description => :text,
:browser_title => :string
})
say_with_time('Migrating Page data to translation tables') do
Page.move_data_to_translation_table
end
end
def self.down
say_with_time('Moving Page translated values into main table') do
Page.move_data_to_model_table
end
Page.drop_translation_table!
end
end
Borrows heavily from Globalize 3 and refinerycms.