I've reviewed many, many responses and still unable to preload rails 5 table with csv file. This is my first attempt to load data from rails console. Please help!
csv file formats liket this with CSV.foreach()
["model_id", "model_make_id", "model_name", "model_trim", "model_year", "model_make_display_name"]
["44328", "acura", "MDX", nil, "2011", "Acura"]
["44331", "acura", "RDX", nil, "2011", "Acura"]
["44332", "acura", "RDX", "SH-AWD", "2011", "Acura"]
["44334", "acura", "RL", nil, "2011", "Acura"]
["44337", "acura", "TL", nil, "2011", "Acura"]
.rb file code
require 'csv'
CSV.foreach('./tmp/cq_sample.csv', :headers => true) do |row|
record = Cqref.new(
:model_id => row[0], # first column of csv file
:model_make_id => row[1], # second column
:model_name => row[2], # third
:model_trim => row[3], # fourth
:model_year => row[4], # fifth
:model_make_display_name => row[5] # sixth
)
record.save
end
Schema
create_table "cqrefs", force: :cascade do |t|
t.string "model_id"
t.string "model_make_id"
t.string "model_name"
t.string "model_trim"
t.string "model_year"
t.string "model_make_display_name"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
end
Error Mesage Extract:
require './tmp/importcsv'
ActiveRecord::DangerousAttributeError: model_name is defined by Active Record. Check to make sure that you don't have an attribute or method with the same name.
from /usr/local/rvm/gems/ruby-2.3.0/gems/activerecord-5.0.2/lib/active_record/attribute_methods.rb:91:in `instance_method_already_implemented?'
from /usr/local/rvm/gems/ruby-2.3.0/gems/activemodel-5.0.2/lib/active_model/attribute_methods.rb:288:in `block in define_attribute_method'
Related
I'm using Ruby on Rails and trying to Get JSON data from a public API and save to a Postgres database.
When I run rake db:seed, I'm getting the following error:
rake aborted!
TypeError: no implicit conversion of String into Integer
/db/seeds.rb:21:in `[]'
/db/seeds.rb:21:in `properties'
/db/seeds.rb:48:in `<top (required)>'
Tasks: TOP => db:seed
(See full trace by running task with --trace)
For reference, line 21 is:
json["data"].map do |property|
Line 48 is where the method is called:
properties
Here's the first item in the JSON array to give you an idea of what I'm trying to map from.
[
{
"PublicationDate": "26/10/2018",
"PropertyNumber": 2195606,
"County": "LAOIS",
"LocalAuthority": "LAOIS COUNTY COUNCIL",
"Valuation": 70600.0,
"Category": "RETAIL (SHOPS)",
"Uses": "SUPERMARKET 2 [500-2500 SQ. M.], -",
"Address1": "36-42A/1 POUND STREET",
"Address2": "RATHDOWNEY",
"Address3": "CO. LAOIS",
"Address4": "",
"Address5": "",
"CarPark": 0,
"Xitm": 628016.65,
"Yitm": 678231.8,
"ValuationReport": [
{
"Level": "0 ",
"FloorUse": "SUPERMARKET",
"Area": 964.62,
"NavPerM2": 60.0000,
"Nav": 57877.200000
},
{
"Level": "0",
"FloorUse": "OFF LICENCE",
"Area": 1.00,
"NavPerM2": 8681.5800,
"Nav": 8681.580000
},
{
"Level": "0",
"FloorUse": "FIT-OUT ALLOWANCE",
"Area": 1.00,
"NavPerM2": 4051.4000,
"Nav": 4051.400000
}
]
},
My database schema follows the JSON structure but not exactly (I'm not sure if this is causing an issue):
create_table "properties", force: :cascade do |t|
t.bigint "user_id"
t.string "publication_date"
t.string "property_number"
t.string "county"
t.string "local_authority"
t.string "valuation"
t.string "category"
t.string "uses"
t.string "address_1"
t.string "address_2"
t.string "address_3"
t.string "address_4"
t.string "address_5"
t.string "car_park"
t.string "xitm"
t.string "yitm"
t.string "valuation_report"
t.datetime "created_at", precision: 6, null: false
t.datetime "updated_at", precision: 6, null: false
t.index ["user_id"], name: "index_properties_on_user_id"
end
Here's my seeds.rb file:
require 'rest-client'
# Define Method
def properties
response = RestClient.get('https://api.valoff.ie/api/Property/GetProperties?Fields=*&LocalAuthority=LAOIS%20COUNTY%20COUNCIL&CategorySelected=RETAIL%20(SHOPS)&Format=csv&Download=false')
json = JSON.parse response
if !json.nil?
json["data"].map do |property|
Property.create(
publication_date: "#{PublicationDate}",
property_number: "#{PropertyNumber}",
county: "#{County}",
local_authority: "#{LocalAuthority}",
valuation: "#{Valuation}",
category: "#{Category}",
uses: "#{Uses}",
address_1: "#{Address1}",
address_2: "#{Address2}",
address_3: "#{Address3}",
address_4: "#{Address4}",
address_5: "#{Address5}",
car_park: "#{CarPark}",
xitm: "#{Xitm}",
yitm: "#{Yitm}",
valuation_report: "#{ValuationReport}"
)
end
else
puts "Error seeding properties."
end
end
# Call Method
properties
When I run rake db:seed, I'm getting the following error:
rake aborted!
TypeError: no implicit conversion of String into Integer
/db/seeds.rb:21:in `[]'
/db/seeds.rb:21:in `properties'
/db/seeds.rb:48:in `<top (required)>'
Tasks: TOP => db:seed
(See full trace by running task with --trace)
For reference, line 21 is:
json["data"].map do |property|
Line 48 is where the method is called:
properties
Any help would be appreciated, thanks
To summarize the recommendations in the various comments, json is an Array after the JSON is parsed. Array indexes are integers, so json["data"] results in a TypeError. Second, there is no data field for the JSON records returned. You can simply iterate over the result array and create your records like so:
def properties
response = RestClient.get('https://api.valoff.ie/api/Property/GetProperties?Fields=*&LocalAuthority=LAOIS%20COUNTY%20COUNCIL&CategorySelected=RETAIL%20(SHOPS)&Format=csv&Download=false')
json = JSON.parse(response)
json.each do |property|
puts "Creating property #{property['PropertyNumber']}"
Property.create!(
publication_date: property['PublicationDate'],
property_number: property['PropertyNumber'],
county: property['County'],
local_authority: property['LocalAuthority'],
valuation_report: property['ValuationReport']
# ... remaining fields omitted for brevity
)
end
end
You don't need really need the .nil? check in your original, since RestClient raises an error for any HTTP response codes other than 200-2007. Likewise, JSON.parse will raise an error if the response isn't valid JSON.
Result:
$ rails db:seed
Creating property 1555903
Creating property 1556133
Creating property 1556998
Creating property 1556516
Creating property 1557007
...
The public option is added to Active Storage in this PR: https://github.com/rails/rails/pull/36729
In it he clearly says: "In the public bucket, the directory structure is /[key]/[filename]"
Which makes sense and is exactly what I want. I want to be able (for example) to email the link to someone and allow them to download the file. So I need the filename.extension. But when I spin up a Rails 6.1 app the uploads to my bucket do not have the filename after them.
The files do appear in my bucket, but only as their key. Not key/filename.
amazon:
service: S3
access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
region: us-east-1
bucket: mybucket
public: true
The uploading works fine. The files are uploaded and appear in my bucket. But in the view <%= #user.avatar.url %> returns https://s3.amazonaws.com/mybucket/g3ci2umbfj6wkxyggx7arhekxfib I want it to return https://s3.amazonaws.com/mybucket/g3ci2umbfj6wkxyggx7arhekxfib/myfile.png
This is really annoying me because in the PR the author clearly states that the public files are saved as key/filename.extension
So the question: In Rails 6.1 does the url method return the filename as part of the path or not. And if not why does the author say that it does? And if not is there a better way than patching key?
TLDR:
= link_to #user.avatar.filename, rails_blob_path(#user.avatar, disposition: "course.avatar"), target: :_blank
will give you a download link like https://corsego-production.s3.eu-central-1.amazonaws.com/3gbpl68kckpkyrbjoslsl254th0u?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGgaCmV1LBuuKkri8zL3ohM4h9STzhTsnavAgulrcpBavL0POXg%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201002T170820Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIA5RINJ20201002%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Signature=0a205d713ebaa9d3ca9d62
#user.avatar.url is not supposed to return anything!
#user.avatar is supposed to return something like #<ActiveStorage::Attached::One:0x00007f15986ee4c0>
When creating an attachment with Active Storage, the following fields about the attachment are automatically populated and saved as active_storage_blobs:
create_table "active_storage_blobs", force: :cascade do |t|
t.string "key", null: false
t.string "filename", null: false
t.string "content_type"
t.text "metadata"
t.bigint "byte_size", null: false
t.string "checksum", null: false
t.datetime "created_at", null: false
t.index ["key"], name: "index_active_storage_blobs_on_key", unique: true
end
Let's say you upload an image with active storage. The above Table will be populated in a following way:
Id: 1
Key: sy2y1ytw7zob6mwtnwc1552cdp5l
Filename: Screenshot 2020-10-02 125405.png
Content_type: image/png
Metadata: {"identified"=>true, "analyzed"=>true}
Byte_size: 1550
Checksum: mGPerbRMj6LXbPKhqKd4bA
To display the user avatar as an image, you must call = image_tag #user.avatar
= rails_blob_url(#user.avatar) will give you something like https://example.com/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBCZz09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--5fb10a9c8e9f2c4e7099eee21c1dc2ff0343c210/Screenshot%202020-10-02%20125405.png
= #user.avatar will give you something like #<ActiveStorage::Attached::One:0x00007f1589293dd0>
= url_for(#user.avatar) will give you something like /rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBCZz09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--5fb10a9c8e9f2c4e7099eee21c1dc2ff0343c210/Screenshot%202020-10-02%20125405.png
The second table in the migrations created by active_storage is this:
create_table "active_storage_attachments", force: :cascade do |t|
t.string "name", null: false
t.string "record_type", null: false
t.bigint "record_id", null: false
t.bigint "blob_id", null: false
t.datetime "created_at", null: false
t.index ["blob_id"], name: "index_active_storage_attachments_on_blob_id"
t.index ["record_type", "record_id", "name", "blob_id"], name: "index_active_storage_attachments_uniqueness", unique: true
end
basically it connects the blob with a specific record in your application (basically the avatar is a blob, the #user is the record, and they are associated via the active_storage_attachments table.
So you can also run the following commands on the #user.avatar method:
#user.avatar.name #Avatar
#user.avatar.record_type #User
#user.avatar.record_id #2 (user id)
#user.avatar.blob_id #1 (blob id)
Now, below is an example url to the above file stored on AWS S3:
https://corsego-production.s3.eu-central-1.amazonaws.com/3gbpl68kckpkyrbjoslsl254th0u?response-content-disposition=inline&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGgaCmRzBFAiBSJ2QIEqs1opj%2BuCR74CDMt67ueDTTQIhAOJGGy2wfmxmGUwpQe9cyc84ZhUhuWKHdVgTUctbtGrVKdinkpg0w7OikcyNYpnbq%2FefcTmEgRvIlPO%2B0itFxUr8mKUvnDYSuKkri8zL3ohM4h9STzhTsnavAgulrcpBavL0POXg%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201002T170820Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=ASIA5RINJLY2F20201002%2Feu-central-2%2Fs3%2Faws4_request&X-Amz-Signature=0a205d713ebaaa339ad4ee09db9dfe16986e69d3ca9d62
See how the url contains a secret token and an expiry time? It is advised not to have permanent urls to files.
P.S. When connecting AWS S3 to your application, don't forget to add CORS configuration:
I am seeding some simple data in my rails program that is using a postgres database.
Currently, it is only persisting certain dates to the database. Other times, it is showing up as null in my API, which is very odd. I will post a picture of my database, and the seeded data, as well as my JSON API.
Here is my table:
create_table "pickup_deliveries", force: :cascade do |t|
t.date "pickup_date"
t.text "pickup_location"
t.money "rate", scale: 2
t.date "delivery_date"
t.text "delivery_location"
t.boolean "local"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.integer "loaded_miles", default: 0
t.integer "deadhead_miles", default: 0
end
Here is my seeded data:
PickupDelivery.create(:pickup_date => '05-10-2019', :pickup_location => 'Kansas City, MO', :rate => '550.00', :delivery_date => '05-11-2019', :delivery_location => 'Wichita, KS', :local => false, :loaded_miles => '550', :deadhead_miles =>'230');
PickupDelivery.create(:pickup_date => '05-20-2019', :pickup_location => 'Kansas City, MO', :rate => '550.00', :delivery_date => '05-25-2019', :delivery_location => 'Wichita, KS', :local => false, :loaded_miles => '550', :deadhead_miles =>'230');
Here is the persisted data in my JSON API:
[
{
id: 1,
pickup_date: "2019-10-05",
pickup_location: "Kansas City, MO",
rate: "550.0",
delivery_date: "2019-11-05",
delivery_location: "Wichita, KS",
local: false,
created_at: "2019-05-09T16:14:35.312Z",
updated_at: "2019-05-09T16:14:35.312Z",
loaded_miles: 550,
deadhead_miles: 230
},
{
id: 2,
pickup_date: null,
pickup_location: "Kansas City, MO",
rate: "550.0",
delivery_date: null,
delivery_location: "Wichita, KS",
local: false,
created_at: "2019-05-09T16:14:35.319Z",
updated_at: "2019-05-09T16:14:35.319Z",
loaded_miles: 550,
deadhead_miles: 230
}
]
As you can see, the dates were both inputted in the same format, but only one came out as intended while the other came out as null
Thanks for reading.
The reason is that you are passing an invalid format in the second record
# Record 1
:delivery_date => '05-11-2019' # dd-mm-yyyy
# to
delivery_date: "2019-11-05"
# Record 2
:delivery_date => '05-25-2019' # dd-mm-yyyy
# Invalud since there is no month 25
so it becomes nil
This date is being parsed as dd-mm-yyyy
The date you are passing in the second record is invalid due to the month
I suggest you pass the date in this format 'dd-mm-yyyy' - '20-05-2019'
NOTE: Even in your first record the date in being parsed as 05 the day, 11 as the month and 2019 as the year
I'm running Rails 5.2 on Windows. I followed advice from #Tom De Leu (Ruby on Rails - Import Data from a CSV file) to import CSV data into a table as follows:
task :import_currencies => :environment do
require 'csv'
filename = File.join Rails.root, "lib/tasks/forex20sep2018.csv"
CSV.foreach(filename, :headers => true) do |row|
Currency.create!(row.to_hash)
end
end
The CSV itself looks like this:
code,name,symbol
AED,UAE Dirham,د.إ
ALL,Lek,Lek
AMD,Armenian Dram,Lek
ANG,Netherlands Antillean Guilder,ƒ
...
but when I run the rake task:
rails import_currencies
it fails with this error:
rails aborted!
ActiveModel::UnknownAttributeError: unknown attribute 'code' for Currency.
So I modified my code as follows:
task :import_currencies => :environment do
require 'csv'
filename = File.join Rails.root, "lib/tasks/forex20sep2018.csv"
CSV.foreach(filename, :headers => true) do |row|
c = Currency.new
c.code = row[0]
c.name = row[1]
c.symbol = row[2]
c.save
puts c.errors.full_messages
end
end
And this runs perfectly well. The trouble is it's neither flexible nor elegant so I'd like to know why the original code didn't work.
(FWIW, I tried changing the CSV header as follows:
'code', 'name', 'symbol'
"code", "name", "symbol"
:code, :name, :symbol
None of these helped at all.)
#Vishal asked for the table:
create_table "currencies", force: :cascade do |t|
t.string "name"
t.string "code"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.string "symbol"
t.index ["code"], name: "index_currencies_on_code"
end
After as few suggestions, I have just made a dummy CSV with one record:
code,name,symbol
AAA,Dollar,$
and running this code:
task :import_currencies_org => :environment do
require 'csv'
filename = File.join Rails.root, "lib/tasks/forex20sep2018dummy.csv"
CSV.foreach(filename, :headers => true) do |row|
puts row.to_hash
Currency.create!(row.to_hash)
end
end
... still get this error:
I'm sorry if this is a stupid question, but in my schema.rb I have several tables like
create_table "messages", :force => true do |t|
t.integer "user_id", :null => false
t.string "message", :null => false
t.datetime "created_at", :null => false
t.string "photo_file_name"
t.string "photo_content_type"
t.integer "photo_file_size"
t.datetime "photo_updated_at"
end
Is it possible to view the contents of each table i.e view each message and associated user id, message content, time created at, linked image, etc?
Thanks
A database schema represents the structure of the database, not the content of it.
If you want to access the content in your database, you would query it to do so. You can do that via command line clients (running $ rails dbconsole will try to open one for the configured database) or graphical tools like Sequel Pro (for MySQL on Mac OS X).
You can also get this through your Rails application by running $ rails console and then using the methods available through ActiveRecord (e.g. Post.all or User.where(:name => 'Billy').limit(5)).
You can use gem "annotate". It is very helpful for me.
For viewing the table structure on the rails console only need to run the table name eg If there is a table with name country. only run the 'Country '
2.4.4 :004 > Country
=> Country(id: integer, name: string, percentage: string, created_at: datetime, updated_at: datetime, is_sync: boolean)
2.4.4 :005 >