I am building a simple script to populate a MongoDB database. This is my first time using a NoSQL DB and I feel I may be thinking about this problem from a SQL DB standpoint.
The basis of this script is to populate a database that holds a few collections that relate to one another. But when I run my script, I see an invalid errors when building/saving the documents.
I have three collections; Book, Author, and Style, with the following relationships.
A Book has many Authors
An Author has many Books
An Author has many Styles
A Style has many Authors
The models are defined as followed:
# Book Model
class Book
include Mongoid::Document
include Mongoid::Timestamps
field :title, type: String
validates :title, presence: true
has_and_belongs_to_many :authors
index({ title: 'text' })
end
# Author Model
class Author
include Mongoid::Document
include Mongoid::Timestamps
field :name, type: String
validates :name, presence: true
has_and_belongs_to_many :books
has_and_belongs_to_many :styles
index({ name: 1 }, { unique: true })
end
# Style Model
class Style
include Mongoid::Document
include Mongoid::Timestamps
field :type, type: String
validates :type, presence: true
has_and_belongs_to_many :authors
index({ type: 1 }, { unique: true, name: "type_index" })
end
And then this is my script:
# script.rb
book = Book.new
book.title = "Good Omens"
['Neil Gaiman', 'Terry Pratchett'].each do |author_name|
author = Author.find_by(name: author_name)
if author.nil?
author = Author.new(name: author_name)
end
# a list of writing styles this author can have
# pretend that there's a list of styles per author
literary_styles.each do |style_name|
style = Style.find_by(type: style_name)
if style.nil?
author.styles.build(Style.new(type: style_name))
else
unless author.styles.include? style.id
author.styles << style
end
end
end
author.valid? #=> false
author.errors #=> #messages={:styles=>["is invalid"]}
book.author.build(book.attributes)
book.save
end
The Book document is created, but the Author and Style do not persist due to the invalid Style validation error. I wish I could see exactly what is causing the validations to fail, but the messaging is very vague. I suspect it is coming from some built in validation from the has_and_belongs_to_many relation between Author and Style but I can't put my finger on it.
What I find interesting is that the Book document has an author_ids property which is populated with id's but when I jump into the console, there are no authors that can be pulled up or tied to the Book.
Happy to give more info if needed.
I think I'd need to know a bit more to definitively tell you what the issue is -- What version of Mongoid are you using? Are you running this script on an empty database, or do the Author and Style documents already exist? What do those look like?
That being said, I do see a couple of bugs in your script, though I'm not sure if they're causing your problem:
if style.nil?
author.styles.build(Style.new(type: style_name))
else
...
I believe the line inside the if statement will throw an error. Instead, it should say author.styles.build(type: style_name).
unless author.styles.include? style.id
author.styles << style
end
The expression after unless is always going to evaluate to false because author.styles is an array of Style objects, not style ids. Instead, it should probably say author.style_ids.include? style.id or author.styles.include? style
Let me know if that helps! I'm happy to debug some more if you provide me with the extra info I requested.
Related
I'm building a system that has some records in tables that are template records, which are viewable by all accounts and can be later copied to create live records for an individual account.
The reasoning behind this design decision is that template records and live records share 95%+ of the same code, so I didn't want to create a separate table to track mostly the same fields.
For instance, I have a workflows table:
id:integer
account_id:integer
name:string (required)
is_a_template:boolean (default: false)
is_in_template_library:boolean (default: false)
In this table, I have some records that are templates. When I go to create a new live record, I can use a template record:
# workflows_controller.rb (pseudo-code, not fully tested)
def create
#workflow_template = Workflow.where(is_a_template: true).find_by(id: params[:workflow_template_id])
#workflow = current_account.workflows.new(workflow_params.merge(#workflow_template.dup))
if #workflow.save
...
else
...
end
end
As I build more functionality, I find that I really need 2 different models that operate differently on the table. There are several more differences, but those listed below are enough to show the differences:
class Workflow < ApplicationRecord
default_scope -> { where(is_a_template: false) }
belongs_to :account
validates :account, presence: true
validates :name, presence: true
end
class WorkflowTemplate < ApplicationRecord
default_scope -> { where(is_a_template: true) }
validates :name, presence: true
end
class WorkflowLibraryTemplate < ApplicationRecord
default_scope -> { where(is_a_template: true, is_in_template_library: true) }
validates :name, presence: true
end
As you can see, the workflows table has 3 different "types" of records:
"live" workflows that belong to an account
template workflows that also belong to an account and are copied to create "live" workflows
library template workflows that do NOT belong to an account and can be viewed by any account, so they can copy them into their own list of templates
Question
What I'm trying to figure out, is at what point do I break up this single table into multiple tables, versus keeping the same table and having multiple models, or what solution is there to a problem like this?
The frustrating part is that there are 5+ other tables that are "children" associations of the workflows table. So if I decide that I need separate tables for each, I would end up going from 6 tables to something like 18, and everytime I add a field, I have to do it to all 3 "versions" of the table.
Thus I'm very reluctant to go down the multiple tables route.
If I keep a single table and multiple models, I then end up with different version of data in the table, which isn't the end of the world. I only interact with the data through my application (or a future API I control).
Another solution I'm thinking about is adding a role:string field to the table, which operates very much like the type field in Rails. I didn't want to use STI, however, because there are too many baked-in requirements with Rails that I don't want to conflict with.
What I'm envisioning is:
class Workflow < ApplicationRecord
scope :templates, -> { where(role: "template") }
scope :library_templates, -> { where(role: "library_template") }
validates :account, presence: true, if: :account_required?
validates :name, presence: true
# If record.role matches one of these, account is required
def account_required
["live", "template"].include?(role.to_s.downcase)
end
end
This seems to address several of the issues, keeps me with 1 table and 1 model, but begins to have conditional logic in the model, which seems like a bad idea to me as well.
Is there a cleaner way to implement a templating system within a table?
So what you are looking at here is called Single Table Inheritance. The models are called polymorphic.
As far as when to break up the STI into distinct tables, the answer is: when you have enough divergence that you start having specialized columns. The problem with STI is that let's say WorkFlows and WorkFlowTemplate start to diverge. Maybe the template starts getting a lot of extra attributes as columns that do not correspond to plain old workflows. Now you have lots of data that is empty for one class (or not needed) and useful and necessary for the other. In this case, I'd probably break the tables apart. The real question you should ask is:
How far will these models diverge from each other in terms of requirements?
How soon will this happen?
If it happens very late in the life of my app:
Will it be difficult/impossible to migrate these tables due to how many rows/how much data I have?
Edit:
Is there a cleaner way? In this specific case, I don't think so given a template and a copy of that template, are likely to be tightly coupled to each other.
The approach I took is decomposition by responsibility.
Decomposition by responsibility:
Right now, you have 3 different sources of data and 2 different ways to create/validate a workflow.
In order to achieve that, you can introduce the concept of Repositories and FormObjects.
Repositories are wrapper objects that will abstract the way you query your model. It doesn't care if it is the same table or multiple. It just knows how to get the data.
For example:
class Workflow < ApplicationRecord
belongs_to :account
end
class WorkflowRepository
def self.all
Workflow.where(is_a_template: false)
end
end
class WorkflowTemplateRepository
def self.all
Workflow.where(is_a_template: true)
end
end
class WorkflowLibraryTemplateRepository
def self.all
Workflow.where(is_a_template: true, is_in_template_library: true)
end
end
This makes sure that no matter what you decide in the future to do, you will not change other parts of the code.
So now let's discuss FormObject
FormObject will abstract the way you validate and build your objects. It might not be a great addition right now but usually, pays off in the long run.
For example
class WorkFlowForm
include ActiveModel::Model
attr_accessor(
:name,
:another_attribute,
:extra_attribute,
:account
)
validates :account, presence: true
validates :name, presence: true
def create
if valid?
account.workflows.create(
name: name, is_a_template: false,
is_in_template_library: false, extra_attribute: extra_attribute)
end
end
end
class WorkflowTemplateForm
include ActiveModel::Model
attr_accessor(
:name,
:another_attribute,
:extra_attribute
)
validates :name, presence: true
def create
if valid?
Workflow.create(
name: name, is_a_template: true,
is_in_template_library: false, extra_attribute: extra_attribute)
end
end
end
class WorkflowLibraryTemplateForm
include ActiveModel::Model
attr_accessor(
:name,
:another_attribute,
:extra_attribute
)
validates :name, presence: true
def create
if valid?
Workflow.create(
name: name, is_a_template: true,
is_in_template_library: true, extra_attribute: extra_attribute)
end
end
end
This approach helps with extendability as everything is a separate object.
The only drawback of that is that In my humble opinion, WorkflowTemplate and WorkflowLibraryTemplate are semantical the same thing with an extra boolean but that's an optional thing you can take or leave.
I can't figure how to properly hook a view / form for searching through my docs with elasticsearch + mongoid.
My Searchkick config is working – I get the results inside the rails console. But I don't know exactly where to put things in what order in the controller/model/view to make it work. From the rails console I get the search results, docs are properly mapped and everything is fine.
I can't find a working example of an simple search form with searchkick including an working example of how a model, controller and a view should look like.
Anybody got an working example to checkout with Rails 4.1rc1 / Mongoid4 / Elasticsearch 1.0.1?
I've got rails 4.0.2, Mongoid4, and ElasticSearch >= 1
Anyway, this works for us in the controller:
class CropSearchesController < ApplicationController
def search
query = params[:q].to_s
#crops = Crop.search(query,
limit: 25,
partial: true,
misspellings: {distance: 2},
fields: ['name^20',
'common_names^10',
'binomial_name^10',
'description'],
boost_by: [:guides_count]
)
if query.empty?
#crops = Crop.search('*', limit: 25, boost_by: [:guides_count])
end
# Use the crop results to look-up guides
crop_ids = #crops.map { |crop| crop.id }
#guides = Guide.search('*', where: {crop_id: crop_ids})
render :show
end
end
And this is our model:
class Crop
include Mongoid::Document
include Mongoid::Timestamps
include Mongoid::Slug
searchkick
field :guides_count, type: Fixnum, default: 0
field :name
field :common_names, type: Array
validates_presence_of :name
field :binomial_name
field :description
belongs_to :crop_data_source
field :sun_requirements
field :sowing_method
field :spread, type: Integer
field :row_spacing, type: Integer
field :height, type: Integer
embeds_many :pictures, cascade_callbacks: true, as: :photographic
accepts_nested_attributes_for :pictures
def search_data
as_json only: [:name, :common_names, :binomial_name, :description, :guides_count]
end
slug :name
end
And then you just accesses the #crops in the view like you would normally.
You can have a look at our source code on github: https://github.com/openfarmcc/OpenFarm
(Just stumbled across this question while looking for how to do the ordering, figured it out, but thought I'd just copy paste this here in case anyone finds this useful).
My application model allows Patients to have CustomFields. All patients have the same customs fields. Customs fields are embedded in the Patient document. I should be able to add, update and remove custom fields and such actions are extended to all patients.
class Patient
include Mongoid::Document
embeds_many :custom_fields, as: :customizable_field
def self.add_custom_field_to_all_patients(custom_field)
Patient.all.add_to_set(:custom_fields, custom_field.as_document)
end
def self.update_custom_field_on_all_patients(custom_field)
Patient.all.each { |patient| patient.update_custom_field(custom_field) }
end
def update_custom_field(custom_field)
self.custom_fields.find(custom_field).update_attributes({ name: custom_field.name, show_on_table: custom_field.show_on_table } )
end
def self.destroy_custom_field_on_all_patients(custom_field)
Patient.all.each { |patient| patient.remove_custom_field(custom_field) }
end
def remove_custom_field(custom_field)
self.custom_fields.find(custom_field).destroy
end
end
class CustomField
include Mongoid::Document
field :name, type: String
field :model, type: Symbol
field :value, type: String
field :show_on_table, type: Boolean, default: false
embedded_in :customizable_field, polymorphic: true
end
All pacients have the same customs fields embedded in. Adding a custom field works very well. My doubt is about updating and destroying.
This works, but it is slow. It makes a query for each pacient. Ideally I would just be able to say to MongoDB 'update the document with id: that is embedded in the array *custom_fields* for all documents in the Patient collection'. Idem for destroy.
How can I do this in Mongoid?
I am using Mongoid 3.1.0 & Rails 3.2.12
I don't think there is a way you can do that with a good efficiency with embedded documents.
Maybe you should consider having a referenced relationship between your models, so that you can use the delete_all and update_all methods on the collection.
So, I am having some issues with user authentication in embedded documents. I have two documents, one embedded in the other. A business has many members. The models look like this:
class Member
include Mongoid::Document
field :username, type: String
field :password, type: String
embedded_in :business
validates :username, :presence => true, :uniqueness => true, :length => 5..60
end
class Business
include Mongoid::Document
field :name, type: String
embeds_many :members
end
The problem is that it isn't validating the username's uniqueness in each model. When I save a member within a business, I can save a thousand of the same name. This of course is not going to work for a good authentication system. I am using Mongoid 2, Rails 3, and Ruby 1.9
This is a normal behavior when using embedded documents as explained here: MongoID validation
validates_uniqueness_of
Validate that the field is unique in the database: Note that for
embedded documents, this will only check that the field is unique
within the context of the parent document, not the entire database.
I think you want to try to create an Index in the username field that would ensure uniqueness among all the objects of that collection. Something like this:
ensureIndex({username:1},{unique:true});
EDIT: If you want Mongo to throw exception if a document with the same index value exists, you must avoid Mongo to do the “fire and forget” pattern. This means that the database will not wait for a response when you perform an update/write operation on a document.
And you want to pass this parameter: safe:true. By doing so Mongo should raise an exception if for any reason the document can't be inserted.
Assume, that a user wants to create a set consisting of items created by other users. Mongoid Document for item has versioning, and the user who creates the set might not enjoy changes the item authors do with the items of the set. Therefore I would like the set document to refer to specific versions of items, allowing the set author to update item references if wanted. I am planning adding an array of item version numbers to the set document, and some methods for getting set items of certain version and for updating the item versions. Do you find this approach reasonable? How would you solve this problem?
class Item
include Mongoid::Document
include Mongoid::Paranoia
include Mongoid::Versioning
field :title, type: String
has_and_belongs_to_many :item_sets
end
class ItemSet
include Mongoid::Document
field :name, type: String
field :item_versions, type: Array
has_and_belongs_to_many :items
end
I solved problems like this by creating a model in the "middle" like "ItemReference"
MongoDB is a document store and not a relational database, so it is legitimate to store duplicate information when necessary. MongoDB has the ability to store embedded documents, so we're gonna use this great feature.
The ItemReference holds all crucial information about the Item which is needed for creating a view. This reduces the queries on the view side but in increases the queries on the insert/update side.
The thing is that you need a "composite primary key" that consists of the item_id and the version number.
Let's talk code:
The Item Model:
class Item
include Mongoid::Document
include Mongoid::Paranoia
include Mongoid::Versioning
field :title, :type => String
# create a reference
def to_reference
# create new reference, containing all crucial attributes for displaying
ItemReference.new(
:item_id => self._parent.nil? ? self.id : self._parent.id,
:version => self.version,
:title => self.title
)
end
# get a certain version of this item
def get_version(version_number)
return self if version_number == self.version
self.versions.where(:version => version_number).first
end
end
The ItemSet Model
class ItemSet
include Mongoid::Document
field :name, :type => String
embeds_many :item_references
end
The ItemReference Model
class ItemReference
include Mongoid::Document
embedded_in :item_sets
field :title, :type => String
# this points to the newest version
belongs_to :item
# get the original version
def original
self.item.get_version(self.version)
end
# update this reference to a certain version
def update_to!(new_version)
new_version = self.item.get_version(new_version)
if new_version.present?
# copy attribute, except id
self.attributes = new_version.to_reference.attributes.reject{|(k,v)| k == "_id"}
self.save
else
# version not found
false
end
end
# update to the newest version
def update_to_head!
self.update_to!(self.item.version)
end
end
This combination allows you to create sets holding Items with different versions, and you can update certain ItemReferences in the set to a specific version.
Here's an example:
first = Item.create(:title => 'Item 1')
first.title = 'Item 1.1'
first.save
myset = ItemSet.create(:title => 'My Set')
myset.item_references << first.to_reference
myset.save
first.title = 'Item 1.2'
first.save
p myset.item_references.first.title # prints Item 1.1
p myset.item_references.first.update_to_head!
p myset.item_references.first.title # prints Item 1.2
p myset.item_references.first.update_to!(1)
p myset.item_references.first.title # prints Item 1