Rails Search One Column On Multiple Saved Terms (Saved Searches In Model) - ruby-on-rails

One table, one column ('headline' in an RSS feed reader). On the front end, I want a text area in which I can enter a comma-separated list of search terms, some multi-word, like for 'politics':
rajoy, pp, "popular party", "socialist party", etc
This could either be stored as part of a separate search model or as a keyword column on the 'category' or 'story' models, so they can be edited and improved with different terms from the front end, as a story develops.
In the RSS reader, have a series of links, one for each story or category, that, on being clicked return the headlines that contain one (or more) of the search terms from the stored list.
In a later version, it would be good to find headlines containing several of the terms in the list, but let's start simple.
Have been doing lots of reading about postgres, rails, different types of searches and queries, but can't seem to find what I want, which I understand is basically "search 'headlines' column against this list of search terms".
Sounds like it might be an array thing that's more to do with controllers in Rails than postgres, or cycling through a giant OR query with some multi-word terms, but I'm not sure.
Does anyone have any better pointers about how to start?

Users
If this will be user specific, I would start with a User model that is responsible for persisting each unique set of search terms. Think logon or session.
Assuming you use the Category method mentioned before, and assuming there's a column called name. Each search term would be stored as a separate instance in the database. Think tags.
headlines that contain one (or more) of the search terms from the stored list
Categories
Since each Category has many terms, and all the queries are going to be OR queries, a model that joins the User and Category, storing a search term would be appropriate.
I'm also assuming you have a Story model that contains the actual stories, although this may not be persisted in the database. I'm predicting your story model has a heading and a body.
Terminal Console
rails generage model SearchTerm query:string user:references category:references && rake db:migrate
Models
On your existing User and Category models you would add:
# app/models/user.rb
has_many :search_terms
has_many categories, through: :search_terms
# app/models/category.rb
has_many :search_terms
has_many :stories
Rails Console
This will automatically make it possible for you to do this:
#user = User.last # this is in the console, just to demonstrate
#category = Category.find_by_name("politics")
#user.search_terms.create {query: "rajoy", category: #category}
#user.search_terms.create {query: "pp", category: #category}
#user.search_terms.where(category_id: #category.id).pluck(:query)
-> ['rajoy', 'pp']
Controllers
What you will want to do with your controller (probably the Category controller) is to parse your text field and update the search terms in the database. If you want to require commas and spaces to separate fields, you could do:
#user.search_terms.where(category: #category).delete_all
params[:search_term][:query].split(", ").map{|x| x.gsub("\"", "")}.each do |term|
#user.search_terms.create({category: #category, query: term})
end
Front End
Personally though, I'd make the front end a bit less complicated to use, like either just require commas, no quotes, or just require spaces and quotes.
Search
For the grand finale, for the Stories to be displayed that have search terms in their heading:
Story.where(#user.search_terms.where(category: #category).pluck(:query).map { |term| "heading like '%#{term}%'" }.join(" OR "))
I would recommend using pg_search gem rather than trying to maintain complicated queries like this.
Note: I'm sure there are errors in this, since I wasn't able to actually create the entire app to answer your questions. I hope this helps you get started with what you actually need to do. I encourage you as you work through this to post questions that have some code.
References
Rails guides: choosing habtm or has many through
gem 'pg_search'
Stack Overflow: Search a database based on query

Related

Postgres HStore vs HABTM

I am building an app that has and model that can be tagged with entries from another model, similar to the tagging function of Stackoverflow.
For example:
class Question < ActiveRecord::Base
has_and_belongs_to_many :tags
end
class Tag < ActiveRecord::Base
has_and_belongs_to_many :questions
end
I am debating between just setting up a has_and_belongs_to_many relationship with a join table, or adding the tags to a hash using Postgres' hstore feature.
Looking for anyone that has had a similar experience that can speak to performance differences, obstacles, or anything else that should persuade me one way or another.
EDIT:
I think I should also mention that this will be a API that will be using an AngularJS frontend.
You are describing the topic of a great debate:) Normalization vs denormalization. Using many to many allows you to do nice queries such as "how many people use a certain tag" in a very simple way. HStore is very nice as well but you end up with thousands of the same tags everywhere. I use both approaches in different projects but the real problem comes when you decide one day to move your database. With Hstore you will be stuck to postgresql or have to rewrite your code. If super high speed is important as well as querying different ways and you often want to load a user record in one fellow swoop as fast as possible and show all used tags I normally do both: create a many to many relationship as tags are normally also connected to more objects like user has many tags from tags table and tags are connected to let's say brands which are connected to products and so on.
Then I create an additional field with hstore or json objects on the user table which adds every tag or removes it when the many to many relationship is destroyed.
To give you an example: in on of my projects I have companies (almost 10 million) who are interested in certain keywords and their ranking on google. This table has millions of rows but connected only to 2 million keywords which are connected to search results. This way I can quickly query which result is searched for by how many people and who they are.
If a customer opens their key word search page I load their keywords from a text column with json which is faster than going through the table.

How to store many optional fields in Rails?

I'm working on an app that displays Products. Each Product is a model (Product) with some common info like name, brand etc. To separate what kind of Product we're looking at we have a product_type (like: Phone, TV, Sofa etc) which is basically a tinyint field (e.g. 1 = Phone, 2 = TV).
Now, depending on the product_type each Product can have different options: if it's a phone the Product should have extra info associated (like weight, does it have LTE, does it have a front-facing camera etc), a TV should have info like display size and so on.
My question is: What is the best /or/ easiest way to add this extra data to the Product, depending on the type of Product?
I was thinking of having an extra model ProductOptions with fields product_id, option_type, option_value (VARCHAR) which would store extra fields, but I'm not sure if this is optional for performance & search. In this case, searching would mean finding all ProductOptions that match a given criteria (e.g. po = ProductOption.where(:option_type => "LTE", :option_value => "yes") ) and then doing a Product.findAllIn(po) to find the actual products.
Or should I go with Postgres and use HStore? Is HStore efficient when searching?
Any ideas?
Rails newbie, all code is pseudo-code
If you're already looking at ProductType, you might make this Single-Table Inheritance and actually define different product types. That would be easier to code anyway. In this case:
class Product < ActiveRecord::Base
# common functionality goes here
end
class Sofa < Product
# sofa specific functionality goes here
end
You can then put all the options on the product table, but have a different attr_accessible, validations, and views for the different product types.
The best part about this setup is it 'just works' in rails, all you have to do is put a type column on the products table. Also, it lets you find all sofas like so:
Sofa.all
and all everything like so
Product.all
As far as database performance, well, you'll end up with a fair number of null fields, but all your attributes will be searchable (unlike serialization), and you only need to query on one table. So, I would expect this to outperform any other option I know of.
Plus, it's nice and object oriented. :)
Good luck, welcome to Rails!

rails semi-complex STI with ancestry data model planning the routes and controllers

I'm trying to figure out the best way to manage my controller(s) and models for a particular use case.
I'm building a review system where a User may build a review of several distinct types with a Polymorphic Reviewable.
Country (has_many reviews & cities)
Subdivision/State (optional, sometimes it doesnt exist, also reviewable, has_many cities)
City (has places & review)
Burrow (optional, also reviewable ex: Brooklyn)
Neighborhood (optional & reviewable, ex: williamsburg)
Place (belongs to city)
I'm also wondering about adding more complexity. I also want to include subdivisions occasionally... ie for the US, I might add Texas or for Germany, Baveria and have it be reviewable as well but not every country has regions and even those that do might never be reviewed. So it's not at all strict. I would like it to as simple and flexible as possible.
It'd kinda be nice if the user could just land on one form and select either a city or a country, and then drill down using data from say Foursquare to find a particular place in a city and make a review.
I'm really not sure which route I should take? For example, what happens if I have a Country, and a City... and then I decide to add a Burrow?
Could I give places tags (ie Williamsburg, Brooklyn) belong_to NY City and the tags belong to NY?
Tags are more flexible and optionally explain what areas they might be in, the tags belong to a city, but also have places and be reviewable?
So I'm looking for suggestions for anyone who's done something related.
Using Rails 3.2, and mongoid.
I've built something very similar and found two totally different way that both worked well.
Way 1: Country » Subcountry » City » Neighborhood
The first way that worked for me is to do it with Country, Subcountry, City, Neighborhood. This maps well to major geocoding services and is sufficient for most simple uses. This can be STI (as in your example) or with multiple tables (how I did it).
In your example you wrote "Subdivision/State". My two cents is to avoid using those terms and instead use "Subcountry" because it's an ISO standard and you'll save yourself some confusion when another developer thinks a subdivision is a tiny neighborhood of houses, or when you have a non-U.S. country that doesn't use states, but instead uses provinces.
This is what I settled on after many experiments with trying model names like Region, District, Area, Zone, etc. and abandoning these as too vague or too specific. In your STI case it may be fine to use more names.
One surprise is that it's a big help to write associations that go multi-level, for example to say country.cities (skipping subcountry). This is because sometimes the intermediary model doesn't exist (i.e. there's no subcountry). In your STI, this may be trickier.
Also you get a big speedup if you denormalize your tables, so for example my city table has a country field. This makes updating info a bit trickier but it's worth it. Your STI could inmplement an equivalent to this by using tags.
Way 2: Zones that are lists of lat/lng shapes with bounding boxes
The second way is to use an arbitrary Zone model and store latitude longitude shapes. This gives you enormous flexibility, and you can pre-calculate when shapes contain other shapes, or intersect them. So your "drill down" becomes "show me shapes immediately inside this one".
Postgres has some good geocoding helpers for this, and you can speed up lookups by doing bounding boxes of min/max lat/lng. We also stored data like the expected center point of a Zone (which is where we would drop a pin on a map), and a radius (useful for calculating queries like "show me all the x items within y distance).
With this approach we were able to do interesting zones like "Broadway in New York" which isn't really a neighborhood so much as long street, and "The Amazon Basin" which is defined by the river, not a country.
STI Model with Ancestry and with Polymprphic Relation
I built something similar for previous projects, and went for STI with ancestry because it is very flexible and allows to model a tree of nodes. Not all intermediate nodes have to exist (as in your example of State/Subdivision/Subcountry).
For Mongoid there are at least two ancestry gems: mongoid-ancestry and mongestry (links below).
As an added benefit of using STI with ancestry, you can also model other location-related nodes, let's say restaurants or other places.
You can also add geo-location information lat/lon to all your nodes, so you can geo-tag them.
In the example below I just used one set of geo-location coordinates (center point) - but you could of course also add several geo-locations to model a bounding box.
You can arrange the nodes in any order you like, e.g. through this_node.children.create(...) .
When using MySQL with ancestry, you can pass-in the type of the newly created node. There must be a similar way with mongoid-ancestry (haven't tried it).
In addition to the tree-structured nodes, you can use a polymorphic collection to model the Reviews, and also Tags (well, there's a gem for acts_as_taggable, so you don't have to models Tags yourself).
Compared to modeling every class with it's own collection, this STI approach is much more flexible and keeps the schema simple. It's very easy to add a new type of node later.
This paradigm can be used with either Mongoid or SQL data stores.
# app/models/geo_node.rb
class GeoNode # this is the parent class; geo_nodes is the table name / collection name.
include Mongoid::Document
has_ancestry # either this
has_mongestry # or this
has_many :reviews, :as => :reviewable
field :lat, type: Float
field :lon, type: Float
field :name, type: String
field :desc, type: String
# ...
end
# app/models/geo_node/country.rb
class Country < GeoNode
end
# app/models/geo_node/subcountry.rb
Class Subcountry < GeoNode
end
# app/models/geo_node/city.rb
class City < GeoNode
end
# app/models/review.rb
class Review
include Mongoid::Document
belongs_to :reviewable, :polymorphic => true
field :title
field :details
end
Check these links:
mongoid-ancestry gem https://github.com/skyeagle/mongoid-ancestry
mongestry gem https://github.com/DailyDeal/mongestry
mongoid-tree gem https://github.com/benedikt/mongoid-tree
Gist on Mongoid STI: https://gist.github.com/507721
ancestry gem (for MySQL)
A big thanks to Stefan Kroes for his awesome ancestry gem, and to Anton Orel for adapting it to Mongoid (mongoid-ancestry). ancestry is of the most useful gems I've seen.
Sounds like a good candidate for nested routes/resources. In routes.rb, do something like:
resources :cities do
resources :reviews
end
resources :countries do
resources :reviews
end
resources :places do
resources :reviews
end
Which should produce something along the lines of rake routes:
reviews_cities GET /cities/:id/reviews {:controller=>"reviews", :action=>"index"}
reviews_countries GET /countries/:id/reviews {:controller=>"reviews", :action=>"index"}
reviews_places GET /countries/:id/reviews {:controller=>"reviews", :action=>"index"}
...etc., etc.
In the controller action, you lookup match up the :id of reviewable record, and only send back reviews that are attached to that reviewable object.
Also, see the nested resources section of the Rails Routing Guide, and this RailsCast on Polymorphic relationships, which has a quick section on routing, and getting everything to line up properly.
I would probably keep my data model very unrestrictive, and handle any specifics related to what filters to display in the controller/view. Make a mapping table where you can map attributes (i.e. city, borough, state, country) polymorphically, also polymorphically to reviewable.
By assuming many-to-many, your schema is as flexible as it can be, and you can restrict which mappings to create using validations or filters in your models.
It's basically using tagging, like you eluded, but not really using a tags model per-se, but rather a polymorphic association to different models that all act like tags.
Keep your DB schema clean and keep the business logic in ruby.

Rails tokenized text search across fields with performance in mind

I have a Rails app on Heroku that I'm looking to increase the user-friendlyness of the search for. To do this, I'd like to allow them to text search across multiple fields on multiple models through associations. The input from the user could be a mix of text from any of these fields (and often might span multiple fields) in no particular order.
Example: if you had a car database and wanted to allow the user to search "Honda Fit 2011", where "Honda" came from the manufacturer table, "Fit" came from the model table, and "2011" came from the model_year table.
I'm thinking that I need to build a single field on the root record that contains the unique list of words from each of these fields, and then tokenize the user's input. But that would cause me to use an IN clause, which I'm not sure could benefit from full-text search plugins like pg_search.
So, my question is what's a good way to active a search like this in Rails?
I would take a look at Sunspot_rails. It uses Solr as it's search engine, but allows you index content in all sorts of fruity ways. For instance, I have models indexed with their associations pretty simply:
searchable do
text :description
text :category do
category.present? ? category.name : ''
end
end
You can then search with:
TYPES = [Asset,Product]
Sunspot.search(*TYPES) do |q|
q.fulltext search_str
end

ElasticSearch search on many types

I'm using Rails with the Tire gem (for ElasticSearch) and I need to search across multiple models. Something like:
# title is a field in all models
Tire.search :tasks, :projects, :posts, { :title => "word" }
I know I can search models one by one and then handle these results, but that should be unecessary considering ElasticSearch(Lucene) is document oriented.
Any thoughts?
Thanks,
One possibility is to see them not as distinct models. A compound model could be that every document can be an item belonging to one or many differnt submodels identified by a string constant which can be multivalued.
If you want to retrieve only results from one of those submodels you could add a fixed part to the query which identifies the set of documents belonging to this submodel.
The only caveeat is that you need to have a primary key which is unique(which is not that bad because you can use something like an implicit document key).

Resources