I'm developing an application with Grails 2.2.2 / MySQL.
I'd like to integrate the Searchable plugin, however my application is a multi-tenant saas app, and I need to restrict the results that are returned somehow. So for instance if I am a user of my application belonging to organisation A and I search for products, I should not see any products that belong to organisation B.
I currently have a service that carries out this kind of data partitioning when using the regular grails CRUD pages which works by applying restrictions to every criteria query executed. I've also integrated this service with the Filterpane plugin, for 'advanced search' style queries.
Is it possible to filter Searchable results in a similar way?
It's been a while since I looked into this, but I did come across an interesting article about partitioning a lucene index so that 1) there is one index for the application and 2) search results for organisation A are not biased in any way by the stuff that has been indexed for organisation B (I didn't even consider that second point).
https://community.jivesoftware.com/community/developer/blog/2013/06/24/a-new-experimental-approach-to-implement-multi-tenancy-with-lucene-4
Also, for my application, I've sort of met the 'full text searching' requirement (for the short term at least) by using the excellent Quick Search plugin for grails.
Hopefully someone will find these resources useful.
Related
I am trying to implement a feature to my project (kind of like a social media site) that could be either basic or complex and I am not sure if I am going to take forever reinventing the wheel or just go on a crazy tangent that won't work. I just need to "check in" so to say.
I am going to use Facebook terminology as an example to simplify the concept but implement similar features with different names. In Facebook you have Pages and Groups, which are similar yet have slight differences (from now, I will call the collection of these DataSets). Both of these can have multiple admins or followers, which are all User roles, and each User can have roles for multiple Groups and multiple Pages (one role per Group or Page). Then for example, you can click a drop down to change your account to post as a Page you are an admin for.
Essentially, the concept I am describing is where a single User can have a role for multiple different types of DataSets. For example, a single User could follow 30 different Pages and 10 different Groups, and be an admin for one Group and two Pages. Does the concept I am describing belong to a particular concept or software design pattern? I am finding it really hard to describe this feature without using Facebook examples.
I have a strategy to implement this type of functionality in Rails, but I feel like using this strategy would be making the problem harder than it is and there is a fancy rails way of doing it, or a Gem, but I just don't know how to research it due to lack of terminology to describe my problem.
Current strategy is:
I have a Users table from Devise. Pages and Groups are each individual models and have their own tables. I have matching database tables to make the many-to-many relationships between Pages and Users, along with Groups and Users (e.g. 3 column design, column for the user_id, column for the page_id and the type of relationship such as admin or follower). Let's call these Group_User and Page_User. I am being flexible at the moment as I may add more DataSets similar to Page and Group.
Then for the Devise User table, I have an extra two columns to track the DataSet that the User is an admin for and currently posting as. One column is for the DataSet type and the other for the id for this instance (e.g. [Group,1] is stored in these two columns to represent Group with group_id:1 and [Page,3] is used to represent Page with page_id:3). These two columns can be checked when displaying options relevant for admins in that Group/Page and a simple drop down at the top of the site changes the values in these columns to any of the Pages/Groups the logged in User is an admin for. This way, one User login can take on many admin roles and change between these easily as needed.
Is there a better way to do this in Rails, such as a gem or specific design pattern? Or am I on track to implement these features myself? I think I understand the problem but my solution just seems simple/raw and possibly might have unintended consequences later down the track (e.g. it seems database intensive).
One way I was thinking of doing this was making a concern that includes methods to build the relationships and pass in the name of the DataSet as an argument, just so I am not rewriting the same methods for Pages, then Groups, then whatever comes next.
I looked at other solutions such as polymorphic typing (which I think is good for if each user only had one role or only managed relationships for one group or one page) and Single Table Inheritance (but I think my Pages and Groups might be too different for this to work). I thought about using inheritance as well (e.g. a parent for both Group and Page) but I am not sure this helps much.
I am just a guy that studied too much computer science and not enough software engineering. Any tips on how to simplify this problem or just a simple "yeah that will work" would be really helpful!
I think you are going great in the database design. Once participated in a social media application like yours which had similar type of design. Your design seems much better than the one I worked with. In my opinion this type of applications are supposed to be database extensive.
There are several design patterns used in RoR. One I heavily use is Service Object Pattern to maintain thin controller and models. Also it helps me to write reusable class.
Another one I like is the Presenter Pattern to simplify views.
You can have a details look at this blog post for more design pattern ideas.
I'm looking for a solution that combines the power of Google Site Search with Elasticsearch for a Rails 4 app preferably as a Rails gem.
As far as I can tell the most popular search solutions, ThinkingSphinx, ElasticSearch with the tire gem, Sunspot Solr and lastly PostreSQL searching functionality, all seem to only handle database searching and do not have page/html template searching functionality.
If I have this wrong then please correct me and I will happily pick one of the above.
The site contains mainly static HTML so the Google Site Search api is the obvious solution but there are some ActiveRecord results that should also be included in the results of a search.
If there really isn't a simple combined solution then I would appreciate any pointers as to how to achieve the merging of an elasticsearch (my preffered AR search solution) with a google site search where the results happen to include the same pages so page results are not duplicated.
To rephrase your question, you want to combine results of Google Site Search with custom search results provided by Sphinx, Solr or Elasticsearch?
First, you cannot really customize the content of Google Site Search results easily. You can customize the design, and could employ some JavaScript tricks to "merge" its results with another data source, but I'd say the approach is not something maintainable and, more importantly, usable.
Notice, that you can display the search results from Elasticsearch with Tire in the same way as ActiveRecord instances, all the usual Rails helpers such as url_for etc. work. The easiest way to evaluate the integration is to generate the example application with the Rails template.
If you want to combine the results from ActiveRecord data and the results from any static pages you might have on your site/application, it wouldn't be hard to write a simple crawler which would retrieve, parse and index the content of static pages and store it as ActiveModel-compatible documents in Elasticsearch.
In an attempt to avoid writing my own aggregator for an existent Ruby on Rails app and WordPress blog, I'd like to be able to aggregate various chatter going on in various mediums at one central page based on particular keywords, etc.
This aggregator can live either on the blog or within the Rails app, it doesn't matter.
For example, let's say I want to continually poll Google News, Twitter, public Facebook, and perhaps others, for "chocolate chip cookies." Is there a package that already facilitates this or am I in a "roll it yourself" situation?
One thing to consider is using HTTParty to deal with the APIs those services expose...
I originally wrote my Ruby on Rails application for one client. Now, I am changing it so that it can be used for different clients. My end-goal is that some user (not me) can click a button and create a new project. Then all the necessary changes (new schema, new tables, handling of code) are generated without anyone needing me to edit a database.yml file or add new schema definitions. I am currently using the SCOPED access. So I have a project model and other associated models have a project_id column.
I have looked at other posts regarding multi-tenant applications in Rails. A lot of people seem to suggest creating a different schema for each new client in Postgres. For me, however, it is not much useful for a new client to have a different schema in terms of data model. Each client will have the same tables, rows, columns, etc.
My vision for each client is that my production database first has a table of different projects/clients. And each one of those tables links to a set of tables that are pretty much the same with different data. In other terms a table of tables. Or in other terms, the first table will map to a different set of data for each client that has the same structure.
Is the way I explained my vision at all similar to the way that Postgres implements different "schemas"? Does it look like nested tables? Or does Postgres have to query all the information in the database anyway? I do not currently use Postgres, but I would be willing to learn if it fits the design. If you know of database software that works with Rails that fits my needs, please do let me know.
Right now, I am using scopes to accomplish multi-tenant applications, but it does not feel scalable or clean. It does however make it very easy for a non-technical user to create a new project provided I give them fillable information. Do you know if it is possible with the multi-schema Postgres defintion to have it work automatically after a user clicks a button? And I would prefer that this be handled by Rails and not by an external script if possible? (please do advise either way)
Most importantly, do you recommend any plugins or that I should adopt a different framework for this task? I have found Rails to be limited in some cases of abstraction as above and this is the first time I have ran into a Rails-scaling issue.
Any advice related to multi-tenant applications or my situation is welcome. Any questions for clarification or additional advice are welcome as well.
Thanks,
--Dave
MSDN has a good introduction to multi-tenant data architecture.
At one end of the spectrum, you have one database per tenant ("shared nothing"). "Shared nothing" makes disaster recovery pretty simple, and has the highest degree of isolation between tenants. But it also has the highest average cost per tenant, and it supports the fewest tenants per server.
At the other end of the spectrum, you store a tenant id number in every row of every shared table ("shared everything"). "Shared everything" makes disaster recovery hard--for a single tenant, you'd have to restore just some rows in every shared table--and it has the lowest degree of isolation. (Badly formed queries can expose private data.) But it has the lowest cost per tenant, and it supports the highest number of tenants per server.
My vision for each client is that my production database first has a
table of different projects/clients. And each one of those tables
links to a set of tables that are pretty much the same with different
data. In other terms a table of tables. Or in other terms, the first
table will map to a different set of data for each client that has the
same structure.
This sounds like you're talking about one schema per tenant. Pay close attention to permissions (SQL GRANT and REVOKE statements. And ALTER DEFAULT PRIVILEGES.)
There are two railscasts on multitenancy that using scopes and subdomains and another to help with handling multiple schemas.
There is also the multitenant gem which could help with your scopes and apartment gem for handling multiple schemas.
Here is also a good presentation on multitenancy-with-rails.
Dont forget about using default scopes, while creating named scops the way you are now works it does feel like it could be done better. I came across this guide by Samuel Kadolph regarding this issue a few months ago and it looks like it could work well for your situation and have the benefit of keeping your application free of some PgSQL only features.
Basically the way he describes setting the application up involves adding the concepts of tennants to your application and then using this to scope the data at query time using the database.
I am wondering on how to implement a search functionality like Github.
Just one search box on the top header right and when searched for a keyword, displays the results for Repository, Code and User.
Is there any tutorial or example to implement this on Rails 3?
Odds are really good they're doing separate searches across the tables for the same value, then combining the results afterwards.
Use Rails to create a small form containing a text field. When it's submitted take the value of the field and do a query using that as the search term.
If you're not sure how to do queries using ActiveRecord, see "Active Record Query Interface" for a nice overview.
You will have to do several queries, one per model, and put the results together on the same view.
If your question is "how do I do full text searches on several activerecord models in a DRY way" then there are basically two paths:
The common solution, but a bit complex, is using a dedicated daemon on your machine, like Sphinx. Sphinx is a service in (like Apache or MySQL) that indexes your content and allows you to do searches. You can use the Thinking Sphinx gem to communicate with it easily from rails. An alternative to Sphinx is Solr (there's also a gem for it called Sunspot)
If you are using Postgresql, there's a simpler alternative that doesn't require external services running on your server. Postgresql has with some full-text search capabilities built-in. There's a gem called texticle that helps using these services from rails. You can have that working very quickly.