Heroku Rails Websolr, sunspot is not free in production? - ruby-on-rails

I've added sunspot gem in my application and tried to send it to production in heroku, but I'm trying to reindex my database, however, I'm getting an error. I did some more digging and I think I have to add websolr as an add-on? This costs $20/month. Is this the only option?
THanks

Founder of Websolr + Bonsai here (Heroku addons for Solr and Elasticsearch).
Rich's answer is pretty solid, with the exception of the SQL LIKE operator, which I do not recommend. The performance does not scale, and you're either going to sink in a lot more time than you might expect in order to eke out baseline search functionality. End result: a lot of time spent, and unhappy users.
Postgres full text search is a reasonable alternative, though the term analysis and result ranking will be lacking compared to Solr/Elasticsearch as your search traffic starts to grow in production.
You might also consider our sister service, Bonsai, which does offer a free Starter plan. It uses Elasticsearch, which means you'd want to use the official Ruby bindings for Elasticsearch rather than Sunspot.
Lastly, if you already have a production app on Heroku, you are welcome to create more than one index in your account, and share those indexes with your staging/qa and other apps.

I've done some more research and found out that there are other options if you don't want to take the websolr path. These other answers are good for some insights, but doesn't give an alternative to what can be used.
For some that's still looking, I suggest taking a look at Elastic Search
Rails Cast has a good tutorial on this as well.
And to use it with heroku, look into Bonsai which gives users a free option.
Hopefully this answer will help those that are also seeking other options than using sunspot gem with solr

Solr on Heroku uses their own add-on, which starts at $20pm:
Although I don't know why it costs up front, and doesn't have a "trial" option like many of the other Heroku Add-ons, there are certain ways around it
Full Text Search
Full text search is what you're performing, and Solr is a tool to make the process much more efficient. Despite being quite DB-expensive, you can use full text searching with Heroku, depending on your DB:
MYSQL
To perform full-text searching on MYSQL, you can simply use the "LIKE" operator with %variable% as your search phrase, like this:
SELECT * FROM `table` WHERE `name` LIKE `%benjamin%`
This basically finds all the records where the name column contains "benjamin" somewhere inside it. This is quite slow
POSTGRESQL
PostgreSQL offers more power in its full text searching, but is nonetheless still quite slow & expensive. You can read more about it here, but with rails, you can use a bunch of gems which do the task for you
We recently used a gem called textacular here: http://firststop.herokuapp.com
Here is the code we used for it:
#Search
def self.search(search)
basic_search(name: search, description: search)
end
Further Reading
You can see how full text searching works here: Any reason not use PostgreSQL's built-in full text search on Heroku?
I would recommend if you're just getting the foundations established for your app. Afterwards, you can upgrade to a more dedicated solution in the form of Solr et al
Here are

If you want to use the Heroku platform it starts for free, but you have to pay for almost every add-on, extra workers, extra storage, search engine, background tasks, you name it.
For $20/month you could also get a decent VPS, but you would have to install and manage that server by yourself.
As for sunspot/solr on Heroku, I don't think you can do that for free.

Related

Integrating GCS on a staging Jekyll website

We currently have a staging website, which has an IP address like xx.xx.xxx.xxx and we would like to have integrated and tested GCS on it before pushing it live. Is it possible?
Otherwise, is there any alternative to GCS to add a search bar in a Jekyll blog without using plugin?
Thanks!
PART I: Google
Google custom search cannot index an application that isn't available to the internet.
No, that's not entirely true. You can arrange something with Google (in theory, never done it) but it doesn't look easy. Or cheap.
You could set up a custom search for an unrelated site and embed those results in your local page, if you want to test out CSS prior to launch.
Remember, Google Custom Search also comes with ads, unless you pay. And the results tend to look like they came from Google.
PART II: Alternatives
I've looked into this extensively and I haven't come up with a good answer. Here are some not-so-good answers:
1) Tapir Search. This actually worked pretty well, but appears to have died. They do have recent twitter activity, however, so maybe worth checking back in a bit. twitter. It's basically a (free) front end for an elasticsearch server. I think. Neat service, obviously not super-dependable.
2) Go javascript. Lunr for example. There are many, many similar solutions available. Sadly, they are client-side and doing a full-text search on even a smallish blog type site can be very slow. Works okay if you limit the search to titles, but then...you're only searching titles.
3) Build a search engine server. Maybe some breed of Lucene. Upside: very robust search while keeping the snappy response of a flat HTML site. Downside: building and maintaining a search engine server is difficult, expensive and probably overkill.
4) Hosted search engine. Algolia for example. They're basically doing 3) for you. Relatively expensive (~$50/month) but well worth the cost, because, seriously, search engine servers are finicky and prone to explosions. I've never gone this route with Jekyll because I've never had a Jekyll project I was quite that serious about, but I did consider it.
If anyone has anything to add, I'd love to hear it. This question has been irritating me for a while.

Elastic Search vs Sunspot comparison on features

Couden't find any compare questions related to sunspot (Solr) to Elastic Search (Lucene)
What would be the pro's and con's on both search engines?
I saw other VS questions to get a better inside in the comparisment of 2 gems so hope this is allowed to get a better insight in the both engines for newbies ( like me ). I have looked at sunspot already but have some issues with it. So I searched
http://www.elasticsearch.org/guide/reference/api/
vs
http://sunspot.github.com/
I started working on a project that needed full text search in Ruby so naturally I started with Solr + Sunspot, but I couldn't get it to work. It was a pain just the get them connected, then tried to figure out if the document indexed correctly, figure out the runtime classpath so I can add additional analyzer/tokenizer classes, editing config.xml/schema.xml, etc. Solr numDocs clearly said it received and indexed them but I couldn't get any query results. I just gave up after a couple days, it was kind of a configuration hell.
ElasticSearch + Tire was a breezy to get it up and running, I got it working in an hour.
Lucene is just a Java search library, hence Solr was developed to be a full service search app, but Solr still have all the trapping of a typical Java webapp: overly complicated XML configurations, schema-heavy, expect XML docs for indexing, requires a Java servlet container (Jetty or Tomcat), which just become too many points of failure for me.
ElasticSearch is based on Lucene too, it has a built-in servlet container so just run like a daemon, use a very straight forward JSON + REST API so it's great for testing and a more natural fit for Ruby. It's schemaless and it worked for me without even editing a config file. Everything worked beautifully.
What I really needed was Chinese search and ElasticSearch already packaged Luecene's SmartChineseAnalyzer as a plugin. Not sure how difficult it will be to customize the analyzer/tokenizer chain if you need that level of customization. Docmentation for ElasticSearch and Tire are both top-notch.
Tire (Ruby library for ElasticSearch)
https://github.com/karmi/tire
You can try out the demo, it'll install a rails searchapp, download the ElasticSearch binary and run it, then start Webrick automatically.
$ rails new searchapp -m https://raw.github.com/karmi/tire/master/examples/rails-application-template.rb
On my system it complained about not having a Javascript engine (Rails 3.2? no longer include thereubyracer gem by default), so I had to:
$ wget https://raw.github.com/karmi/tire/master/examples/rails-application-template.rb
$ nano rails-application-template.rb
add gem 'therubyracer' in the file (look for gem 'tire' and gem 'will_paginate'), then...
$ rails new searchapp -m rails-application-template.rb
For developing my own app, I just downladed the ElasticSearch tarball and run in the foreground with the -f switch (so I can easily stop it by Ctrl-C)
$ bin/elasticsearch -f
You can install the eleasticsearch-head plugin to get a web admin interface
https://github.com/mobz/elasticsearch-head
Also something I found out: if you have one-to-many relationship models, Tire won't resolve them for you in the search results, it just returns a flat collection. Your has_many and belongs_to relationships will just be object ids in the collection rather than full objects.
I think you should search for a comparison between Solr and elastic search.
In fact sunspot is based on Solr, and both Solr and elastic search are based on Lucene. They are two different projects with similar goals, both built on top of Lucene.
Here are two comparisons:
ElasticSearch, Sphinx, Lucene, Solr, Xapian. Which fits for which usage?
http://www.findbestopensource.com/article-detail/solr-vs-elasticsearch
Here is the most complete up-to-date post on the topic: http://solr-vs-elasticsearch.com/
My recommendations as of May 2018
Here are some simple guidelines if the crazy long grid of features
above did not help.
Choose Solr if any of the following are true...
Your team consists mainly of Java programmers
You're already using ZooKeeper in your stack
You're already using Java in your stack
You are building a search application that has specific and nuanced relevancy requirements
You are building an ecommerce, job, or product search engine
Search is a central part of your product and user experience and there is the organizational mandate for search to be a core strength
Choose Elasticsearch if any of the following are true...
Your team consists mainly of Ruby/PHP/Python/full stack programmers (and your application does not have specific and nuanced relevancy
requirements)
You live and breathe JSON
You already use Kibana/ELK for managing your logs
Your application is analytics-heavy
If in doubt...
Every serious search application I have worked on has required
in-depth customization of the search workflow and relevancy tweaks,
and at the time of writing, this is simply not possible in
Elasticsearch without major hacking. If in doubt, go Solr.

Advice needed to full text searching on mongoid

I've a mongoid embedded one to many model on Rails 3.1, to full text search within. I neet something very light and simple to deploy on heroku too, without having to pay for add-ons, initially.
All heroku Full-Text Search add-on currently, seem to have just paying plans (which is no good to start with), see Flying Sphinx and Websolr.
I need advice on a good solution (a ruby gem deployable on heroku) to start with and than to scale to other cloud services eventually.
Maybe MongoDB's core functionalities are enough for your needs:
http://www.mongodb.org/display/DOCS/Full+Text+Search+in+Mongo
There are two more possibilities that come into my mind:
1) you can use this gem:
https://github.com/mauriciozaffari/mongoid_search
2) you can use elasticsearch (http://www.elasticsearch.org/) and use the following gem:
https://github.com/karmi/tire
However, you couldnt use this solution with heroku only, you would have to setup your own server, for which in the case you want to use heroku, EC2 would be recommendable
We were using the sunspot_mongo gem with solr on Mongoid 2.4.
But after upgrading to Mongoid 3, support for sunspot seems to not be there. So we're investigating a move to elasticsearch with the tire gem. There are some new offerings in the "search as a service space" for elasticsearch, but they don't seem quite production ready yet, so hoping that changes quickly.
Hope it helps!

Best search option for a heroku-hosted Rails app?

I've been working on a new project lately where a fantastic search engine is crucial. It's a rails3 app hosted on heroku and I'm looking into possible solutions(a rubygem would be ideal) which offer a easy way to have powerful full-text search.
Right now, I'm using acts_as_tsearch which leverages PostgreSQL and performs a basic MATCH query. Though, it's not really pulling back good results(for example, if I search for "create a project" and "how do i create a project" exists as a query, it doesn't find it).
Can anyone share their experiences with full text search, anyone tried out Solr ?
IndexTank is your best bet. They were recently added as a Heroku add-on.
We recently tried to just run our own search for our Heroku app and it's just not worth it because you have to worry about stability and scaling of that search box. It's better to go with a provider, like IndexTank.
IndexTank also powers Reddit and Wordpress.com, so can bet it'll be reliable.
SOLR works very nicely -- it's a bit pricey to get starts ($20 a month), but it just works, and works well.
They recently added the ability to ask the user "Did you mean to search for [correct spelling]".
You can easily cross-model search (search for Users and Cars and Dealerships).
Heroku offers addons which you can easily add to your application. You should take a look at Solr and IndexTank.
There's a free solution in the Texticle gem. It uses PostgreSQL's (> 8.3) full text index support and creates a search method on your models. If you create indexes, the speed is very good (for a free solution).
Hope that helps!

Are there other search options for heroku

I am about to launch a beta site, and heroku looks like a great option. The only think that is getting me down is that the only search option is $20/mth for the Websolr add-on.
I am sure that Websolr is great, but at this very early point in this project, I rather not light up that expense.
Are there any free search options to couple with heroku's Blossom (free) plan.
I feel like such a cheapskate!
This post seems to have good options:
Leveraging the full text search of postgrSQL:
http://tenderlovemaking.com/2009/10/17/full-text-search-on-heroku.html
Also explains the options of Ferret and Solr.
IndexTank has a heroku addon you can use for free.
It has some advantages over websolr, like realtimeness, fast (all in ram), and a very flexible scoring system that doesn't require to reindex (allows for very easy a/b testing).
My gem pg_search does full-text search against PostgreSQL, and works directly on Heroku.
Check it out and let me know if it works for you!
acts_as_tsearch works great. No configuration needed if you have postgresql > 8.3. Have to experiment with multiple tables though. Will use it on heroku till i can afford the WebSolr Add-on. I found it a better option compared to the texticle method as explained in the article link above (tendermaking).
acts_as_tsearch: http://github.com/pka/acts_as_tsearch
No, I was looking for that too a week ago, and didn't find anything...
And I don't think there is any work in progress on another add-ons like this as they already have one, so they won't put another that is free... :/
Anyway, heroku is amazing, so try to make it work with code or just spend $20 :)
acts_as_ferret won't work as Heroku cleans up the /tmp directory regularly. Even i am in need of a full-text solution. Thinking of trying out the acts_as_tsearch plugin.
Looks like IndexTank was purchased by LinkedIn and will be discontinuing support (although some portions might be open-sourced in the future). See this post for more info: https://indextank.com/documentation/faq2
If you're using Postgres for your Rails app then take a look at this free way to do full text search:
Part 1 and
Part 2
This uses the pg_search gem to allow you to use PostgreSQL's pg_search_scopes feature and have full text search without any other dependencies.

Resources