sunspot_rails unable to index due to "404 Not Found" error - ruby-on-rails

Question

I'm trying to install Sunspot in a small Rails app, exactly following the gem setup instructions, but every time I run into RSolr::Error::Http: RSolr::Error::Http - 404 Not Found errors when I try to index data. I can reproduce this with a fresh app; here are the exact steps I follow:
Create a fresh Rails 4.2.5 app:
$ rails new test_sunspot
$ cd test_sunspot/
$ spring stop # spring can cause `generate` commands to hang
$ rails g model Thing title:string
$ rake db:migrate
$ rails c
> Thing.create!(title: "Cats")
> Thing.create!(title: "Pizza")
> exit
Add a Sunspot index to the model:
class Thing < ActiveRecord::Base
searchable do
text :title
end
end
Add Sunspot to Gemfile:
...
gem 'sunspot_rails', '2.2.2'
gem 'sunspot_solr', '2.2.2'
...
Install, start, and reindex Sunspot:
$ bundle install
$ rails g sunspot_rails:install # default sunspot.yml is not changed
$ ps aux | grep solr # confirm that no Solr services are running
$ bundle exec rake sunspot:solr:start # generates solr/ dir; no errors
$ bundle exec rake sunspot:solr:reindex
This reindex command yields the following output. When I go into the Rails console and tried to create a new Thing object, it triggers the same error (because Sunspot attempts to update the index):
Skipping progress bar: for progress reporting, add gem 'progress_bar' to your Gemfile
rake aborted!
RSolr::Error::Http: RSolr::Error::Http - 404 Not Found
Error: Not Found
URI: http://localhost:8982/solr/development/update?wt=ruby
Request Headers: {"Content-Type"=>"text/xml"}
Request Data: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><delete><query>type:Thing</query></delete>"
Backtrace: /Users/topher/.rvm/gems/ruby-2.2.0/gems/rsolr-1.0.13/lib/rsolr/client.rb:284:in `adapt_response'
/Users/topher/.rvm/gems/ruby-2.2.0/gems/rsolr-1.0.13/lib/rsolr/client.rb:190:in `execute'
/Users/topher/.rvm/gems/ruby-2.2.0/gems/rsolr-1.0.13/lib/rsolr/client.rb:176:in `send_and_receive'
# ...lots of backtrace omitted...
/Users/topher/.rvm/gems/ruby-2.2.0/gems/sunspot_rails-2.2.2/lib/sunspot/rails/tasks.rb:19:in `block (2 levels) in <top (required)>'
/Users/topher/.rvm/gems/ruby-2.2.0/bin/ruby_executable_hooks:15:in `eval'
/Users/topher/.rvm/gems/ruby-2.2.0/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => sunspot:solr:reindex => sunspot:reindex
(See full trace by running task with --trace)
The output of ps aux | grep solr (after starting Solr): Note that the PID mentioned in solr/pids/development/sunspot-solr-development.pid is 62449, which matches the third line:
topher 62617 0.0 0.0 2432772 520 s002 R+ 3:00PM 0:00.00 grep solr
topher 62484 0.0 1.3 3274624 105756 ?? S 2:57PM 0:03.65 /usr/bin/java -server -Xss256k -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/Users/topher/.rvm/gems/ruby-2.2.0/gems/sunspot_solr-2.2.2/solr/server/logs/solr_gc.log -DSTOP.PORT=7982 -DSTOP.KEY=solrrocks -Djetty.port=8982 -Dsolr.solr.home=/Users/topher/Sites/john_kole/test_sunspot/solr -Dsolr.install.dir=/Users/topher/.rvm/gems/ruby-2.2.0/gems/sunspot_solr-2.2.2/solr -Duser.timezone=UTC -Djava.net.preferIPv4Stack=true -jar start.jar
topher 62449 0.0 0.0 2444632 1304 ?? Ss 2:57PM 0:00.04 bash ./solr start -f -p 8982 -s /Users/topher/Sites/john_kole/test_sunspot/solr
Other details:
I'm on Mac OSX Yosemite
I've uninstalled and reinstalled the relevant gems, tried downgrading to Sunspot 1.3.0, and even ran gem pristine --all, with no change in outcome
EDIT: I have read through the other similar tickets on Sunspot/Solr 404 Not Found errors. It looks like the solutions in those cases amounted to "resetting" the Solr config and aren't relevant to a fresh (dev environment) project: no prior Solr instances are running before I run rake sunspot:solr:start; this is a completely fresh Rails project so the solr/ directory was just newly generated anyway; this is in development, not production; and adding solr_home: solr to sunspot.yml or updating path: /solr/default have no effect on the 404 outcome.
Questions:
Any idea why this is happening?
Could this breakage be due to something mis-installed in my OSX environment? What should I look at?
Can you get Sunspot working properly on a fresh install of current Rails, following these same steps?
Thanks in advance!

Answer 1

Remove the sunspot_solr gem and the install solr on your machine by the following commands
brew install solr
if it doesn't work run brew update first to fetch the latest solr links
brew will install all missing dependencies automatically.
To start solr:
solr start

Answer 2

TL;DR: You have done everything right, but the user installing the gems is not the same as the one running solr (aka permissions issue)
I tried to reproduce your case and succeeded, which felt very wrong. The issue is that installing the gem sunspot_solr ships an embedded Java webserver Jetty. I managed to trace the cause of the problem to the fact that this webserver is not run as the person who installed it.
This can be verified by installing gems per:
$ touch Gemfile # to mark it as edited
$ bundle install --path vendor/bundle # to install as user who runs the server later
instead of the usual
$ gem install ... bundle install
as another (possibly root) user.
Now you can succeed performing all your steps as mentioned in your post.