I have been stuck on this issue for the past 3 days and unsure where to look now.
I have a simple Sidekiq implementation into my rails app.
I am working on: Rails 4.2.0, Sidekiq 4.1.2, Redis 3.0.6
The production app is running live with heroku, and I have 1 worker dyno and 1 web dyno.
The issue is this, and I am unsure on how to approach it or what I did to make it do this.
When I run the redis-cli on heroku I can see the clients that I have running. At most I have 2 or 3 clients running at any given time. I can easily kill the clients with
CLIENT KILL TYPE normal
So that's all fine and dandy. The part when things get a little tricky is when I fire up my server locally, and I am working in development. All of a sudden my redic-cli shows that I have 19 clients running. This will result in me logging
Err max clients reached
My assumption is that somehow locally I am directing sidekiq to work off the redis production url. I have to admit what I know about Redis and Sidekiq is limited, but I do have a basic understanding of how it should be working.
Any help or guidance would be appreciated.
Try using sidekiq -c 3 to limit your concurrency.
This ended up being a configuration error. Just in case anyone stumbles upon this question hopefully this will help them not overlook something like I did.
This issue was happening only when I was firing up my local server, so I knew it had something to do with me locally. I noticed that on my production redis:cli I was seeing clients that had my local IP in the ADDR column.
This led me to believe that my local machine was pushing clients to my production Redis server. Looking at my logs when I fired up my Procfile I saw the Redis url there so that only confirmed it.
Finally after searching through my code, I discovered that I had actually added the url into my .env, so when I fired up my server it was using that production Redis url. So I changed it to the appropriate IP address for local development on my .env file redis://127.0.0.1:6379 and everything is now working as normal.
Related
I have a Rails 4.2 app running on Heroku. Occasionally there is an issue that causes most incoming requests to get a server error. For example, there could be a memory leak or a max database connection issue. How can I setup a script or service to automatically restart the server when it detects errors?
I think this service could ping the app every few minutes and if it detects an error, it should confirm there's really a problem and then run heroku restart. How could this be set up?
After Googling this topic, I came across Neptune.io, which seems to provide a useful service for this task.
We're running a rails app via heroku that connects to a windows azure VM, where I've set up a redis master/slave to act as a cache (slash quick reference data store). The problem is, we sporadically get redis timeouts. This is with a timeout of 10 seconds (which I know is more than it needs), and reestablishing redis connections on fork. And using hiredis as a driver.
Anyone have a clue why this might be happening? I know heroku and the azure vm are hosted on different coasts, so there's a bit of latency; could there be TCP request drops? I'm fairly out of ideas.
Trying to do something very basic and test my Rails app on my phone.
Starting the server with:
rails s
But when accessing mymachinename.local:3000, Mobile Safari says that the server stopped responding after about 30 seconds.
I do this all the time on other machines, but I have no idea what could be causing it here. Any suggestions?
I've had a similar issue before trying to access the server from a virtual machine, I had to use the following format for the rails server command:
rails s -b your.dev.ip.address
Whoops. Firewall. Don't know why I even had it turned on.
I'm trying to deploy my first Rails app here, and I've been stuck on something since last night. I'm encountering some weird behaviors I can't explain.
I'm running Rails, Apache, Phusion Passenger, and for the moment, SQLite 3. (I'll move that over to MySQL shortly.) Currently this is being hosted on a too-small EC2 slice running Ubuntu Server 11.04 (Natty).
When I visit the address of the EC2 slice in the browser, I get the default Rails 500 notice. Here's what's weird, though. When I tail /log/production.log, I see the following error:
ActionView::Template::Error (SQLite3::SQLException: no such table: offers: SELECT "offers".* FROM "offers" WHERE (code = '') ORDER BY created_at desc LIMIT 25 OFFSET 0):
So, I manually opened up the development database in SQLite3, and saw that table in there. The production database, however, does not have that table.
OK, so I'm getting errors with the production database logged in the production log. The application has to be running in production mode, right?
That is what's throwing me. First of all, it's running in development mode on my development machine, and I didn't change any of the files when I deployed it. Neither did I use any fancy deployment tools to deploy it (which may have switched something without my knowledge) - I just did a simple git push.
Furthermore, I added the following to my httdp.conf VirtualHost config:
RailsEnv development
Also, when I run rails console, I can get the following:
irb(main):002:0> Rails.env
=> "development"
So, the application really should be running in development mode, right? In fact, it seems to think (partially) that it is, right?
I'm really not sure what's happening here, and I'd really appreciate some expert advice.
Thanks everyone.
Edit - A few server reboots later, and now the thing just hangs when I try to view it in a browser. Also, Apache seems to hang when I try to restart it (hence the server reboots), related problem, or different problem altogether?
Well, this isn't a 100% satisfactory answer for me, but I did two things, and I think I got it working.
First, I re-installed the passenger Apache module. That may or may not have been necessary.
This was the big thing, though: after I had added the line to httpd.conf to pass the Rails Environment over to Passenger, I believe Apache restarted incorrectly. (Rather, I believe I've been restarting Apache incorrectly for my whole life!)
I was trying to restart Apache this way:
sudo /etc/init.d/apache2 restart
That has always worked for me (when programming PHP), but it simply wasn't working here. Apache would just stall on the restart.
This, however, works fine:
sudo apachectl restart
I'll have to ask Server Fault what the significant difference is between the two.
I hope that helps someone out.
I'm trying to make Resque work with my project, but unfortunately it seems that for some reasons Resque is not able to write on Redis.
Redis seems to be configured correctly, I'm able to connect with redis-cli and issue commands, runs on port 6379 as configured inside my Rails 3.0.5 app.
When I try to Resque enqueue something the job is queued, but it doesn't seem that something actually happens on Redis (0 clients connected inside my Redis logs).
When I restart the console, the queue is empty, with no workers running.
Everything fails silently, I have nothing in my rails logs, nothing on the console, nothing if I start a worker, it just (obviously) doesn't find any job to perform.
https://gist.github.com/867620
Any suggestions on how to fix or debug this ?
The problem was that I was including resque_spec in the bundle.
Obviously, resque_spec was stubbing Resque.enqueue, making my mistake very stupid and very difficult to spot.