In my current project, there is a Redis Instance on Heroku and Sidekiq is configured to process jobs from that instance. We need to migrate the Redis instance to Azure and I wanted to use a configuration where one Worker connects to the Redis Instance on Heroku to process any queued jobs and the rest of the Workers connecting to the new instance on Azure to avoid any data loss.
I am new to Rails and Sidekiq. Please suggest a way for achieving the desired config
The docs say that you can run this command to get the credentials:
heroku redis:credentials
Then get the connection string with:
heroku config:get REDIS_URL -a example-app
Then you can use the connection string from REDIS_URL to connect externally for a bit.
Related
We are getting a lot of sentries issue with PG:ConnectionBad on Postgres in RDS AWS and Ruby on Rails
PG::ConnectionBad Sidekiq/BookingExtensionCheckWorker
could not translate host name “ls-XXXXXXXXXXXXfee44.XXXXXXXXX.eu-west-1.rds.amazonaws.com” to address: Name or service not known
In two weeks ago, we have migrate a new database and we change endpoint in the RoR files api
new database endpoint,
ls-XXXXXXXXXXXXXXXf3d4a.XXXXXXXX.eu-west-1.rds.amazonaws.com
It working fine with no issue between the new database and Ruby on Rails api. however, I get a lot of sentries issue said that sidekiq having an issue with connection database as they use old database address which it’s no longer used. I have to check sidekiq database; the code shows it’s connected to the new database. They keep go back to the old database when I run Ruby on rails.
Is there some way to find why sidekiq keep connection old database
Sidekiq is background service. So when someone deployed he has used some sort of like. Right now you are getting issue you think is fine but actually most jobs are not running. Which you will noticed in few days.
How can you check jobs If you sidekiq setup. Probably following url will lead to all jobs.
your_url/sidekiq
It will show you all jobs, I think you have option here to restart services. Just click to restart sidekiq. And everything would be fine.
How to start
As your sidekiq running with old configuration. So following steps could be dangrous, I think you must check how you have started your process. Otherwise following are some way people configure it and runt it.
systemctl restart sidekiq
If this does not work, check your command your deployment guy has setup some sort of scripts inside /etc/init.d folder
Some time developer use following simple line to run sidekiq
bundle exec sidekiq -d -P tmp/sidekiq.pid -L log/sidekiq.log
or
bundle exec sidekiqctl stop
Application is Rails 4.1.4, Ruby 2.1.2.
Using sidekiq 3.2.6, redis 3.1.0, celluloid 0.15.2. The sidekiq implementation is as default as can be, with the exception of connecting to a remote redis queue (elastic cache).
When certain events are processed, we use sidekiq to queue up calls to an external API. The API is reachable through curl from the server our application is hosted on. All other functionality seems to still be performing as expected. This functionality has worked for weeks on the current server implementation/architecture.
After a successful deploy (with Capistrano, through Jenkins) to and EC2 instance, which is behind an elastic load balancer, and an auto-scaling group sidekiq will no longer connect(?) to elasticcache.
SocketError: getaddrinfo: Name or service not known
/gems/redis-3.1.0/lib/redis/connection/ruby.rb:152 in getaddrinfo
/gems/redis-3.1.0/lib/redis/connection/ruby.rb:152 in connect
/gems/redis-3.1.0/lib/redis/connection/ruby.rb:211 in connect
/gems/redis-3.1.0/lib/redis/client.rb:304 in establish_connection
/gems/redis-3.1.0/lib/redis/client.rb:85 in block in connect
/gems/redis-3.1.0/lib/redis/client.rb:266 in with_reconnect
/gems/redis-3.1.0/lib/redis/client.rb:84 in connect
/gems/redis-3.1.0/lib/redis/client.rb:326 in ensure_connected
/gems/redis-3.1.0/lib/redis/client.rb:197 in block in process
/gems/redis-3.1.0/lib/redis/client.rb:279 in logging
/gems/redis-3.1.0/lib/redis/client.rb:196 in process
/gems/redis-3.1.0/lib/redis/client.rb:102 in call
/gems/redis-3.1.0/lib/redis.rb:1315 in block in smembers
/gems/redis-3.1.0/lib/redis.rb:37 in block in synchronize
/usr/local/rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211 in mon_synchronize
/gems/redis-3.1.0/lib/redis.rb:37 in synchronize
/gems/redis-3.1.0/lib/redis.rb:1314 in smembers
/gems/sidekiq-3.2.6/lib/sidekiq/api.rb:557 in block in cleanup
/gems/connection_pool-2.0.0/lib/connection_pool.rb:58 in with
/gems/sidekiq-3.2.6/lib/sidekiq.rb:72 in redis
/gems/sidekiq-3.2.6/lib/sidekiq/api.rb:556 in cleanup
/gems/sidekiq-3.2.6/lib/sidekiq/api.rb:549 in initialize
/gems/sidekiq-3.2.6/lib/sidekiq/scheduled.rb:79 in new
/gems/sidekiq-3.2.6/lib/sidekiq/scheduled.rb:79 in poll_interval
/gems/sidekiq-3.2.6/lib/sidekiq/scheduled.rb:58 in block in poll
/gems/sidekiq-3.2.6/lib/sidekiq/util.rb:15 in watchdog
/gems/sidekiq-3.2.6/lib/sidekiq/scheduled.rb:23 in poll
/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25 in public_send
/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25 in dispatch
/gems/celluloid-0.15.2/lib/celluloid/calls.rb:122 in dispatch
/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322 in block in handle_message
/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416 in block in task
/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55 in block in initialize
/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13 in block in create
We have restarted sidekiq, restarted elastic cache, restarted the server, inspected the redis queue with redis-cli and seen nothing noteworthy.
As implied, we can connect to elastic cache using redis-cli, however, using sidekiq/apifrom the console, we get the same SocketError
Any ideas on how to remedy? The application is neigh unusable at this point.
Thanks!
Yay for embarrassing errors! There was a typo in the ENV var url. 10 hours later, between me and the devops, and it was a copy and paste issue.
Thanks
Problem: need to send e-mails from Rails asynchronously.
Environment: Windows 7, Ruby 2.0, Rails 4.1, Sidekiq, Redis
After setting everything up, starting Sidekiq and starting Redis, I can see the mail request queued to Redis through the monitor:
1414256204.699674 "exec"
1414256204.710675 "multi"
1414256204.710675 "sadd" "queues" "default"
1414256204.710675 "lpush" "queue:default" "{\"retry\":true,\"queue\":\"default\",\"class\":\"Sidekiq::Extensions::DelayedMailer\",\"args\":[\"---\\n- !ruby/class 'UserMailer'\\n- :async_reminder\\n- - 673\\n\"],\"jid\":\"d4024c0c219201e5d1649c54\",\"enqueued_at\":1414256204.709674}"
But the mailer method never seems to get executed. The mail doesn't get sent and none of the log messages show up.
How does Redis know to execute the job on the queue and does something else need to be setup in the environment for it to know where the application resides?
Is delayed_job a better solution?
I started redis in one window, bundle exec sidekiq in another window, and rails server in a third window.
How does an item on the redis queue get picked up and processed? Is sidekiq both putting things on the redis queue and checking to see if something was added that needs to be processed?
Redis is used just for storage. It stores jobs to be done. It does not execute anything. DelayedJob uses your database for job storage instead of Redis.
Rails process pushes new jobs to Redis.
Sidekiq process pops jobs from Redis and executes them.
In your MONITOR output, you should see LPUSH commands when Rails sends mail. You should also see BRPOP commands from Sidekiq.
You need to make sure that both Rails and Sidekiq processes use the same Redis server, database number, and namespace (if any). It's a frequent problem that they don't.
I have a rails app on AWS Elastic Beanstalk. I process background tasks using delayed_job. I set up an eb worker instance to handle this. It works but shows as failed (red) in the dashboard. I believe this is because of the following error that I get every few seconds:
error: AWS::SQS::Errors::AccessDenied: Access to the resource https://sqs.us-west-2.amazonaws.com/xxx...xxxx is denied
I tried to remove sqs by means of the following to no avail:
services:
sysvinit:
aws-sqsd:
enabled: false
ensureRunning: false
How do I stop sqs? Ideally it would never be installed in the first place. If I can't modify the install configuration is there a way to prevent this error from affecting the status of my environment?
You are launching a worker tier environment. That is why SQS is being created for your environment. You should launch a "Web Server" environment if you do not want to launch a worker environment. Worker Tier environment in Elastic Beanstalk allows you to poll messages from an SQS queue periodically.
Read more about worker tier environments here:
http://aws.amazon.com/blogs/aws/background-task-handling-for-aws-elastic-beanstalk/
Read more about environment tiers here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features-managing-env-tiers.html
If you want to use a Worker Tier environment with SQS enabled you can get rid of the AccessDenied exception by giving access to the IAM instance profile to access SQS as explained here:
https://stackoverflow.com/a/24880344/161628
I have one Rails instance and another Node.js instance. The plan is to publish information from the Rails project using Redis's pubsub and subscribe by the Node.js program. The node.js program will publish the information to other clients through web sockets (or long polls).
The whole thing is working perfectly while deployed in combination of remote linux machines. On Heroku, Redis To Go is added as an add-on with the Rails instance. Redis can be connected and working fine from this instance.
But I am not getting how to reach to this Redis add-on from the Node.js instance. I've added REDISTOGO_URL from the environment of Rails instance to the environment of Node.js, but it's not working. Sample errors I'm getting (from logs and REPL) are
redis.set("a","b");
==> Error: Connection in pub/sub mode, only pub/sub commands may be used
redis.psubscribe('*');
==> false
Can somebody please show me a way how to accomplish this?
Yes, all you have to do is set the heroku config value similar to:
'heroku config:add REDISTOGO_URL=[your redis togo url]'
Heroku will then add your config var and restart your app.
You can check to make sure it's added correctly by doing a 'heroku config' on both repositories and comparing them.