So I'm trying to broadcast a message in one of the jobs. the job is being executed but it's not broadcasting the message.
Is it because actioncable requires Puma to perform message broadcasting?
So I guess the reason why it wasn't working was because the cable adapter wasn't set to redis in config/cable.yml
Related
I have a microservice that is taking in webhooks to process but it is currently getting pounded by the sender of said webhooks. Right now I am taking them and inserting the webhooks into the db for processing but the data is so bursty at times that I don't have enough bandwidth to manage the flood of requests and I cannot scale anymore as I'm out of db connections. The current thought is to just take the webhooks and throw them into a Kafka queue for processing; using Kafka I can scale up the number of frontend workers to whatever I need to handle the deluge of requests and I have the replayability of Kafka. By throwing the webhooks into Kafka, the frontend web server no longer needs a pool of db connections as it literally is just taking the request and throwing into the queue for processing. Does anyone have any knowledge on removing the db connectivity from Puma or have an alternative to do what's being asked?
Currently running
ruby 2.6.3
rails 6.0.1
puma 3.11
Ended up using Puma's before fork and on_worker_boot methods to not re-establish the database connection for those particular web workers within the config
I have a rails app which talks to a socket.io app via Redis.
I can create a message on the rails app and socket.io broadcasts it, but I have no idea how to handle the incoming messages (i.e. I want a service to always be listening to Redis and process the incoming messages).
Can you please tell me how I can achieve this?
you can use sidekiq gem here which is by default using redis so whenever you enqueue a job to redis via perform_later method provided by sidekiq. Sidekiq worker will run it whenever a worker is free.
For Pub/sub as you mentioned in the header you can go for Action Cable feature provided by Rails.
Problem: need to send e-mails from Rails asynchronously.
Environment: Windows 7, Ruby 2.0, Rails 4.1, Sidekiq, Redis
After setting everything up, starting Sidekiq and starting Redis, I can see the mail request queued to Redis through the monitor:
1414256204.699674 "exec"
1414256204.710675 "multi"
1414256204.710675 "sadd" "queues" "default"
1414256204.710675 "lpush" "queue:default" "{\"retry\":true,\"queue\":\"default\",\"class\":\"Sidekiq::Extensions::DelayedMailer\",\"args\":[\"---\\n- !ruby/class 'UserMailer'\\n- :async_reminder\\n- - 673\\n\"],\"jid\":\"d4024c0c219201e5d1649c54\",\"enqueued_at\":1414256204.709674}"
But the mailer method never seems to get executed. The mail doesn't get sent and none of the log messages show up.
How does Redis know to execute the job on the queue and does something else need to be setup in the environment for it to know where the application resides?
Is delayed_job a better solution?
I started redis in one window, bundle exec sidekiq in another window, and rails server in a third window.
How does an item on the redis queue get picked up and processed? Is sidekiq both putting things on the redis queue and checking to see if something was added that needs to be processed?
Redis is used just for storage. It stores jobs to be done. It does not execute anything. DelayedJob uses your database for job storage instead of Redis.
Rails process pushes new jobs to Redis.
Sidekiq process pops jobs from Redis and executes them.
In your MONITOR output, you should see LPUSH commands when Rails sends mail. You should also see BRPOP commands from Sidekiq.
You need to make sure that both Rails and Sidekiq processes use the same Redis server, database number, and namespace (if any). It's a frequent problem that they don't.
So, i have a big rails application that is using delayed_job to send emails and SMS to the users.
Once in a while, the delayed_job process will simply stop working without any message on the logs. I have finally pinpointed the problem as the delayed_job process crashs when it coulnd't connect to the database.
Is there any configuration i can make so it will retry the connection instead of just crashing? I've tried setting the reconnect: true on the database.yml file with no success.
Another option that i'm looking for is maybe using a monitoring tool like god or bluepill.
Try using the -m flag when you start delayed job - that should start a separate monitor process that in my experience has been very good about restarting the process.
I've been struggling with this error for quite some time:
Redis::ProtocolError: Got 'i' as initial reply byte.
If you're running in a multi-threaded environment, make sure you pass the :thread_safe
option when initializing the connection. If you're in a forking environment, such as
Unicorn, you need to connect to Redis after forking.
It happens intermittently in apps that use Unicorn and Redis. From this redis-rb Github issue it looks the :thread_safe option is now enabled by default. I am using redis 2.2.2 because redis 3.0.1 is not compatible with the latest version of resque.
In my Unicorn config I'm using Redis.current.quit after the fork.
I'm also connecting to Redis with a gem called ruote-redis which is a storage implementation for the workflow engine Ruote.
How can I ensure that all of my Redis connections are stable and that I don't get this error anymore which is disrupting to the normal use of our app?
Unicorn is not multi-threaded. Are you using threads yourself?
As stated in the docs, the problem you're hitting is that multiple Unicorn workers are sharing the same connection (ie the same underlying file descriptor).
This change, included in version redis-rb 3.0, makes it even clearer.
If you're still hitting this error please post your Unicorn configuration.