Now I am using Ruby on Rails framework. We use delayed_job to run a job to operate aws s3 file by aws-sdk. We deal with many files on aws s3.
When we run the job, we got this error:
e.message => Access Denied
e.backtrace.join("\n") =>
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/aws-sdk-core/plugins/s3_sse_cpk.rb:19:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/aws-sdk-core/plugins/s3_accelerate.rb:33:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/aws-sdk-core/plugins/param_converter.rb:20:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/aws-sdk-core/plugins/response_paging.rb:26:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/seahorse/client/plugins/response_target.rb:21:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/seahorse/client/request.rb:70:in `send_request'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-core-2.5.2/lib/seahorse/client/base.rb:207:in `block (2 levels) in define_operation_methods'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.2/lib/aws-sdk-resources/request.rb:24:in `call'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.2/lib/aws-sdk-resources/operations.rb:139:in `all_batches'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.2/lib/aws-sdk-resources/collection.rb:18:in `each'
/app/shared/bundle/ruby/2.3.0/gems/aws-sdk-resources-2.5.2/lib/aws-sdk-resources/collection.rb:18:in `each'
/app/releases/20161107014751/app/jobs/my_job.rb:35:in `perform'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/execution.rb:34:in `block in perform_now'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:126:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:506:in `block (2 levels) in compile'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:455:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:448:in `block (2 levels) in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:286:in `block (2 levels) in halting'
/app/shared/bundle/ruby/2.3.0/gems/i18n-0.7.0/lib/i18n.rb:257:in `with_locale'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/translation.rb:7:in `block (2 levels) in <module:Translation>'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `instance_exec'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `block in make_lambda'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:285:in `block in halting'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:447:in `block in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:455:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:448:in `block (2 levels) in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:286:in `block (2 levels) in halting'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/logging.rb:24:in `block (4 levels) in <module:Logging>'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/notifications.rb:164:in `block in instrument'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/notifications.rb:164:in `instrument'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/logging.rb:23:in `block (3 levels) in <module:Logging>'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/logging.rb:44:in `block in tag_logger'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/tagged_logging.rb:70:in `block in tagged'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/tagged_logging.rb:26:in `tagged'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/tagged_logging.rb:70:in `tagged'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/logging.rb:44:in `tag_logger'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/logging.rb:20:in `block (2 levels) in <module:Logging>'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `instance_exec'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `block in make_lambda'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:285:in `block in halting'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:447:in `block in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:455:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:101:in `__run_callbacks__'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:750:in `_run_perform_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:90:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/execution.rb:33:in `perform_now'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/execution.rb:22:in `block in execute'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:126:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:506:in `block (2 levels) in compile'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:455:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:448:in `block (2 levels) in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:286:in `block (2 levels) in halting'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/railtie.rb:26:in `block (4 levels) in <class:Railtie>'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/execution_wrapper.rb:76:in `wrap'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/reloader.rb:68:in `block in wrap'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/execution_wrapper.rb:76:in `wrap'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/reloader.rb:67:in `wrap'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/railtie.rb:25:in `block (3 levels) in <class:Railtie>'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `instance_exec'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:391:in `block in make_lambda'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:285:in `block in halting'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:447:in `block in around'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:455:in `call'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:101:in `__run_callbacks__'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:750:in `_run_execute_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/activesupport-5.0.0/lib/active_support/callbacks.rb:90:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/execution.rb:20:in `execute'
/app/shared/bundle/ruby/2.3.0/gems/activejob-5.0.0/lib/active_job/queue_adapters/delayed_job_adapter.rb:36:in `perform'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:84:in `block in invoke_job'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:61:in `block in initialize'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:66:in `execute'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:40:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:81:in `invoke_job'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:230:in `block (2 levels) in run'
/usr/lib64/ruby/2.3.0/timeout.rb:91:in `block in timeout'
/usr/lib64/ruby/2.3.0/timeout.rb:101:in `timeout'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:230:in `block in run'
/usr/lib64/ruby/2.3.0/benchmark.rb:308:in `realtime'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:229:in `run'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:306:in `block in reserve_and_run_one_job'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:61:in `block in initialize'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:66:in `execute'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:40:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:306:in `reserve_and_run_one_job'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:213:in `block in work_off'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:212:in `times'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:212:in `work_off'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:175:in `block (4 levels) in start'
/usr/lib64/ruby/2.3.0/benchmark.rb:308:in `realtime'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:174:in `block (3 levels) in start'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:61:in `block in initialize'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:66:in `execute'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:40:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:173:in `block (2 levels) in start'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:172:in `loop'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:172:in `block in start'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:79:in `block (2 levels) in add'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:61:in `block in initialize'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:79:in `block in add'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:66:in `execute'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/lifecycle.rb:40:in `run_callbacks'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/worker.rb:171:in `start'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:132:in `run'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:120:in `block in run_process'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/application.rb:265:in `block in start_proc'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/daemonize.rb:84:in `call_as_daemon'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/application.rb:269:in `start_proc'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/application.rb:295:in `start'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/controller.rb:56:in `run'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons.rb:193:in `block in run_proc'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons/cmdline.rb:88:in `catch_exceptions'
/app/shared/bundle/ruby/2.3.0/gems/daemons-1.2.3/lib/daemons.rb:192:in `run_proc'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:118:in `run_process'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:99:in `block in daemonize'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:97:in `times'
/app/shared/bundle/ruby/2.3.0/gems/delayed_job-4.1.2/lib/delayed/command.rb:97:in `daemonize'
/app/current/bin/delayed_job:5:in `<main>'
I don't know whether it is the delayed_job problem or the aws-sdk problem, or s3 network problem.
For sure its related to AWS not with delayed job , you need to check your s3 creds or contact to them for more details
Reference
http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html
Related
I've managed to consistently reproduce a deadlock situation (I believe) despite following the following resources:
https://guides.rubyonrails.org/threading_and_code_execution.html#executor
https://github.com/rails/rails/issues/27025#issuecomment-260169414
The following code will deadlock:
Parallel
.map(domains, in_threads: 4) do |domain|
Rails
.application
.executor
.wrap { EcommEnrichmentUtils.qualified_domain(domain) }
end
.compact
end
The contents of EcommEnrichmentUtils essentially sends out some HTTP requests.
Output for /rails/locks
Thread 0 [0x1a518 sleep] No lock (yielded share)
Waiting in start_exclusive to "load"
may be pre-empted for: "load"
blocked by: 1, 2
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:108:in `sleep'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:108:in `wait'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:108:in `wait_for_cond'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:108:in `wait'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:116:in `wait_while'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:220:in `wait_for'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:83:in `block (2 levels) in start_exclusive'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:82:in `block in start_exclusive'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:202:in `synchronize'
/Users/amirsharif/.rvm/rubies/ruby-2.7.4/lib/ruby/2.7.0/monitor.rb:202:in `mon_synchronize'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:77:in `start_exclusive'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:149:in `exclusive'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:13:in `loading'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:39:in `load_interlock'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:402:in `require_or_load'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:558:in `load_missing_constant'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies.rb:213:in `const_missing'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:90:in `block (2 levels) in validated_url_without_subdomain'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:89:in `each'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:89:in `detect'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:89:in `block in validated_url_without_subdomain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:102:in `block (2 levels) in get'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:189:in `calc'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:102:in `block in get'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:204:in `block in synchronized'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronize'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronized'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:102:in `get'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:68:in `validated_url_without_subdomain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `validate_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/_methods.rb:270:in `block in _on_method_added'
/Users/amirsharif/Projects/sendapostcardonline/app/typed_service_objects/ecomm_enrichment_utils.rb:22:in `qualified_domain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `validate_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/_methods.rb:270:in `block in _on_method_added'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (3 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (2 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:507:in `call_with_index'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:367:in `block (2 levels) in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:518:in `with_instrumentation'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:366:in `block in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:215:in `block (4 levels) in in_threads'
---
Thread 1 [0x1a52c sleep] Sharing
blocking: 0
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronize'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronized'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:102:in `get'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:68:in `validated_url_without_subdomain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation_2_7.rb:703:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation_2_7.rb:703:in `block in create_validator_method_medium1'
/Users/amirsharif/Projects/sendapostcardonline/app/typed_service_objects/ecomm_enrichment_utils.rb:22:in `qualified_domain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `validate_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/_methods.rb:270:in `block in _on_method_added'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (3 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (2 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:507:in `call_with_index'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:367:in `block (2 levels) in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:518:in `with_instrumentation'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:366:in `block in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:215:in `block (4 levels) in in_threads'
---
Thread 2 [0x1a540 sleep] Sharing
blocking: 0
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronize'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:199:in `synchronized'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/zache-0.12.0/lib/zache.rb:102:in `get'
/Users/amirsharif/Projects/sendapostcardonline/app/services/domain_helper.rb:68:in `validated_url_without_subdomain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation_2_7.rb:703:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation_2_7.rb:703:in `block in create_validator_method_medium1'
/Users/amirsharif/Projects/sendapostcardonline/app/typed_service_objects/ecomm_enrichment_utils.rb:22:in `qualified_domain'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `validate_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/_methods.rb:270:in `block in _on_method_added'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (3 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:374:in `block (2 levels) in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:507:in `call_with_index'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:367:in `block (2 levels) in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:518:in `with_instrumentation'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:366:in `block in work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:215:in `block (4 levels) in in_threads'
---
Thread 3 [0x15f7c sleep] No lock (yielded share)
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:217:in `value'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:217:in `map'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:217:in `block (2 levels) in in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:213:in `handle_interrupt'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:213:in `block in in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:211:in `handle_interrupt'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:211:in `in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:360:in `work_in_threads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/parallel-1.20.1/lib/parallel.rb:281:in `map'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:370:in `block in normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:368:in `normalize_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `bind_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/call_validation.rb:161:in `validate_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/sorbet-runtime-0.5.9465/lib/types/private/methods/_methods.rb:270:in `block in _on_method_added'
/Users/amirsharif/Projects/sendapostcardonline/app/controllers/ecomm_analyze/collections_controller.rb:177:in `bulk_add_domains'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/abstract_controller/base.rb:228:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/rendering.rb:30:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:117:in `block in run_callbacks'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actiontext-6.1.4.1/lib/action_text/rendering.rb:20:in `with_renderer'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actiontext-6.1.4.1/lib/action_text/engine.rb:59:in `block (4 levels) in <class:Engine>'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:126:in `instance_exec'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:137:in `run_callbacks'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/abstract_controller/callbacks.rb:41:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/rescue.rb:22:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:203:in `block in instrument'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/notifications.rb:203:in `instrument'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/instrumentation.rb:33:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activerecord-6.1.4.1/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/abstract_controller/base.rb:165:in `process'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionview-6.1.4.1/lib/action_view/rendering.rb:39:in `process'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal.rb:190:in `dispatch'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_controller/metal.rb:254:in `dispatch'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/route_set.rb:33:in `serve'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `each'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/journey/router.rb:32:in `serve'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/routing/route_set.rb:842:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/meta_request-0.7.3/lib/meta_request/middlewares/app_request_handler.rb:15:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/meta_request-0.7.3/lib/meta_request/middlewares/meta_request_handler.rb:15:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/flipper-0.22.1/lib/flipper/middleware/memoizer.rb:77:in `memoized_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/flipper-0.22.1/lib/flipper/middleware/memoizer.rb:42:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/warden-1.2.9/lib/warden/manager.rb:36:in `block in call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/warden-1.2.9/lib/warden/manager.rb:34:in `catch'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/warden-1.2.9/lib/warden/manager.rb:34:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/conditional_get.rb:40:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/http/permissions_policy.rb:22:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/cookies.rb:689:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activerecord-6.1.4.1/lib/active_record/migration.rb:601:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/callbacks.rb:98:in `run_callbacks'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rollbar-3.2.0/lib/rollbar/middleware/rails/rollbar.rb:25:in `block in call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rollbar-3.2.0/lib/rollbar.rb:145:in `scoped'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rollbar-3.2.0/lib/rollbar/middleware/rails/rollbar.rb:22:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rollbar-3.2.0/lib/rollbar/middleware/rails/show_exceptions.rb:22:in `call_with_rollbar'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/web-console-4.1.0/lib/web_console/middleware.rb:132:in `call_app'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/web-console-4.1.0/lib/web_console/middleware.rb:28:in `block in call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/web-console-4.1.0/lib/web_console/middleware.rb:17:in `catch'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/web-console-4.1.0/lib/web_console/middleware.rb:17:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-contrib-2.3.0/lib/rack/contrib/response_headers.rb:19:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/meta_request-0.7.3/lib/meta_request/middlewares/headers.rb:18:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:37:in `call_app'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `block in call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `block in tagged'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:37:in `tagged'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/tagged_logging.rb:99:in `tagged'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/railties-6.1.4.1/lib/rails/rack/logger.rb:26:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/request_store-1.5.0/lib/request_store/middleware.rb:19:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/request_id.rb:26:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/activesupport-6.1.4.1/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-livereload-0.3.17/lib/rack/livereload.rb:23:in `_call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-livereload-0.3.17/lib/rack/livereload.rb:14:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/static.rb:24:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/static.rb:24:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/debug_locks.rb:41:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/actionpack-6.1.4.1/lib/action_dispatch/middleware/host_authorization.rb:98:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-mini-profiler-2.3.3/lib/mini_profiler/profiler.rb:393:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/webpacker-5.4.3/lib/webpacker/dev_server_proxy.rb:25:in `perform_request'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/rack-proxy-0.7.0/lib/rack/proxy.rb:63:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/railties-6.1.4.1/lib/rails/engine.rb:539:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/configuration.rb:249:in `call'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/request.rb:77:in `block in handle_request'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/thread_pool.rb:340:in `with_force_shutdown'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/request.rb:76:in `handle_request'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/server.rb:447:in `process_client'
/Users/amirsharif/.rvm/gems/ruby-2.7.4/gems/puma-5.5.2/lib/puma/thread_pool.rb:147:in `block in spawn_thread'
Unfortunately I don't know how to parse this or what a reasonable next step is.
Can anyone shed some light on this?
Okay, I figured it out.
It seems that this is an issue with the Zache gem.
See this line:
https://github.com/yegor256/zache/blob/master/lib/zache.rb#L206
This I believe introduces an external sleep.
I was able to fix my code by wrapping the method BEFORE called zache.get
Something like this:
ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
Rails
.application
.executor
.wrap do
zache.get(url, lifetime: 10.minutes) do
# do work here
end
end
end
Getting this error even though email_verification does not exist in our codebase. Here's the stack trace:
NoMethodError: undefined method `email_verification' for UserMailer:Class
from action_mailer/base.rb:582:in `method_missing'
from action_mailer/delivery_job.rb:16:in `public_send'
from action_mailer/delivery_job.rb:16:in `perform'
from active_job/execution.rb:39:in `block in perform_now'
from active_support/callbacks.rb:109:in `block in run_callbacks'
from raven/integrations/rails/active_job.rb:18:in `capture_and_reraise_with_sentry'
from raven/integrations/rails/active_job.rb:12:in `block (2 levels) in included'
from active_support/callbacks.rb:118:in `instance_exec'
from active_support/callbacks.rb:118:in `block in run_callbacks'
from new_relic/agent/instrumentation/active_job.rb:53:in `perform'
from new_relic/agent/instrumentation/active_job.rb:23:in `block (4 levels) in <main>'
from active_support/callbacks.rb:118:in `instance_exec'
from active_support/callbacks.rb:118:in `block in run_callbacks'
from i18n.rb:313:in `with_locale'
from active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
from active_support/callbacks.rb:118:in `instance_exec'
from active_support/callbacks.rb:118:in `block in run_callbacks'
from active_job/logging.rb:26:in `block (4 levels) in <module:Logging>'
from active_support/notifications.rb:168:in `block in instrument'
from active_support/notifications/instrumenter.rb:23:in `instrument'
from active_support/notifications.rb:168:in `instrument'
from active_job/logging.rb:25:in `block (3 levels) in <module:Logging>'
from active_job/logging.rb:46:in `block in tag_logger'
from active_support/tagged_logging.rb:71:in `block in tagged'
from active_support/tagged_logging.rb:28:in `tagged'
from active_support/tagged_logging.rb:71:in `tagged'
from active_job/logging.rb:46:in `tag_logger'
from active_job/logging.rb:22:in `block (2 levels) in <module:Logging>'
from active_support/callbacks.rb:118:in `instance_exec'
from active_support/callbacks.rb:118:in `block in run_callbacks'
from active_support/callbacks.rb:136:in `run_callbacks'
from active_job/execution.rb:38:in `perform_now'
from active_job/execution.rb:24:in `block in execute'
from active_support/callbacks.rb:109:in `block in run_callbacks'
from active_job/railtie.rb:28:in `block (4 levels) in <class:Railtie>'
from active_support/execution_wrapper.rb:87:in `wrap'
from active_support/reloader.rb:73:in `block in wrap'
from active_support/execution_wrapper.rb:83:in `wrap'
from active_support/reloader.rb:72:in `wrap'
from active_job/railtie.rb:27:in `block (3 levels) in <class:Railtie>'
from active_support/callbacks.rb:118:in `instance_exec'
from active_support/callbacks.rb:118:in `block in run_callbacks'
from active_support/callbacks.rb:136:in `run_callbacks'
from active_job/execution.rb:22:in `execute'
from active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
from sidekiq/processor.rb:192:in `execute_job'
from sidekiq/processor.rb:165:in `block (2 levels) in process'
from sidekiq/middleware/chain.rb:128:in `block in invoke'
from new_relic/agent/instrumentation/sidekiq.rb:33:in `block in call'
from new_relic/agent/instrumentation/controller_instrumentation.rb:376:in `perform_action_with_newrelic_trace'
from new_relic/agent/instrumentation/sidekiq.rb:29:in `call'
from sidekiq/middleware/chain.rb:130:in `block in invoke'
from raven/integrations/sidekiq.rb:9:in `call'
from sidekiq/middleware/chain.rb:130:in `block in invoke'
from sidekiq/middleware/chain.rb:133:in `invoke'
from sidekiq/processor.rb:164:in `block in process'
from sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
from sidekiq/job_retry.rb:109:in `local'
from sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
from sidekiq/rails.rb:43:in `block in call'
from active_support/execution_wrapper.rb:87:in `wrap'
from active_support/reloader.rb:73:in `block in wrap'
from active_support/execution_wrapper.rb:87:in `wrap'
from active_support/reloader.rb:72:in `wrap'
from sidekiq/rails.rb:42:in `call'
from sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
from sidekiq/processor.rb:250:in `stats'
from sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
from sidekiq/job_logger.rb:8:in `call'
from sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
from sidekiq/job_retry.rb:74:in `global'
from sidekiq/processor.rb:125:in `block in dispatch'
from sidekiq/logging.rb:48:in `with_context'
from sidekiq/logging.rb:42:in `with_job_hash_context'
from sidekiq/processor.rb:124:in `dispatch'
from sidekiq/processor.rb:163:in `process'
from sidekiq/processor.rb:83:in `process_one'
from sidekiq/processor.rb:71:in `run'
from sidekiq/util.rb:16:in `watchdog'
from sidekiq/util.rb:25:in `block in safe_thread'
Unresolved Rails issue: https://github.com/rails/rails/issues/40607
Please see update at bottom
I'm trying to use the 'delayed_jobs' gem in my project. I got everything working in my local environment, however when I moved the project to production on Heroku I was unable to 'rake db:migrate', and am not able to get my server started. Here are the steps I followed.
*Run bundle_install with 'delayed_job' and 'delayed_job_active_record' in my gem file
*I did not run 'ails generate delayed_job:active_record' as the migration file was moved to the server from when I ran 'ails generate delayed_job:active_record' in DEV
*I ran rake db:migrate and it failed with the following error, not sure how to fix. (Note: It appears that the migration is failing due to the table not existing, which is a big issue as I need to run the migration to create the table).
[?25l[?25hUnable to write PID file to /app/tmp/pids/delayed_job.pid
[ActiveJob] Enqueued SquareOrderCleanupJob (Job ID: c364f5c1-a3c3-401a-95a6-a567a2813942) to DelayedJob(default) at 2016-06-04 14:57:56 UTC
[ActiveJob] Mysql2::Error: Table 'heroku_fc2daea183dcdd6.delayed_jobs' doesn't exist: SHOW FULL FIELDS FROM `delayed_jobs`
rake aborted!
ActiveRecord::StatementInvalid: Mysql2::Error: Table 'heroku_fc2daea183dcdd6.delayed_jobs' doesn't exist: SHOW FULL FIELDS FROM `delayed_jobs`
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `block in execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:473:in `block in log'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:467:in `log'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:312:in `execute_and_free'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:457:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/schema_cache.rb:43:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/attributes.rb:93:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/attributes.rb:98:in `columns_hash'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:205:in `subclass_from_attributes?'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:54:in `new'
/app/vendor/bundle/ruby/2.2.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:16:in `enqueue_job'
/app/vendor/bundle/ruby/2.2.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:12:in `enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/queue_adapters/delayed_job_adapter.rb:22:in `enqueue_at'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/enqueuing.rb:69:in `block in enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:553:in `block (2 levels) in compile'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:496:in `block (2 levels) in around'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:341:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:341:in `block (2 levels) in simple'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:14:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:14:in `block (3 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:43:in `block in tag_logger'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:68:in `block in tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:26:in `tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:68:in `tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:43:in `tag_logger'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:13:in `block (2 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:439:in `instance_exec'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:439:in `block in make_lambda'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:340:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:340:in `block in simple'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:495:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:495:in `block in around'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:88:in `run_callbacks'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/enqueuing.rb:67:in `enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/configured_job.rb:13:in `perform_later'
/app/config/initializers/schedule_jobs.rb:2:in `'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `load'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `block in load'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:240:in `load_dependency'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `load'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:652:in `block in load_config_initializer'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/notifications.rb:166:in `instrument'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:651:in `load_config_initializer'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:616:in `block (2 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:615:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:615:in `block in '
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:30:in `instance_exec'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:30:in `run'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:55:in `block in run_initializers'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `tsort_each_child'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `tsort_each_child'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:54:in `run_initializers'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:352:in `initialize!'
/app/config/environment.rb:5:in `'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `require'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `block in require'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:240:in `load_dependency'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `require'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:328:in `require_environment!'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:457:in `block in run_tasks_blocks'
Mysql2::Error: Table 'heroku_fc2daea183dcdd6.delayed_jobs' doesn't exist
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'
/app/vendor/bundle/ruby/2.2.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `block in execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:473:in `block in log'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:467:in `log'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:305:in `execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:312:in `execute_and_free'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:457:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/connection_adapters/schema_cache.rb:43:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/attributes.rb:93:in `columns'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/attributes.rb:98:in `columns_hash'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:205:in `subclass_from_attributes?'
/app/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.3/lib/active_record/inheritance.rb:54:in `new'
/app/vendor/bundle/ruby/2.2.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:16:in `enqueue_job'
/app/vendor/bundle/ruby/2.2.0/gems/delayed_job-4.1.2/lib/delayed/backend/base.rb:12:in `enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/queue_adapters/delayed_job_adapter.rb:22:in `enqueue_at'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/enqueuing.rb:69:in `block in enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:115:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:553:in `block (2 levels) in compile'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:496:in `block (2 levels) in around'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:341:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:341:in `block (2 levels) in simple'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:14:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:14:in `block (3 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:43:in `block in tag_logger'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:68:in `block in tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:26:in `tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/tagged_logging.rb:68:in `tagged'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:43:in `tag_logger'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/logging.rb:13:in `block (2 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:439:in `instance_exec'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:439:in `block in make_lambda'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:340:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:340:in `block in simple'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:495:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:495:in `block in around'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:503:in `call'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/callbacks.rb:88:in `run_callbacks'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/enqueuing.rb:67:in `enqueue'
/app/vendor/bundle/ruby/2.2.0/gems/activejob-4.2.3/lib/active_job/configured_job.rb:13:in `perform_later'
/app/config/initializers/schedule_jobs.rb:2:in `'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `load'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `block in load'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:240:in `load_dependency'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:268:in `load'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:652:in `block in load_config_initializer'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/notifications.rb:166:in `instrument'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:651:in `load_config_initializer'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:616:in `block (2 levels) in '
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:615:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/engine.rb:615:in `block in '
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:30:in `instance_exec'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:30:in `run'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:55:in `block in run_initializers'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `tsort_each_child'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `each'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:44:in `tsort_each_child'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/initializable.rb:54:in `run_initializers'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:352:in `initialize!'
/app/config/environment.rb:5:in `'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `require'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `block in require'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:240:in `load_dependency'
/app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/dependencies.rb:274:in `require'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:328:in `require_environment!'
/app/vendor/bundle/ruby/2.2.0/gems/railties-4.2.3/lib/rails/application.rb:457:in `block in run_tasks_blocks'
Tasks: TOP => db:migrate => environment
(See full trace by running task with --trace)
[?25h
UPDATE
I figured out the problem was caused by executing the following code at the initialization of my rails app. I was able to get the migration to work by temporarily removing it, and then adding it back in after the migration was complete (the code require the migration to be run before it was run). This is a temporary patch so I am going to work on getting a better solution for future PROD pushes. I'm thinking I can find a way to check if the table exists before running the code, but if anyone knows of a better way please let me know.
system("ruby lib/background_job start")
SquareOrderCleanupJob.set(wait: 10.minute).perform_later()
at_exit do
system("ruby lib/background_job stop")
end
You can check if a table exists, and therefore if migrations have been run:
ActiveRecord::Base.connection.table_exists? 'table_name'
That answer is also in SO here.
After upgrading to Ruby 2.1.5 from Ruby 2.1.2, sending emails with delayed_mailhopper creates this error:
ArgumentError: wrong number of arguments (1 for 0)
…ailhopper-0.0.7/app/models/delayed_mailhopper/email.rb: 9:in `initialize'
…ailhopper-0.0.7/app/models/delayed_mailhopper/email.rb: 9:in `new'
…ailhopper-0.0.7/app/models/delayed_mailhopper/email.rb: 9:in `enqueue'
…0/gems/mailhopper-0.3.0/app/models/mailhopper/email.rb: 25:in `create_from_mail'
…by/2.1.0/gems/mailhopper-0.3.0/lib/mailhopper/queue.rb: 7:in `deliver!'
…/bundle/ruby/2.1.0/gems/mail-2.5.4/lib/mail/message.rb:2129:in `do_delivery'
…/bundle/ruby/2.1.0/gems/mail-2.5.4/lib/mail/message.rb: 232:in `block in deliver'
…/bundle/ruby/2.1.0/gems/mail-2.5.4/lib/mail/message.rb: 232:in `deliver'
…ms/delayed_job-4.0.0/lib/delayed/performable_mailer.rb: 6:in `perform'
…1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb: 105:in `block in invoke_job'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `block in initialize'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `execute'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 38:in `run_callbacks'
…1.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb: 102:in `invoke_job'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 206:in `block (2 levels) in run'
/usr/local/lib/ruby/2.1.0/timeout.rb: 91:in `block in timeout'
/usr/local/lib/ruby/2.1.0/timeout.rb: 101:in `call'
/usr/local/lib/ruby/2.1.0/timeout.rb: 101:in `timeout'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 206:in `block in run'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 205:in `run'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 267:in `block in reserve_and_run_one_job'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `block in initialize'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `execute'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 38:in `run_callbacks'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 267:in `reserve_and_run_one_job'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 189:in `block in work_off'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 188:in `times'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 188:in `work_off'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 153:in `block (4 levels) in start'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 152:in `block (3 levels) in start'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `block in initialize'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `execute'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 38:in `run_callbacks'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 151:in `block (2 levels) in start'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 150:in `loop'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 150:in `block in start'
…s/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb: 7:in `call'
…s/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb: 7:in `block (2 levels) in <class:ClearLocks>'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 78:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 78:in `block (2 levels) in add'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 60:in `block in initialize'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 78:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 78:in `block in add'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `call'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 65:in `execute'
…/2.1.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb: 38:in `run_callbacks'
…uby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb: 149:in `start'
…ruby/2.1.0/gems/delayed_job-4.0.0/lib/delayed/tasks.rb: 9:in `block (2 levels) in <top (required)>'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 240:in `call'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 240:in `block in execute'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 235:in `each'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 235:in `execute'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 179:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/2.1.0/monitor.rb: 211:in `mon_synchronize'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 172:in `invoke_with_call_chain'
…ed/bundle/ruby/2.1.0/gems/rake-10.3.2/lib/rake/task.rb: 165:in `invoke'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 150:in `invoke_task'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 106:in `block (2 levels) in top_level'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 106:in `each'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 106:in `block in top_level'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 115:in `run_with_threads'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 100:in `top_level'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 78:in `block in run'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 176:in `standard_exception_handling'
…le/ruby/2.1.0/gems/rake-10.3.2/lib/rake/application.rb: 75:in `run'
I've narrowed it down to this line of code Delayed::Job.enqueue(SendJob.new(self.id)) (view code) and I've tried just pasting Delayed::Job.enqueue(SendJob.new(1)) and SendJob.new(1) into rails console, and I get the same errors:
ArgumentError: wrong number of arguments (1 for 0)
from (irb):14:in `initialize'
from (irb):14:in `new'
from (irb):14
from /var/.../web_head/shared/bundle/ruby/2.1.0/gems/railties-3.2.21/lib/rails/commands/console.rb:47:in `start'
from /var/.../web_head/shared/bundle/ruby/2.1.0/gems/railties-3.2.21/lib/rails/commands/console.rb:8:in `start'
from /var/.../web_head/shared/bundle/ruby/2.1.0/gems/railties-3.2.21/lib/rails/commands.rb:41:in `<top (required)>'
from script/rails:6:in `require'
from script/rails:6:in `<main>'
I'm running Rails 2.3.21 for both Ruby environments. It still works in the older Ruby 2.1.2 environment. Any thoughts?
I'm using Delayed Job gem to dispatch emails using Amazon SES. However it runs occasionally into execution expired error. Probably some connection issues to SES. I'm wondering how can I catch this exception and re-schedule the email manually? Due to internal reasons Delayed Job is configured to single max attempt
| execution expired
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `initialize'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `open'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:540:in `tcp_socket'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:550:in `block in do_start'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:549:in `do_start'
/usr/local/lib/ruby/2.0.0/net/smtp.rb:519:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/network/delivery_methods/smtp.rb:112:in `deliver!'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:2129:in `do_delivery'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:232:in `block in deliver'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/actionmailer-4.0.0/lib/action_mailer/base.rb:456:in `block in deliver_mail'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `block in instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/activesupport-4.0.0/lib/active_support/notifications.rb:159:in `instrument'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/actionmailer-4.0.0/lib/action_mailer/base.rb:454:in `deliver_mail'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/mail-2.5.4/lib/mail/message.rb:232:in `deliver'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/performable_mailer.rb:6:in `perform'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:105:in `block in invoke_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/backend/base.rb:102:in `invoke_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block (2 levels) in run'
/usr/local/lib/ruby/2.0.0/timeout.rb:66:in `timeout'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:206:in `block in run'
/usr/local/lib/ruby/2.0.0/benchmark.rb:296:in `realtime'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:205:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `block in reserve_and_run_one_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:267:in `reserve_and_run_one_job'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:189:in `block in work_off'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `times'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:188:in `work_off'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:153:in `block (4 levels) in start'
/usr/local/lib/ruby/2.0.0/benchmark.rb:296:in `realtime'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:152:in `block (3 levels) in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:151:in `block (2 levels) in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `loop'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:150:in `block in start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/plugins/clear_locks.rb:7:in `block (2 levels) in <class:ClearLocks>'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block (2 levels) in add'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:60:in `block in initialize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:78:in `block in add'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:65:in `execute'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/lifecycle.rb:38:in `run_callbacks'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/worker.rb:149:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:104:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:92:in `block in run_process'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:255:in `block in start_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/daemonize.rb:82:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/daemonize.rb:82:in `call_as_daemon'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:259:in `start_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application.rb:296:in `start'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:159:in `block (2 levels) in start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:158:in `fork'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:158:in `block in start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:157:in `each'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/application_group.rb:157:in `start_all'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/controller.rb:80:in `run'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons.rb:197:in `block in run_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `call'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons/cmdline.rb:109:in `catch_exceptions'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/daemons-1.1.9/lib/daemons.rb:196:in `run_proc'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:90:in `run_process'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:83:in `block in daemonize'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:81:in `times'
/home/ubuntu/www/tf/shared/bundle/ruby/2.0.0/gems/delayed_job-4.0.0/lib/delayed/command.rb:81:in `daemonize'
/home/ubuntu/www/tf/current/bin/delayed_job:5:in `<main>'
delayed_job provides some hooks, you can use the error hook to rescheduled the task.
def error(job, exception)
# if exception is the one occurring right now for expired execution
# reschedule the job
# be careful to avoid infinite rescheduling
end