Resque could not find job class ZF2 namespaces - zend-framework2

Attempting to use the Redis backed PHP-Resque project https://github.com/chrisboulton/php-resque from within a ZF2 project. (Apigility to be specific)
What I'm having trouble with, is combining the ZF2 namespaces and the classes
for instance :
Controller
//Enqueue a worker
$args = array(
'name' => 'EMCP'
);
Resque::enqueue('default', 'phpresque\\V1\\Model\\MyResqueJob', $args);
MyResqueJob Class
namespace phpresque\V1\Model;
class MyResqueJob
{
public function perform()
{
// Work work work
echo "helloWorld";
}
}
Error message given by resque :
ubuntu#/zf2projectname/vendor/chrisboulton/php-resque$ QUEUE=* APP_INCLUDE=/zf2projectname/vendor/autoload.php VVERBOSE=1 php resque.php
** [03:01:37 2014-10-26] Sleeping for 5
** [03:01:42 2014-10-26] Checking default
** [03:01:42 2014-10-26] Found job on default
** [03:01:42 2014-10-26] got (Job{default} | ID: 38fa104b11de81731c15ba9c2f1853ab | phpresque\V1\Model\MyResqueJob | [{"name":"EMCP"}])
** [03:01:42 2014-10-26] Forked 30642 at 2014-10-26 03:01:42
** [03:01:42 2014-10-26] Processing default since 2014-10-26 03:01:42
** [03:01:42 2014-10-26] (Job{default} | ID: 38fa104b11de81731c15ba9c2f1853ab | phpresque\V1\Model\MyResqueJob | [{"name":"EMCP"}]) failed: Could not find job class phpresque\V1\Model\MyResqueJob.
** [03:01:42 2014-10-26] Checking default

Edit : I dropped using Resque but here's the suggestion below on how it could work.. I ended up using SlmQueue instead https://github.com/juriansluiman/SlmQueue
According to the ZF2 github community, the following should be done :
Shouldn't your APP_INCLUDE environment var contain the path to composer's init_autoloader.php? Just including the autoloader class will not suffice, as you not configuring and registering the autoloader anywhere.
and
YMMV, but what I've done to integrate my ZF2 app into Resque is create a separate entry point script that initializes the ZF2 app within the context of the Resque worker:
<?php
// resque_context.php
require_once 'vendor/autoload.php';
$application = \Zend\Mvc\Application::init(include 'config/application.config.php');
$config = $application->getServiceManager()->get('config');
And provide it's path to the Resque run script via the APP_INCLUDE environment variable.

Related

Resque not failing an obviously buggy job

I have the following problem with Resque and Rails 6 (ruby 2.7.2). I am new to this gem and wanted to do some simple testing to see how things are handled, so I created a simple failing job:
class TestJob < ApplicationJob
#queue = :default
def self.perform(*args)
0/0
end
end
Obviously, when I go to the console and run it, I get the error:
irb(main):001:0> TestJob.perform
Traceback (most recent call last):
3: from (irb):1
2: from app/jobs/test_job.rb:5:in `perform'
1: from app/jobs/test_job.rb:5:in `/'
ZeroDivisionError (divided by 0)
But when I run my worker with the command:
QUEUE=* VVERBOSE=true rake resque:work
And then enqueue my job in rails console, the output I get is:
irb(main):002:0> Resque.enqueue(TestJob)
=> true
And in the resque logs:
** [21:32:57 2020-12-09] 15198: resque-2.0.0: Waiting for default
** [21:33:02 2020-12-09] 15198: Checking default
** [21:33:02 2020-12-09] 15198: Found job on default
** [21:33:02 2020-12-09] 15198: resque-2.0.0: Processing default since 1607545982 [TestJob]
** [21:33:02 2020-12-09] 15198: got: (Job{default} | TestJob | [])
** [21:33:02 2020-12-09] 15198: resque-2.0.0: Forked 15765 at 1607545982
** [21:33:02 2020-12-09] 15765: done: (Job{default} | TestJob | [])
Additionally, in the resque web UI it counts this process as successful. The number of failed jobs stays 0. Furthermore, any puts and raise I put into the perform method are not showing in resque/rails console logs.
I have done a basic resque setup on a fresh rails app following the README on the resque's github page. I did not do anything custom. Obviously I restarted my console and worker after I made changes to the test_job.rb file, so this is not a problem of loading (maybe?). I also tried changing self.perform to perform.
I don't know why, but this is the solution. Rather than running:
Resque.enqueue(TestJob)
one should run:
TestJob.perform_later
The thing is, the first way is from the Resque README, and the second one is from the Rails docs. I guess I was fixated on doing this the first way, but it looks like going with the rails conventions is always best.

Knife bootstrap failing through Jenkins execute shell

I'm trying to perform a "knife bootstrap" command through Jenkins web UI execute shell, but I keep getting this error message :
(this is the knife bootstrap command I'm using) :
"knife bootstrap [the node's IP] --ssh-user ec2-user --sudo --identity-file "[my key to the node]" --node-name My123 --run-list 'role[role1]' "
and this is the error message:
" ERROR: Errno::ENOENT: No such file or directory # rb_sysopen - /etc/chef/validation.pem "
when I run the 'knife bootstrap' command directly through the CLI it works fine.
any idea why it's not working from Jenkins execute shell?
It is due to validation.pem file is missing, this is default path for validation file. Either you can set path in /chef-repo/.chef/knife.rb file or you can use default location /etc/chef/validation.pem.
You can regenerate validation key from webUI and replace the existing one, this should resolve your issue.

symfony/yaml backed symfony/config not parsing environment variables

I have recreated a simple example in this tiny github repo. I am attempting to use symfony/dependency-injection to configure monolog/monolog to write logs to php://stderr. I am using a yaml file called services.yml to configure dependency injection.
This all works fine if my yml file looks like this:
parameters:
log.file: 'php://stderr'
log.level: 'DEBUG'
services:
stream_handler:
class: \Monolog\Handler\StreamHandler
arguments:
- '%log.file%'
- '%log.level%'
log:
class: \Monolog\Logger
arguments: [ 'default', ['#stream_handler'] ]
However, my goal is to read the path of the log files and the log level from environment variables, $APP_LOG and LOG_LEVEL respectively. According to The symphony documentations on external paramaters the correct way to do that in the services.yml file is like this:
parameters:
log.file: '%env(APP_LOG)%'
log.level: '%env(LOGGING_LEVEL)%'
In my sample app I verified PHP can read these environment variables with the following:
echo "Hello World!\n\n";
echo 'APP_LOG=' . (getenv('APP_LOG') ?? '__NULL__') . "\n";
echo 'LOG_LEVEL=' . (getenv('LOG_LEVEL') ?? '__NULL__') . "\n";
Which writes the following to the browser when I use my original services.yml with hard coded values.:
Hello World!
APP_LOG=php://stderr
LOG_LEVEL=debug
However, if I use the %env(VAR_NAME)% syntax in services.yml, I get the following error:
Fatal error: Uncaught UnexpectedValueException: The stream or file "env_PATH_a61e1e48db268605210ee2286597d6fb" could not be opened: failed to open stream: Permission denied in /var/www/vendor/monolog/monolog/src/Monolog/Handler/StreamHandler.php:107 Stack trace: #0 /var/www/vendor/monolog/monolog/src/Monolog/Handler/AbstractProcessingHandler.php(37): Monolog\Handler\StreamHandler->write(Array) #1 /var/www/vendor/monolog/monolog/src/Monolog/Logger.php(337): Monolog\Handler\AbstractProcessingHandler->handle(Array) #2 /var/www/vendor/monolog/monolog/src/Monolog/Logger.php(532): Monolog\Logger->addRecord(100, 'Initialized dep...', Array) #3 /var/www/html/index.php(17): Monolog\Logger->debug('Initialized dep...') #4 {main} thrown in /var/www/vendor/monolog/monolog/src/Monolog/Handler/StreamHandler.php on line 107
What am I doing wrong?
Ok you need a few things here. First of all you need version 3.3 of Symfony, which is still in beta. 3.2 was the released version when I encountered this. Second you need to "compile" the environment variables.
Edit your composer.json with the following values and run composer update. You might need to update other dependencies. You can substitute ^3.3 with dev-master.
"symfony/config": "^3.3",
"symfony/console": "^3.3",
"symfony/dependency-injection": "^3.3",
"symfony/yaml": "^3.3",
You will likely have to do this for symfony/__WHATEVER__ if you have other symfony components.
Now in you're code after you load your yaml configuration into your dependency container you compile it.
So after you're lines here (perhaps in bin/console):
$container = new ContainerBuilder();
$loader = new YamlFileLoader($container, new FileLocator(__DIR__ . DIRECTORY_SEPARATOR . '..'));
$loader->load('services.yml');
Do this:
$container->compile(true);
Your IDE's intellisense might tell you compile takes no parameters. That's ok. That's because compile() grabs its args indirectly via func_get_arg().
public function compile(/*$resolveEnvPlaceholders = false*/)
{
if (1 <= func_num_args()) {
$resolveEnvPlaceholders = func_get_arg(0);
} else {
. . .
}
References
Github issue where this was discussed
Pull request to add compile(true)
Using this command after loading your services.yaml file should help.
$containerBuilder->compile(true);
given your files gets also validated by the checks for proper configurations which this method also does. The parameter is $resolveEnvPlaceholders which makes environmental variables accessible to the yaml services configuration.

how to set a grails 3 app to use a root context path?

I'm trying to set a root context path for my application in `conf/application.yml' like so:
server:
'context-path': '/'
However, when trying to start grails> run-app, I get the following exception:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':bootRun'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_20.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1
From Grails 3.0.3, the expected configuration key is contextPath, camel case and not hyphenated. So the right setting in your application.yml is:
server:
contextPath: '/my-path'
Or, if you're using application.groovy:
server.contextPath='/my-path'
I should have paid more attention to the error log output from run-app. The correct setting is:
server:
'context-path': ''
>= Grails 3.0.3:
server:
contextPath: ''

Resque-Scheduler not working with ActiveJob in Rails 4.2

Has anyone been able to get scheduled jobs to work in Rails 4.2?
I am using resque, and I am attempting to use resque-scheduler to schedule jobs. I have a schedule that get loaded and the scheduler runs, and even looks like it is running the jobs but it doesn't do anything.
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Starting
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Loading Schedule
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Scheduling friends
resque-scheduler: [INFO] 2014-09-16T01:54:25-07:00: Schedules Loaded
resque-scheduler: [INFO] 2014-09-16T01:54:55-07:00: queueing FriendsJob (friends)
I can enqueue jobs like this and they get processed.
TestJob.enqueue(params[:id])
and I get this output in the worker log
got: (Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])
** [01:24:01 2014-09-16] 54841: resque-1.25.2: Processing default since 1410855841 [ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper]
** [01:24:01 2014-09-16] 54841: Running before_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]
** [01:24:01 2014-09-16] 54841: resque-1.25.2: Forked 54882 at 1410855841
** [01:24:01 2014-09-16] 54882: Running after_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]
Hello World!!
** [01:24:01 2014-09-16] 54882: done: (Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])
But when I try to schedule a job, it looks like the are getting enqueue but they well not doing anything.
Here is the schedule.yml
friends:
every: "30s"
queue: "friends"
class: "FriendsJob"
args: 8
description: "Friends jobs scheduler"
Here is the output from the scheduled job.
** [01:23:36 2014-09-16] 54841: got: (Job{friends} | FriendsJob | [8])
** [01:23:36 2014-09-16] 54841: resque-1.25.2: Processing friends since 1410855816 [FriendsJob]
** [01:23:36 2014-09-16] 54841: Running before_fork hooks with [(Job{friends} | FriendsJob | [8])]
** [01:23:36 2014-09-16] 54841: resque-1.25.2: Forked 54880 at 1410855816
** [01:23:36 2014-09-16] 54880: Running after_fork hooks with [(Job{friends} | FriendsJob | [8])]
** [01:23:36 2014-09-16] 54880: done: (Job{friends} | FriendsJob | [8])
After reading this http://dev.mikamai.com/post/96343027199/rails-4-2-new-gems-active-job-and-global-id
I am suspecting it has something to do with ActiveJob and GlobalId.
Take a look at the difference enqueued
** [01:24:01 2014-09-16] 54841: Running before_fork hooks with [(Job{default} | ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper | ["TestJob", "98732ce5-17f7-4da3-9a03-a5d2f8f74e84", "8"])]
vs scheduled
** [01:23:36 2014-09-16] 54841: Running before_fork hooks with [(Job{friends} | FriendsJob | [8])]
The jobs themselves were generated via
rails g job <JobName>
They look like this:
class TestJob < ActiveJob::Base
queue_as :default
def perform(friend_id)
friend = Friend.find(friend_id)
name = friend.name.swapcase
puts "Hello World!!"
end
end
class FriendsJob < ActiveJob::Base
queue_as :friends
def perform(friend_id)
friend = Friend.find(friend_id)
name = friend.name.swapcase
puts "Hello World!!"
end
end
Any help with this will be greatly appreciated and thanks in advance.
********* UPDATE *********
I've removed the action_job railtie and I am using Resque and Resque-Scheduler only, and scheduled jobs are working now. So this does seems to be related to ActionJob/GlobalId. I've open an issue in the rails project. Hopefully, they will fix it soon.
****** SECOND UPDATE *********
I got an update on this. Cristianbica who works in the ActiveJob codebase said this.
"We haven't thought at recurring jobs so far and we don't support this as none of the adapters support this without an external gem. However this is a very nice feature but I don't think we can make it in time for 4.2. Also I'm not sure it will suitable to be included in rails"
https://github.com/JustinAiken/active_scheduler is a gem that wraps the one into the other
It seems that ActiveJob in Rails 4.2 is not supported by resque-scheduler. Therefore, jobs are not scheduled correctly, that explains the difference in the log when a job is enqueued using ActiveJob API and resque-scheduler.
To fix this, we should find a way to schedule job within ActiveJob wrapper:
ActiveJob::QueueAdapters::ResqueAdapter::JobWrapper
Resque-scheduler provides the way to support extensions that are not supported by default. To do this, we should extend custom job class to support the #scheduled method. This way we can enqueue job manually using ActiveJob API.
The easiest way is to write general code method in the base job, and then extend all jobs from it:
# custom base job
class Job < ActiveJob::Base
# method called by resque-scheduler to enqueue job in a queue
def self.scheduled(queue, klass, *args)
# create the job instance and pass the arguments
job = self.job_or_instantiate(*args)
# set correct queue
job.queue_name = queue
# enqueue job using ActiveJob API
job.enqueue
end
end
Reque-scheduler will call this method to schedule every single job extended from Job class. This way jobs are going to be enqueued within the ActiveJob wrapper. Result will be same as calling MyJob.perform_later(*args).
UPDATE: 4/4/16 - Whilst the below answer is still correct with the current version of Rails, I now use the active_scheduler gem created by Justin as mentioned in the answer above: https://stackoverflow.com/a/29155372/1299792
ORIGINAL ANSWER:
Avoid using ActiveJob if you need to schedule recurring jobs.
From the issue that Luis raised
As we are not currently supporting recurring jobs with ActiveJob we're going to close this. If it's possible to support recurring jobs I'm seeing this as a separate gem or in rails 5. Feature requests and talks around them are usually talked in the mailing list (https://groups.google.com/forum/#!forum/rubyonrails-core).
As a solution to your problem #luismadrigal I suggest you use the resque-scheduler way to do recurring job.
https://github.com/rails/rails/issues/16933#issuecomment-58945932
There was talk about creating a gem in the mailing list but I can't find any further information on it.

Resources