Quartz.net db Jobs only work if added as singleton - quartz.net

I have a program where I add my jobs into a db and schedule them as needed. I used to add my jobs in program.cs as singletons, which works fine but only allows me to add a single job per scheduler and I can't check if it's already there. So I moved all that into one scheduler, where I now check if the job's already in my db and if it isn't then I add it.
It adds the jobs fine but they don't work at all, whenever I try to call one I get a nullreference exception from JobRunShell.Run(). I can't for the life of me figure out why, the job looks exactly the same in the db and I can't find anything wrong with my code. Am I not supposed to add jobs like that?
Program.cs with job added as singleton
builder.Services.AddSingleton<IJobFactory, JobFactory>();
builder.Services.AddSingleton<ISchedulerFactory, StdSchedulerFactory>();
builder.Services.AddSingleton<MailHRNewEmployee>();
builder.Services.AddSingleton(new JobMetadata(Guid.NewGuid(), typeof(MailHRNewEmployee), "HR First Contact", "job blueprint"));
builder.Services.AddHostedService<MyScheduler>();
MyScheduler.cs
public async Task StartAsync(CancellationToken cancellationToken)
{
//Creating Scheduler
Scheduler = await schedulerFactory.GetScheduler();
Scheduler.JobFactory = jobFactory;
//Start Scheduler
await Scheduler.Start(cancellationToken);
//Create Jobs
var istrue = await Scheduler.CheckExists(new JobKey("HR First Contact", "DEFAULT"));
if (!(bool)istrue)
{
jobMetaData = new JobMetadata(Guid.NewGuid(), typeof(MailHRNewEmployee), "HR First Contact", "job blueprint");
IJobDetail jobDetail = CreateJob(jobMetaData);
await Scheduler.AddJob(jobDetail, true);
}
...(checking all the other jobs)
private IJobDetail CreateJob(JobMetadata jobMetadata)
{
return JobBuilder.Create(jobMetaData.JobType).WithIdentity(jobMetaData.JobName.ToString()).StoreDurably(true).WithDescription(jobMetaData.JobDescription).Build();
}
Exception Stack Trace for NullReferenceException
[10:43:31 ERR] Job DEFAULT.HR First Contact threw an unhandled Exception:
System.NullReferenceException: Object reference not set to an instance of an object.
at Quartz.Core.JobRunShell.Run(CancellationToken cancellationToken)
[10:43:32 ERR] Job DEFAULT.HR First Contact threw an exception.
Quartz.SchedulerException: Job threw an unhandled exception.
---> System.NullReferenceException: Object reference not set to an instance of an object.
at Quartz.Core.JobRunShell.Run(CancellationToken cancellationToken)
--- End of inner exception stack trace --- [See nested exception: System.NullReferenceException: Object reference not set to an instance of an object.
at Quartz.Core.JobRunShell.Run(CancellationToken cancellationToken)]
Edit:
I tried using the MS DI Integration library with Quartz Hosted services but couldn't get it to work with persistent Job storage & MySql.
builder.Services.Configure<QuartzOptions>(builder.Configuration.GetSection("Quartz"));
builder.Services.Configure<QuartzOptions>(options =>
{
options.Scheduling.IgnoreDuplicates = true;
options.Scheduling.OverWriteExistingData = true;
});
builder.Services.AddQuartz(q =>
{
q.SchedulerId = "Job Creator";
q.UseMicrosoftDependencyInjectionJobFactory();
var jobKey = new JobKey("HR First Contact", "DEFAULT");
q.AddJob<MailHRNewEmployee>(jobKey, j => j
.StoreDurably()
.WithDescription("job blueprint"));
q.UsePersistentStore(s =>
{
s.PerformSchemaValidation = true;
s.UseProperties = true;
s.RetryInterval = TimeSpan.FromSeconds(15);
//Neither .UseMySql nor UseMySqlConnector work
//MySqlConnector should be the one I want though
s.UseMySqlConnector(MySql =>
{
//I am loading the configuration above but don't know how to use the connection string here?
//I thought it would maybe be Configuration.GetConnectionString("xyz");
MySql.ConnectionString = "MyConnectionString";
MySql.TablePrefix = "QRTZ_";
});
s.UseJsonSerializer();
});
});
builder.Services.AddTransient<MailHRNewEmployee>();
builder.Services.AddQuartzHostedService(options =>
{
options.WaitForJobsToComplete = true;
});

Jobs should be registered as transient, Quartz expects to get a new instance from job factory. Have you considered using the MS DI integration library and hosted services integration library?

Related

Awaited call loses context in Unit Test

We have the following ASP.NET code (I've cut it down for clarity):
var httpContent = new StringContent(postData, Encoding.UTF8);
using (var client = this.GetClient(url, contentType))
{
using (var response = await client.PostAsync(url, httpContent).ConfigureAwait(true);
{
ret.Response = await response.Content.ReadAsStringAsync().ConfigureAwait(true);
}
}
'client' is an HttpClient object when run in PROD, but in our Unit Test it's:
MockRepository.GenerateMock<IHttpClient>();
Notice that we use .ConfigureAwait(true). The reason for this is that we store objects in the HttpContext.Current.Items collection (this probably isn't ideal, but it is what it is). If we didn't persist the Thread's Context then HttpContext.Current returns as null.
This all works as expected when run in PROD.
However, when running the Unit Test, execution of the following line loses Context and HttpContext.Current becomes null:
await response.Content.ReadAsStringAsync().ConfigureAwait(true);
Note - the proceeding call of PostAsync on the Stub maintains the Context.
So how do we program the Stub?
using (var httpResponse = new HttpResponseMessage())
{
httpResponse.Content = new StringContent(msg, Encoding.UTF8);
httpResponse.StatusCode = HttpStatusCode.OK;
this.FakeHttpClient
.Stub(i => i.PostAsync("", "")).IgnoreArguments()
.Return(Task.FromResult(this.httpResponse)).Repeat.Once();
// Act
}
When the Test runs, the data is returned so the fake objects behave perfectly EXCEPT that the ReadAsStringAsync().ConfigureAwait(true) destroys the Context.
StringContent can't be mocked directly as it doesn't have a parameterless constructor. I created a wrapper class to see if I could mock that, but when I had the following code:
var fakeStringContent = MockRepository.GenerateStub<Wrap>();
fakeStringContent
.Stub(fsc => fsc.ReadAsStringAsync())
.Return(Task.FromResult("<response/>"))
.Repeat.Once();
Then there was an Exception thrown on the .Stub(...) line which was:
System.InvalidOperationException: 'The async operation did not return
a System.Threading.Tasks.Task object.'

Error during merge after async task

I have an application which basically calls multiple webservices, stores the data received from those webservices and renders it to the user. I have an async task that call all the webservices and it looks something like this:
List<Promise> t = new ArrayList<Promise>()
def TIMEOUT_TIME = 6
t[0] = task {
def responseFrom0 = webserviceRequestService.getResponse(0)
if(responseFrom0){
return responseFrom0
}
}
t[1] = task {
def responseFrom1 = webserviceRequestService.getResponse(1)
if(responseFrom1){
return responseFrom1
}
}
The action getResponse looks something like this:
List<ResponseResult> result = new ArrayList<TravelQuote>()
try {
wsClient = prepareRequestMap()
wsResponse = externalWebservice.getQuotes(wsClient)
wsResponse.responseList.each {
ResponseResult responseResult = new ResponseResult()
//populate properties of ResponseResult
responseResult.save(failOnError:true, flush:true)
result.add(responseResult)
}
} catch(Exception e){
log.error e.message
e.printStackTrace()
}
return result
And at the end, I collect all the responses from all webservices like this:
result.each {
if(it){
try{
it=it.merge()
}catch (Exception e){
log.error("Error occured while merging responses... : ${e.message}")
}
}
}
Now, the issue here is I get this exception from the last block of code
not-null property references a null or transient value: ResponseResult.dateCreated; nested exception is org.hibernate.PropertyValueException: not-null property references a null or transient value: ResponseResult.dateCreated
The dateCreated comes from this class which I have implemented on all of my domain classes.
abstract class AbstractPersistentObject implements Serializable{
static final long serialVersionUID = 1L;
Date dateCreated
Date lastUpdated
}
The weird thing about this issue is, this happens only on production environment, no matter what I do, I cannot replicate this in any other environments. And also, once this happens, the server just throws that particular issue every time that code is run until the server is restarted. After a restart, this code works fine.
I am running out of ideas, anyone has any ideas?
The answer seems to be in the exception thrown: the external web service returns record(s) without value for dateCreated field.
In my experience data from production environments almost always contains missing or improperly formatted values. You should account for that by changing the definition of ResponseResult.dateCreated to allow null values and handle this scenario in your code.

Task.Factory.StartNew does not start when method inside have a new parameter

(Problem solved)
I have an MVC application, in my Action:
First case: Task never started.
public ActionResult Insert(NewsManagementModel model) {
//Do some stuff
//Insert history
//new object NewsHistoryDto as the parameter
Task.Factory.StartNew(() => InsertNewsHistory(new NewsHistoryDto {
UserId = 1234,
Time = DateTime.Now,
Id = 1234
}));
return RedirectToAction("Index", "NewsManagement");
}
Second case: Task run normally
public ActionResult Insert(NewsManagementModel model) {
//Do some stuff
//Insert history
//object NewsHistoryDto was declared outside
var history = new NewsHistoryDto {
UserId = 1234,
Time = DateTime.Now,
Id = 1234
};
Task.Factory.StartNew(() => InsertNewsHistory(history));
return RedirectToAction("Index", "NewsManagement");
}
My question is: When Task.Factory.StartNew and i put a method into it, the parameter of that method (object) must be declare outside??? Because when i write shortly like the first case, i put the "new" keyword in the parameter and the task never run.
Reason: In action, i wanna return view as soon as possible, any other stuff not related to that view will be excute in a task and client no need to wait for completed.
I'm very sorry about my bad english :)
Updated 1:
Thanks Panagiotis Kanavos, I used QueueBackgroundWorkItem but the problem still the same, if i declare the object outside, this method run normally. But when i use new keyword inside the parameter, this method never run. No exception, no errors. Can anyone explain to me how this possible :(
Updated 2:
I try two case:
First:
HostingEnvironment.QueueBackgroundWorkItem(delegate {
var handler = m_bussinessHandler;
handler.InsertNewsHistoryAsync(new NewsHistoryDto {
UserId = UserModel.Current.UserId,
Time = DateTime.Now,
Id = newsId
});
});-> still doens't works
Second:
var history = new NewsHistoryDto {
UserId = UserModel.Current.UserId,
Time = DateTime.Now,
Id = newsId
};
HostingEnvironment.QueueBackgroundWorkItem(delegate {
var handler = m_bussinessHandler;
handler.InsertNewsHistoryAsync(history);
});-> works normally
So where is the problem here??? That's not about m_bussinessHandler because i copied.
Updated 3: I found the reason.
The reason is UserModel.Current, this is an object in HttpContext.Current.Session["UserModel"], in this case when i call async method, when this method actually excute, it can access to HttpContext.Current which is null. So i can solve this problem by declare object outside to store data and pass it into method or I capture UserModel.Current and pass it into this method to use UserModel.Current.UserId.
My problem actually solved, thanks everyone for helping me, especially Panagiotis Kanavos.
As it is, your code returns to the caller before the task finishes. The task may not have even started executing by the time the code returns, especially if you are debugging the method. The debugger freezes all threads and executes only one of them step by step.
Moreover, .NET's reference capture means that when you use m_businessHandler you capture a reference to the m_businessHandler field, not its value. If your controller gets garbage collected, this will result in a NullReferenceException. To avoid this you have to make a copy of the field's value inside your lambda.
You should write a proper asynchronous method instead, that returns to the user only when the asynchronous operation completes:
public async Task<ActionResult> Insert(NewsManagementModel model) {
//Do some stuff
//Insert history
//new object NewsHistoryDto as the parameter
await Task.Run(() =>
var handler=m_bussinessHandler;
handler.InsertNewsHistory(new NewsHistoryDto {
UserId = 1234,
Time = DateTime.Now,
Id = 1234
}));
return RedirectToAction("Index", "NewsManagement");
}
Task.Run or Task.Factory.StartNew are roughly equivalent.
Even so, it's not a good idea to use Task.Run to fake asynchronous execution - you simply switched execution from one server thread to another. You should use asynchronous methods all the way down to the database, eg. use ExecuteNonQueryAsync if you are using ADO.NET or SaveChangesAsync in Entity Framework. This way no thread executes or blocks while the code waits for the database call to complete.
The compiler also takes care of capturing the field's value so you don't need to copy anything. The resulting code is a lot cleaner:
public async Task<ActionResult> Insert(NewsManagementModel model) {
//Do some stuff
//Insert history
//new object NewsHistoryDto as the parameter
await m_bussinessHandler.InsertNewsHistoryAsync(new NewsHistoryDto {
UserId = 1234,
Time = DateTime.Now,
Id = 1234
};
return RedirectToAction("Index", "NewsManagement");
}
If you do want the operation to run in the background and return to the client immediately, you can use QueueBackgroundWorkItem which starts a new Task and registers it with IIS. In this case you have to copy the field's value again:
public ActionResult Insert(NewsManagementModel model) {
//Do some stuff
//Insert history
//new object NewsHistoryDto as the parameter
HostingEnvironment.QueueBackgroundWorkItem(ct=>
var handler=m_bussinessHandler;
handler.InsertNewsHistoryAsync(new NewsHistoryDto {
UserId = 1234,
Time = DateTime.Now,
Id = 1234
});
return RedirectToAction("Index", "NewsManagement");
}
IIS can still cancel the task, eg when the application pool recycles. Before it does so, it will notify the task through the CancellationTask passed to the lambda (the ct parameter). If the task doesn't finish in time, IIS will go on and abort the thread anyway. A long running task should check the token periodically.
You can pass the cancellation token to most asynchronous methods, eg ExecuteNonQueryAsync(CancellationToken). This will cause the IO operation to cancel as soon as it's safe to do so instead of waiting until the remote server responds.
Scott Hanselman has a nice article describing all the ways available to rung tasks and even scheduled jobs in the background in How to run Background Tasks in ASP.NET
Is your m_bussinessHandler an instance field? Because it can got disposed after you finished the actions.

How to check for existing Quartz.net schedulers?

Recently upgraded my embedded Quartz.net scheduler to 2.x at which point I had to give up on giving my Zero Thread Schedulers unique names and now I have a problem that (very rarely) an object trying to create an instance of ZT scheduler throws an exception because another object already has an instance of ZT scheduler instantiated and since all my ZT schedulers now have the default 'QuartzScheduler' name this throws an exception...
I tried checking for scheduler count using MySchedFactory.AllSchedulers.Count after calling MySchedFactory = new StdSchedulerFactory(properties) but the StdSchedulerFactory creates an instance of ZT scheduler as soon as it's instantiated and not when GetScheduler() method is called so that is a dead end...
I could not find any other way of checking for existing schedulers before instantiating the StdSchedulerFactory and, as I mentioned already, as soon as it is instantiated, it creates an instance of ZT scheduler so I ended up using a while loop in my catch block which is just a horrible solution so I'm hoping someone knows a better way of checking for existing ZT schedulers...
try
{
//setting properties
MySchedFactory = new StdSchedulerFactory(properties);
BaseScheduler = schedFactory.GetScheduler();
}
catch (Exception ex)
{
var exMsg = ex.InnerException == null ?
ex.Message :
ex.Message + Environment.NewLine + ex.InnerException.Message;
while (exMsg.Contains("Scheduler with name 'QuartzScheduler' already exists"))
{
try
{
MySchedFactory = new StdSchedulerFactory(properties);
BaseScheduler = schedFactory.GetScheduler();
}
catch (Exception vex)
{
exMsg = vex.InnerException == null ?
vex.Message :
vex.Message + Environment.NewLine + vex.InnerException.Message;
}
}
}
Any ideas?
How about keeping a reference to the scheduler factory as a singleton instead of creating a new one?
Quartz.net scheduler must be singleton. You can read something here.

how to perform two task in the backgroudworker process in asp.net MVC

I have to do two task parallelly in the backgroud worker process in the asp.net.
The following are the two task
Sent a email alert to support guys if any error event occurs in any of the server in a domain (All the time)
Sent the status report of error messages for all the servers in a domain at a time for every 4 hours or based on the user configuration.
I have already implemented the first task through the background worker process (
Now I want to implement second one and that fetch the data from the database for every 4 hours or user configured.
Please find the below are source code that i have used for implementation.
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
RegisterRoutes(RouteTable.Routes);
BackgroundWorker worker = new BackgroundWorker();
worker.DoWork += new DoWorkEventHandler(DoWork);
worker.WorkerReportsProgress = false;
worker.WorkerSupportsCancellation = true;
worker.RunWorkerCompleted +=
new RunWorkerCompletedEventHandler(WorkerCompleted);
worker.RunWorkerAsync();
}
private static void DoWork(object sender, DoWorkEventArgs e)
{
GetEmailAlertData objGetEmailAlertData = new GetEmailAlertData();
List<Email> lstEmaildetails = objGetEmailAlertData.GetEmailAlertDetails();
SentinelAlerter objSentinelAlerter = new SentinelAlerter();
foreach (Email objEmail in lstEmaildetails)
{
try
{
// I have implement first task
objAlerter.SendAlert(objEmail);
}
catch (Exception ex)
{
}
finally
{
}
}
}
Can anyone suggest how to implement the second task in the same background worker process.
I would recommend you to use Task Scheduler, its easy and will sort your issues out.

Resources