iBatis - select environment using XML - environment

I have this configuration in ibatis-config.xml
<configuration>
<properties resource="collector.properties"/>
<environments default="development">
<environment id="development">
<transactionManager type="JDBC" />
<dataSource type="POOLED">
<property name="driver" value="${dev.jdbc.driver}" />
<property name="url" value="${dev.jdbc.url}" />
</dataSource>
</environment>
<environment id="test">
<transactionManager type="JDBC" />
<dataSource type="POOLED">
<property name="driver" value="${test.jdbc.driver}" />
<property name="url" value="${test.jdbc.url}" />
</dataSource>
</environment>
</environments>
<mappers>
</mappers>
</configuration>
As shown it will load datasource from <environment id="development">
QUESTION: Is it possible at run time switch to use <environment id="test"> without modifying XML? For example - I have a test file where I'm using SqlSessionFactory and want to set it programmatically to use test environment?

SqlSessionFactoryBuilder.build() method can select a specific environment in XML.
For example,
private Reader reader;
private SqlSessionFactory sqlSessionFactorys;
private SqlSession session;
reader = Resources.getResourceAsReader("ibatis-config.xml");
sqlSessionFactorys = new SqlSessionFactoryBuilder().build(reader, "test");
testSession = sqlSessionFactorys.openSession(); // test env
sqlSessionFactorys = new SqlSessionFactoryBuilder().build(reader, "development");
devSession = sqlSessionFactorys.openSession(); // dev env

According to this site:
http://codenav.org/code.html?project=/org/mybatis/mybatis/3.2.5&path=/Source%20Packages/org.apache.ibatis.session/SqlSessionFactoryBuilder.java
The build() method closes the reader/inputstream before returning SqlSessionFactory now. So you will need to open a new reader/stream in order to load the second session. I discovered this when I separated out my account/security tables to a separate database from the main application DB. My first go around I kept getting errors when the bean was trying to load the session factory due to an input stream error (closed).
e.g.
try {
inputStream = Resources.getResourceAsStream(MYBATIS_CONFIG_PATH);
prodDbSqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream, prodDbEnvironment);
inputStream = Resources.getResourceAsStream(MYBATIS_CONFIG_PATH);
securityDbSqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream, securityDbEnvironment);
} catch (IOException ex) {
String msg = "Unable to get SqlSessionFactory";
CustomizedLogger.LOG(Level.SEVERE, this.getClass().getCanonicalName(), "methodName", msg, ex);
}
Although I put them in separate try catch blocks so that I know which one failed specifically right away in the log file.
I also implement this as a singleton so that it only has to load load resources once.
Context: I run this in a Java EE container and use MyBatis for straight forward queries and for where I would use native queries since it is a much simpler and straight forward framework. I might switch to using it over JPA everywhere, but that is still up for debate.

Related

NLog extensibility - How to add custom field using ExtendValues?

I try to add some custom fields to NLog using extensibility.
Part of my nlog.config file looks like that : (simplified for exhibit)
<nlog>
<extensions>
<add assembly="Logzio.DotNet.NLog"/>
</extensions>
<variable name="currentUser" value="test" />
<targets async="true">
<target name="logzio" type="Logzio" token="myToken">
<contextproperty name="currentUser" layout="${currentUser}" />
</target>
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="logzio" />
</rules>
</nlog>
In every controller, I have something like that (I'm using ASP.NET MVC5)
private static Logger logger = LogManager.GetCurrentClassLogger();
Then I send my logs to logzio using
logger.Fatal("Something bad happens");
Right now, currentUser always have the value test, which is logical.
However, despite of the documentation, I don't understand how to dynamically change currentUser value by the ID of my current logged user.
Should I create a sort of factory ? (if yes, how ? I'm not at ease with factories)
Should I change my logger variable ? If so, how ?
A piece of code would be extremly welcome.
Thank you for pointing my out where I'm wrong
EDIT
After #Rolf's answer, I've created this custom layout renderer
[LayoutRenderer("custom-layout")]
public class CustomLayoutRenderer : LayoutRenderer
{
public string IdUser { get; set; }
protected override void Append(StringBuilder builder, LogEventInfo logEvent)
{
logEvent.Properties.Add("currentUser", "HowToPassCustomDataHere?");
builder.Append("test from custom-layout");
}
}
and I changed the nlog.config accordingly, adding
layout="${message} ${custom-layout}"
to my <target>
However, I still don't understand how to pass custom value to currentUser. In logz.io, I have "HowToPassCustomDataHere?" as a value of currentUser.
(BTW, ${aspnet-user-identity} is great and works fine ; however I'd like to understand how to pass a custom value to my layout renderer. In my case, something like ${aspnet-user-id})
You can try one of these NLog layoutrenderers to acquire the current username:
${aspnet-user-identity} Wiki
${windows-identity} Wiki
You can also create your own custom NLog LayoutRenderer: https://github.com/NLog/NLog/wiki/How-to-write-a-custom-layout-renderer
Example of how to provide it as currentUser:
<target name="logzio" type="Logzio" token="myToken">
<contextproperty name="currentUser" layout="${aspnet-user-identity}" />
</target>

Log4Net can't locate logfile anywhere

I have followed many different guides on how to configure the log4net, it is up and running but i can't find a log file anywhere ...
This is how my configuration look like:
Web.Config
<section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
<log4net debug="true">
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<file value="C:\\temp\\Log.txt" />
<appendToFile value="true" />
<rollingStyle value="Size" />
<maxSizeRollBackups value="10" />
<maximumFileSize value="10MB" />
<staticLogFileName value="true" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%-5p %d %5rms %-22.22c{1} %-18.18M - %m%n" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="RollingLogFileAppender" />
</root>
</log4net>
Global.asax:
XmlConfigurator.Configure(new FileInfo(Server.MapPath("~/Web.config")));
//XmlConfigurator.Configure();
StartUp.cs
//[assembly: XmlConfigurator(ConfigFile = "Web.config", Watch = true)]
[assembly: XmlConfigurator(Watch = true)]
Declaration
readonly log4net.ILog logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
Logging
BasicConfigurator.Configure();
logger.Info("Info logging ...");
logger.Error("Homepage loading test logging ...");
Where my file value is: <file value="C:\\temp\\Log.txt" />
I have tried several paths, and commented out what above but no success.
What am i doing wrong?
UPDATE:
As suggested by John H i have tried configuring and calling the logger in the Application_Start method and tried several alternative configs with it but with no luck. Here are 2 screenshots of some debugging info:
Main properties:
Below are the Logger properties:
What am i doing wrong?
OK so i got it to work following this tutorial: log4net-guide-dotnet-logging
I have created a log4net.config file with content as showed in tutorial.
used [assembly: XmlConfigurator(ConfigFile = "log4net.config", Watch = true)]
Called it like this:
ILog logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);
logger.Info("Application started.");
file is created and content logged as well.
I am gonna compare the config files content and see if the difference is in there and go gradually comparing everything till i have found what caused it not to work.
Thank you for helping me!
Kind regards
From your screenshots, we can see that your logger is not being initialised with your configuration, because IsDebug is false. One thing I notice from your screenshot, is you're trying to pass the path to Web.config directly to the Configure() method. I realise that may be an attempt to solve the problem, so you may have already tried my next suggestion, but calling Configure() in the manner you currently have won't work because Web.config is not published to your bin\debug folder. It will called Web.projectname.config. Calling
XmlConfigurator.Configure()
with no parameters, will automatically resolve the correct configuration file in your output directory. I'm guessing you've tried that, but if that still doesn't work, try this as well:
using log4net;
protected void Application_Start(object sender, EventArgs e)
{
// Initialising configuration before requesting a logger.
XmlConfigurator.Configure();
// Requesting a logger only after the configuration has been initialised.
var logger = LogManager.GetLogger(typeof(Global));
logger.Info("Application started.");
}
I'm not sure it will make any difference, but your configuration looks fine to me.
But by inspecting the IsDebug property on the logger, you'll at least be able to tell if the configuration has even been read.
Edit: One other thing, make sure the application will have the permissions to write to the file. From the documentation:
The RollingFileAppender extends the FileAppender and has the same behavior when opening the log file. The appender will first try to open the file for writing when ActivateOptions() is called. This will typically be during configuration. If the file cannot be opened for writing the appender will attempt to open the file again each time a message is logged to the appender. If the file cannot be opened for writing when a message is logged then the message will be discarded by this appender.

'Job Exists' error when using AdoJobStore

"Couldn't store job: Unable to store Job: 'QbBackupGroup.QbBackup', because one already exists with this identification."
I'm getting this anytime after the first run on a clean database (SQLCE). I've searched and found a few suggestions (i.e. here and here), but they pertain to running Quartz.NET without TopShelf.
The suggested code is based on standard Quartz.NET architecture (e.g. variables and values), but the Quartz.NET code for TopShelf is all delegates. Nothing seems to directly translate.
My code is below.
However... all that said... it may not matter, as my eventual goal is to be able to add/remove jobs/triggers at runtime without restarting the service. This hard-coded design may not even apply, but I haven't figured out how to do the runtime bit yet.
If I can get past this, the runtime bit is next. Unless the runtime bit negates the need for this. Catch-21.
Please advise.
Service:
Sub Main()
Dim oSchedule As Action(Of SimpleScheduleBuilder)
Dim oTrigger As Func(Of ITrigger)
Dim oDetail As Func(Of IJobDetail)
Dim oJob As Action(Of QuartzConfigurator)
oSchedule = Function(ScheduleBuilder) As SimpleScheduleBuilder
Return ScheduleBuilder.WithIntervalInSeconds(5).RepeatForever
End Function
oTrigger = Function() As ITrigger
Return TriggerBuilder.Create.WithIdentity(QbBackup.Job.Trigger, QbBackup.Job.Group).WithSimpleSchedule(oSchedule).Build
End Function
oDetail = Function()
Return JobBuilder.Create(Of QbBackup.Job).WithIdentity(QbBackup.Job.Name, QbBackup.Job.Group).Build
End Function
oJob = Function(Configurator As QuartzConfigurator)
Return Configurator.WithJob(oDetail).AddTrigger(oTrigger)
End Function
HostFactory.Run(Sub(Configurator)
Configurator.Service(Of Manager)(Sub(Service)
Service.ConstructUsing(Function(Factory) As ServiceControl
Return New Manager
End Function)
Service.WhenStarted(Function(Notifier, HostControl) As Boolean
Return Notifier.StartService(HostControl)
End Function)
Service.WhenStopped(Function(Notifier, HostControl) As Boolean
Return Notifier.StopService(HostControl)
End Function)
Service.ScheduleQuartzJob(oJob)
End Sub)
Configurator.SetDescription(SchedulerInfo.Description)
Configurator.SetServiceName(SchedulerInfo.Product)
Configurator.SetDisplayName(SchedulerInfo.Title)
Configurator.StartAutomatically()
Configurator.RunAsLocalSystem()
End Sub)
End Sub
Job:
Imports Common.Logging
Public Class Job
Implements IJob
Private Shared Logger As ILog = LogManager.GetLogger(GetType(Job))
Public Sub Execute(Context As IJobExecutionContext) Implements IJob.Execute
Try
Job.Logger.Info(Now.ToString)
Catch ex As Exception
Throw New JobExecutionException(ex.Message, ex)
End Try
End Sub
Public Shared ReadOnly Property Name As String
Get
Return QbBackupInfo.Product
End Get
End Property
Public Shared ReadOnly Property Trigger As String
Get
Return "{0}Trigger".ToFormat(Job.Name)
End Get
End Property
Public Shared ReadOnly Property Group As String
Get
Return "{0}Group".ToFormat(Job.Name)
End Get
End Property
End Class
App.config:
<quartz>
<!-- Configure Scheduler -->
<add key="quartz.scheduler.instanceName" value="Scheduler" />
<add key="quartz.scheduler.instanceId" value="Scheduler" />
<!-- Configure Thread Pool -->
<add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
<add key="quartz.threadPool.threadCount" value="10" />
<add key="quartz.threadPool.threadPriority" value="Normal" />
<!-- Configure Job Store -->
<add key="quartz.jobStore.misfireThreshold" value="60000" />
<add key="quartz.jobStore.type" value="Quartz.Impl.AdoJobStore.JobStoreTX, Quartz" />
<add key="quartz.jobStore.useProperties" value="true" />
<add key="quartz.jobStore.dataSource" value="default" />
<add key="quartz.jobStore.tablePrefix" value="QRTZ_" />
<add key="quartz.jobStore.lockHandler.type" value="Quartz.Impl.AdoJobStore.UpdateLockRowSemaphore, Quartz" />
<!-- Configure Data Source -->
<add key="quartz.dataSource.default.connectionString" value="Data Source=C:\ProgramData\Scheduler\Scheduler.sdf;Max Database Size=4091;Persist Security Info=False;" />
<add key="quartz.dataSource.default.provider" value="SqlServerCe-400" />
</quartz>
The combination of a job's group and its name are the job's unique key in Quartz.Net. They should be unique across all jobs. The same happens for triggers. The fix is to change the job's name for each job if you really need to have multiple jobs of the same type running. You can also just have one job with multiple triggers instead. Each trigger has its own data map so you could add custom information there. One way to get around this is to append a timestamp the job's name so that they are all uniquely named.

Create job in code that persists the "quartz_jobs.xml" file

I have a quartz_jobs.xml file that has jobs defined in it.
I can load the quartz_jobs.xml file configuration and jobs start firing.
Aka, "reading" the jobs from the quartz_jobs.xml file works fine.
However, if I manually add a job to the IScheduler, this manually added job will start running......(along with jobs defined in the quartz_jobs.xml file)....
BUT the job is not ~written to the xml.
Is there a way to add a job to the IScheduler and have it write-back to the quartz_jobs.xml file?
Note, when I wired up the exact same code to a AdoStore, and I call the IScheduler.Start(), the jobs do get added to the database tables (aka, they persist).
But the same code wired to a RamStore running against Xml does not "save" the jobs to the quartz_jobs.xml file.
Thanks.
Here is my quartz.config (not the jobs) file.
<quartz>
<add key="quartz.plugin.jobInitializer.type" value="Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin" />
<add key="quartz.scheduler.instanceName" value="DefaultQuartzScheduler" />
<add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
<add key="quartz.threadPool.threadCount" value="10" />
<add key="quartz.threadPool.threadPriority" value="2" />
<add key="quartz.jobStore.misfireThreshold" value="60000" />
<add key="quartz.jobStore.type" value="Quartz.Simpl.RAMJobStore, Quartz" />
<add key="quartz.plugin.jobInitializer.fileNames" value="quartz_jobs.xml" />
<add key="quartz.plugin.jobInitializer.failOnFileNotFound" value="true" />
<add key="quartz.plugin.jobInitializer.scanInterval" value="120" />
</quartz>
http://quartznet.sourceforge.net/apidoc/2.0/html/html/2909678f-44c6-6e13-afa5-c50e7b5ee435.htm
XMLSchedulingDataProcessorPlugin Class Quartz.NET API Documentation
This plugin loads XML file(s) to add jobs and schedule them with triggers as the scheduler is initialized, and can optionally periodically scan the file for changes.
I don't think you can.
XMLSchedulingDataProcessor doesn't have any methods to serialize your jobs back to the file and QuartzXmlConfiguration20 seems to have only read-only collections.
I've tried to do some experiments.
Apparently you can manage to whole process of loading and processing the xml file:
ITypeLoadHelper loadHelper = new SimpleTypeLoadHelper();
loadHelper.Initialize();
XMLSchedulingDataProcessor processor = new XMLSchedulingDataProcessor(loadHelper);
processor.OverWriteExistingData = true;
processor.ProcessFileAndScheduleJobs("my_jobs.xml", Scheduler);
(so you do not have to use any configuration in your config file)
but, again, there's no way to append extra elements.
Another way could be to deserialize the xml file and try to manipulate it:
string xml = string.Empty;
using (var xmlJobFile = new System.IO.StreamReader("my_jobs.xml"))
{
xml = xmlJobFile.ReadToEnd();
}
XmlSerializer xs = new XmlSerializer(typeof(QuartzXmlConfiguration20));
QuartzXmlConfiguration20 data = (QuartzXmlConfiguration20)xs.Deserialize(new StringReader(xml));
if (data == null)
{
throw new SchedulerConfigException("Job definition data from XML was null after deserialization");
}
But it gets too complicated.
I reckon that the best option is to use AdoJobStore.

RAMJobStore (quartz_jobs.xml) to AdoJobStore Data Move

My team and I are trying to figure out a way to "load up" our Sql Server database with the Quartz.NET schema installed.
<add key="quartz.dataSource.default.provider" value="SqlServer-20"/>
For demo's, we've been storing our job-setups in .xml (quartz_jobs.xml).
My question is :
Is there a way to "load up" the scheduling data from .xml (quartz_jobs.xml) (Quartz.Simpl.RAMJobStore), and then "save it off" to a AdoJobStore (Quartz.Impl.AdoJobStore.JobStoreTX) ?
The reason is that our "start up" data could be easily written placed in the .xml.
Right now, the only way I see putting jobs into a AdoJobStore is "coding them up" in c# code through the Quartz.Net object model.
Or "playing back" some profiled TSQL (using Sql Profiler) :(
Direct question is above "(getting xml into sql-server)".....the higher level question is "How does one populate a AdoJobStore with start up data...that isn't "coding them up" in c# code.
EDIT:
I'm putting in my code that works......using Marko's (accepted as the answer) response.
My configuration file:
<quartz>
<add key="quartz.plugin.xml.type" value="Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin, Quartz" />
<add key="quartz.plugin.xml.fileNames" value="~/Quartz_Jobs_001.xml" />
<add key="quartz.plugin.xml.ScanInterval" value="10" />
<add key="quartz.jobStore.type" value="Quartz.Impl.AdoJobStore.JobStoreTX, Quartz" />
<add key="quartz.jobStore.driverDelegateType" value="Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz"/>
<add key="quartz.jobStore.dataSource" value="default"/>
<add key="quartz.dataSource.default.connectionString" value="Server=MyServer\MyInstance;Database=QuartzDB;Trusted_Connection=True;Application Name='quartz_config';"/>
<add key="quartz.dataSource.default.provider" value="SqlServer-20"/>
</quartz>
My code:
NameValueCollection config = (NameValueCollection)ConfigurationManager.GetSection("quartz");
ISchedulerFactory factory = new StdSchedulerFactory(config);
IScheduler sched = factory.GetScheduler();
sched.Clear();
sched.Start();
NOTE:
I had to call IScheduler.Start() for the values to persist to the database.
The consequence of adding this line:
<add key="quartz.plugin.xml.ScanInterval" value="10" />
was that I could add entries into the quartz_job.xml, and it would would (append-only) the data in the database (while the engine was running).
Aka, I can "add lookup data" (to the database) "on the fly"....without stopping the service. A nice little tidbit.
Removing a job requires a restart.
You should be able to do this quite easily. You can combine the XML configuration and ADO job store. This will make the XML processor update the jobs in the persistent store.
Here's minimal configuration:
NameValueCollection properties = new NameValueCollection();
properties["quartz.jobStore.type"] = "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz";
properties["quartz.jobStore.dataSource"] = "default";
properties["quartz.jobStore.driverDelegateType"] = "Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz";
properties["quartz.dataSource.default.connectionString"] = "Server=(local);Database=quartz;Trusted_Connection=True;";
properties["quartz.dataSource.default.provider"] = "SqlServer-20";
// job initialization plugin handles our xml reading, without it defaults are used
properties["quartz.plugin.xml.type"] = "Quartz.Plugin.Xml.XMLSchedulingDataProcessorPlugin, Quartz";
properties["quartz.plugin.xml.fileNames"] = "~/quartz_jobs.xml";
// First we must get a reference to a scheduler
ISchedulerFactory sf = new StdSchedulerFactory(properties);
IScheduler sched = sf.GetScheduler();
And our XML configuration contains overwrite instructions so that job store is refreshed:
<?xml version="1.0" encoding="UTF-8"?>
<job-scheduling-data xmlns="http://quartznet.sourceforge.net/JobSchedulingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
version="2.0">
<processing-directives>
<overwrite-existing-data>true</overwrite-existing-data>
</processing-directives>
<schedule>
<job>
<name>jobName1</name>
<group>jobGroup1</group>
<description>jobDesciption1</description>
<job-type>Quartz.Examples.Example15.SimpleJob, Quartz.Examples</job-type>
<durable>true</durable>
<recover>false</recover>
</job>
<trigger>
<simple>
<name>simpleName</name>
<group>simpleGroup</group>
<description>SimpleTriggerDescription</description>
<job-name>jobName1</job-name>
<job-group>jobGroup1</job-group>
<start-time>1982-06-28T18:15:00.0Z</start-time>
<repeat-count>-1</repeat-count>
<repeat-interval>3000</repeat-interval>
</simple>
</trigger>
</schedule>
</job-scheduling-data>
You can also make the XML to auto refresh the store on change (checked every 10 seconds) if you define:
properties["quartz.plugin.xml.ScanInterval"] = "10";

Resources