I have a question about Quartz and running threads inside Service class.
I got my previous question answered: Grails background process, however I have another issue.
Setup: I have a Job that is setup to run a Service and it works perfectly. However inside a Service class I have an algorithm that can run in parallel.
Issue: Typically I would setup code to run in parallel in the following very simple way:
Item.each {
Thread.start {
do some calculations here
write to DB
}
}
However, since my code need to write into DB and I need to leverage domain classes and at that point my code brakes. Hibernate complains that threads don't have access to something.
I am not sure why I can't use threads inside Service class and leverage domain class. Can someone help me with this dilemma?
Do I need to create threads in a special way? May be I shouldn't be creating threads in Service class (since Service class seem to be running inside threads )? Do I need to move my code into Job class?
Please help.
Thank you.
The new Threads won't have a Hibernate Session bound to them by default. To attach a Hibernate Session, try the following:
Item.each {
Thread.start {
Item.withTransaction {
do some calculations here
write to DB
}
}
}
You could also look into GPars for an easy to use parallelization framework.
Related
We are using apache beam and would like to setup the logback MDC. logback MDC is a great GREAT resource when you have a request come in and you store let's say a userId (in our case, it's custId, fileId, requestId), then anytime a developer logs, it magically stamps that information on to the developers log. the developer no longer forgets to add it every log statement he adds.
I am starting in an end to end integration type test with apache beam direct runner embedded in our microservice for testing (in production, the microservice calls dataflow). currently, I am see that the MDC is good up until after the expand() methods are called. Once the processElement methods are called, the context is of course gone since I am in another thread.
So, trying to fix this piece first. Where should I put this context such that I can restore it at the beginning of this thread.
As an example, if I have an Executor.execute(runnable), then I simply transfer context using that runnable like so
public class MDCContextRunnable implements Runnable {
private final Map<String, String> mdcSnapshot;
private Runnable runnable;
public MDCContextRunnable(Runnable runnable) {
this.runnable = runnable;
mdcSnapshot = MDC.getCopyOfContextMap();
}
#Override
public void run() {
try {
MDC.setContextMap(mdcSnapshot);
runnable.run();
} Catch {
//Must log errors before mdc is cleared
log.error("message", e);. /// Logs error and MDC
} finally {
MDC.clear();
}
}
}
so I need to do the same with apache beam basically. I need to
Have a point to capture the MDC
Have a point to restore the MDC
Have a point to clear out the MDC to prevent it leaking to another request(really in case I missed something which seems to happen now and then)
Any ideas on how to do this?
oh, bonus points if it the MDC can be there when any exceptions are logged by the framework!!!! (ie. ideally, frameworks are supposed to do this for you but apache beam seems like it is not doing this. Most web frameworks have this built in).
thanks,
Dean
Based on the context and examples you gave, it sounds like you want to use MDC to automatically capture more information for your own DoFns. Your best bet for this is, depending on the lifetime you need your context available for, to use either the StartBundle/FinishBundle or Setup/Teardown methods on your DoFns to create your MDC context (see this answer for an explanation of the differences between the two). The important thing is that these methods are executed for each instance of a DoFn, meaning they will be called on the new threads created to execute these DoFns.
Under the Hood
I should explain what's happening here and how this approach differs from your original goal. The way Apache Beam executes is that your written pipeline executes on your own machine and performs pipeline construction (which is where all the expand calls are occurring). However, once a pipeline is constructed, it is sent to a runner which is often executing on a separate application unless it's the Direct Runner, and then the runner either directly executes your user code or runs it in a docker environment.
In your original approach it makes sense that you would successfully apply MDC to all logs until execution begins, because execution might not only be occurring in a different thread, but potentially also a different application or machine. However, the methods described above are executed as part of your user code, so setting up your MDC there will allow it to function on whatever thread/application/machine is executing transforms.
Just keep in mind that those methods get called for every DoFn and you will often have mutiple DoFns per thread, which is something you may need to be wary of depending on how MDC works.
I was trying to implement a grails SearchService that indexes certain text and stores it in memory for faster lookup. In order to store this data, I was trying to use a private static property in the Service to store the data, but the property was randomly resetting values. After rereading documentation, I realized that this is likely because grails services are supposed to be stateless since the employee the singleton pattern. Still, not sure I understand how a static variable can vary. Does the JVM load separate copies of service classes per thread? Not sure I'm wrapping my head around what's happening.
Nonetheless, now that I know I can't rely on static variables to store application-wide data, what's the best approach to store and access data for use across the application, while keeping synchronization and avoiding races?
Caused by: java.lang.IllegalStateException: Method on class [TEXTSTORE] was used outside of a Grails application. If running in the context of a test using the mocking API or bootstrap Grails correctly.
at SearchService.buildIndex(SearchService.groovy:63)
at SearchService$_indexAllDomains_closure2.doCall(SearchService.groovy:42)
at SearchService.indexAllDomains(SearchService.groovy:41)
at SearchService.$tt__rebuildIndex(SearchService.groovy:48)
at SearchService.afterPropertiesSet(SearchService.groovy:35)
... 4 more
You seem to be a bit confused about services in Grails. There is no reason why a service (defaulting to singleton) can't have shared state. It's not uncommon for a service to populate some cached or indexed data when it is created so it can be used by multiple callers.
Most often this is done by implementing the org.springframework.beans.factory.InitializingBean interface and making use of the afterPropertiesSet() method which is called when the service (Spring bean) has been created in the application context and all dependencies have been resolved.
For example:
package com.example
import org.springframework.beans.factory.InitializingBean
class MyExampleService implements InitializingBean {
private List data
def otherService
void afterPropertiesSet() {
data = otherService.doSomethingToFetchData()
}
// .. other stuff
}
By hooking into the lifecycle of the bean you can be fairly sure that even in development (when your service reloads because you've changed some code) it will still have the data needed.
I need to to read data from database to be read into Config.groovy.
Is that possible to get data from Database into Config.groovy?
No it is not possible. In the sequence of events when a Grails application is starting up the Config.groovy is processed before the Datasource is made available to the application.
Without knowing what you are trying to accomplish I can't make a suggestion on how else to approach this issue.
Updated (based on comment)
In your comment you explain that you are trying to use the feature switch plugin (which is designed to be run time and not persistent). Looking at the source code for the plugin you should be able to make your own service that will load the settings from the database and toggles/updates the feature switch settings. Here is just a simple sketch/example:
package example
import org.springframework.beans.factory.InitializingBean
class MyExampleService implements InitializingBean {
def grailsApplication
void afterPropertiesSet() {
// here is where you would do whatever you needed to load the settings
grailsApplication.config.features['somefeature'].enabled = true
grailsApplication.config.features['otherfeature'].enabled = false
}
}
That should at least give you an idea.
Alternatively you could just write it all in Bootstrap.groovy which can access the datasource/GORM as well.
I have an application using the Entity Framework code first. My setup is that I have a core service which all other services inherit from. The core service contains the following code:
public static DatabaseContext db = new DatabaseContext();
public CoreService()
{
db.Database.Initialize(force: false);
}
Then, another class will inherit from CoreService and when it needs to query the database will just run some code such as:
db.Products.Where(blah => blah.IsEnabled);
However, I seem to be getting conflicting stories as to which is best.
Some people advise NOT to do what I'm doing.
Other people say that you should define the context for each class (rather than use a base class to instantiate it)
Others say that for EVERY database call, I should wrap it in a using block. I've never seen this in any of the examples from Microsoft.
Can anyone clarify?
I'm currently at a point where refactoring is possible and quite quick, so I'd like some general advice if possible.
You should wrap one context per web request. Hold it open for as long as you need it, then get rid of it when you are finished. That's what the using is for.
Do NOT wrap up your context in a Singleton. That is not a good idea.
If you are working with clients like WinForms then I think you would wrap the context around each form but that's not my area.
Also, make sure you know when you are going to be actually executing against your datasource so you don't end up enumerating multiple times when you might only need to do so once to work with the results.
Lastly, you have seen this practice from MS as lots of the ADO stuff supports being wrapped in a using but hardly anyone realises this.
I suggest to use design principle "prefer composition over inheritance".
You can have the reference of the database context in your base class.
Implement a singleton for getting the DataContext and assign the datacontext to this reference.
The conflicts you get are not related to sharing the context between classes but are caused by the static declaration of your context. If you make the context an instance field of your service class, so that every service instance gets its own context, there should be no issues.
The using pattern you mention is not required but instead you should make sure that context.Dispose() is called at the service disposal.
I'm using the resources.groovy to declare a service e.g.
aService(com.foo.OrganizationService)
so that I can tie aService to my controllers instead of using organizationService which could change in the future.
I've noticed that the OrganizationService doesn't get treated special like other services "not" declared in the resources.groovy. For example it doesn't get injected with grailsApplication, and likely a hibernateSession etc and other things I've not hit yet....
Now, I know I can manually wire in stuff to my service but I'd rather not have to maintain that...
Is there a special way to declare a service in the resources.groovy so that gets treated like another service that grails loads up?
TIA
The short answer to your question is "no".
Under the covers, Grails services are driven by some intelligent code that is referencing a specific location and expecting certain properties.
Viewing the source code (especially around the ServicesGrailsPlugin.groovy) is a good way to see the "magic" in how these are wired together.
Is there a reason you wouldn't want to use a bonafide Grails service to solve your problem? If you are expecting things like a grailsApplication, it seems like that use is pretty specific to Grails and would be a good candidate for porting over to a "true" Grails service.
Good luck!
So I've come full circle on this. This is a timing problem. Where services haven't been grails initialized yet.
Basically when you use the resources.groovy to do service wiring you run the risk of using a Service that might initialize itself e.g. afterPropertiesSet method or static initializers that use grails services (log, hibernate session, ..) that haven't been injected yet.
So... What I've turned to instead is to create my own BeanBuilder in a BootStrap.groovy file.
BeanBuilder builder = new BeanBuilder(grailsApplication.parentContext)
def bb = builder.beans {
LoginListener(com.foo.LoginListener) {
springSecurityService = ref("springSecurityService")
userService = ref("userService")
}
}
bb.registerBeans(grailsApplication.mainContext)