I run some R commands with R.NET from a C# trading simulation application. When I run a first simulation everything works fine. When I runa second simulation, it hangs. The simulation app runs each simulation in a new app domain. As I understand it, R.NET is run in only one app domain. Is there a way ro fix this, as even the dispose method does not allow another initialization of the REngine object.
Do not use
using (engine = REngine.GetInstance())
{
...
}
because this would finally dispose the REngine instance and cause
the known RDotNet issue
"The single REngine instance has already been disposed of (i.e. shut down). Multiple engine restart is not possible."
Use instead:
var engine = REngine.GetInstance();
...
and do not dispose it!
Related
We are using apache beam and would like to setup the logback MDC. logback MDC is a great GREAT resource when you have a request come in and you store let's say a userId (in our case, it's custId, fileId, requestId), then anytime a developer logs, it magically stamps that information on to the developers log. the developer no longer forgets to add it every log statement he adds.
I am starting in an end to end integration type test with apache beam direct runner embedded in our microservice for testing (in production, the microservice calls dataflow). currently, I am see that the MDC is good up until after the expand() methods are called. Once the processElement methods are called, the context is of course gone since I am in another thread.
So, trying to fix this piece first. Where should I put this context such that I can restore it at the beginning of this thread.
As an example, if I have an Executor.execute(runnable), then I simply transfer context using that runnable like so
public class MDCContextRunnable implements Runnable {
private final Map<String, String> mdcSnapshot;
private Runnable runnable;
public MDCContextRunnable(Runnable runnable) {
this.runnable = runnable;
mdcSnapshot = MDC.getCopyOfContextMap();
}
#Override
public void run() {
try {
MDC.setContextMap(mdcSnapshot);
runnable.run();
} Catch {
//Must log errors before mdc is cleared
log.error("message", e);. /// Logs error and MDC
} finally {
MDC.clear();
}
}
}
so I need to do the same with apache beam basically. I need to
Have a point to capture the MDC
Have a point to restore the MDC
Have a point to clear out the MDC to prevent it leaking to another request(really in case I missed something which seems to happen now and then)
Any ideas on how to do this?
oh, bonus points if it the MDC can be there when any exceptions are logged by the framework!!!! (ie. ideally, frameworks are supposed to do this for you but apache beam seems like it is not doing this. Most web frameworks have this built in).
thanks,
Dean
Based on the context and examples you gave, it sounds like you want to use MDC to automatically capture more information for your own DoFns. Your best bet for this is, depending on the lifetime you need your context available for, to use either the StartBundle/FinishBundle or Setup/Teardown methods on your DoFns to create your MDC context (see this answer for an explanation of the differences between the two). The important thing is that these methods are executed for each instance of a DoFn, meaning they will be called on the new threads created to execute these DoFns.
Under the Hood
I should explain what's happening here and how this approach differs from your original goal. The way Apache Beam executes is that your written pipeline executes on your own machine and performs pipeline construction (which is where all the expand calls are occurring). However, once a pipeline is constructed, it is sent to a runner which is often executing on a separate application unless it's the Direct Runner, and then the runner either directly executes your user code or runs it in a docker environment.
In your original approach it makes sense that you would successfully apply MDC to all logs until execution begins, because execution might not only be occurring in a different thread, but potentially also a different application or machine. However, the methods described above are executed as part of your user code, so setting up your MDC there will allow it to function on whatever thread/application/machine is executing transforms.
Just keep in mind that those methods get called for every DoFn and you will often have mutiple DoFns per thread, which is something you may need to be wary of depending on how MDC works.
I'm using a "nested container per request" pattern. It's wrapped in a using block so the nested container is disposed after handling the request. This causes all instances created by the nested container to also be disposed at the end of the request.
This works great and is usually exactly what I want to happen. However I have ran into a use case where I don't want one particular instance to be disposed while everything else is.
I tried explicitly removing the instance at the bottom of the using block like so:
using (var nestedContainer = _container.GetNestedContainer())
{
nestedContainer.Configure ( x =>
x.For<IFoo>()
.Use(getFooForRequest())
);
var handler = (IRequestHandler) nestedContainer.GetInstance(handlerType);
handler.execute(....)
nestedContainer.EjectAllInstancesOf<IFoo>();
}
Unfortunately it appears that EjectAllInstancesOf is also calling dispose.
My application has a few different instances of IFoo, and they all need to live for the entire lifetime of the application. However incoming requests need to be dynamically associated with one particular instance of IFoo.
Injecting it into the nested container like above accomplished this goal, but my IFoo is getting disposed with the nested container, which is bad.
So how do I block the dispose for IFoo while still letting it dispose everything else?
If that's not possible, is there some other way to dynamically select my per-request IFoo without manual injection via nested.Configure()?
I have a question about Quartz and running threads inside Service class.
I got my previous question answered: Grails background process, however I have another issue.
Setup: I have a Job that is setup to run a Service and it works perfectly. However inside a Service class I have an algorithm that can run in parallel.
Issue: Typically I would setup code to run in parallel in the following very simple way:
Item.each {
Thread.start {
do some calculations here
write to DB
}
}
However, since my code need to write into DB and I need to leverage domain classes and at that point my code brakes. Hibernate complains that threads don't have access to something.
I am not sure why I can't use threads inside Service class and leverage domain class. Can someone help me with this dilemma?
Do I need to create threads in a special way? May be I shouldn't be creating threads in Service class (since Service class seem to be running inside threads )? Do I need to move my code into Job class?
Please help.
Thank you.
The new Threads won't have a Hibernate Session bound to them by default. To attach a Hibernate Session, try the following:
Item.each {
Thread.start {
Item.withTransaction {
do some calculations here
write to DB
}
}
}
You could also look into GPars for an easy to use parallelization framework.
My question is not clear at title [i can not write it exactly]
e.g Texture2D picture = Content.Load<Texture2D>("myPicture");
what does happen on memory if the code above runs ? As I know Content caches the "myPicture" to the memory and return a reference to the Texture2D picture. Am I wrong ? If "myPicture" is loaded to another Texture2D object "myPicture" is not duplicated so it returns only a reference.
Is each file (or content-file) loaded over Content cached to memory (also allocated on Ram) without duplicating ? (i believe this my question with all written above should be checked)
Thanks !
Each instance of ContentManager will only load any given resource once. The second time you ask for a resource, it will return the same instance that it returned last time.
ReferenceEquals(Content.Load<Texture2D>("something"),
Content.Load<Texture2D>("something")) == true
To do this, ContentManager maintains a list of all the content it has loaded internally. This list prevents the garbage collector from cleaning up those resources - even if you are not using them.
To unload the resources and clear that internal list, call ContentManager.Unload. This will free up the memory the loaded resources were using. Now if you ask for the same resource again - it will be re-loaded.
Of course, if you are using those resources when you call Unload, all of those shared instances that you loaded will be disposed and unusable.
Finally, don't call Dispose on anything that comes out of ContentManager.Load, as this will break all the instances that are being shared and cause problems when ContentManager tries to dispose of them in Unload later on.
There's this software, X, which has this really complicated API that I have to write a facade for. I wrote a class library, XClientLibrary, and I made it using DI and IoC container (Unity). This was possible because my library exports services (interfaces) so users are not aware of the concrete classes which use constructor DI. They're also unaware of the IoC container.
The "root service" is a IXClient instance which is supposed to be created once and used as long as the application runs. (It is a desktop application btw). The X-client allows users to connect to X-hosts if they know the URL. A X-host allows users to access host's services and their services and so on (quite a complex object graph). This is sample user code:
// 1. app startup
XClientProvider provider = new XClientProvider(); // call only once per app
IXClient xClient = provider.GetClient(); // always returns the same instance
xClient.Startup();
// 2. app normal usage
IXHost host = xClient.ConnectToHost(new Uri("http://localhost")); // return new instance each time
IXService1 service = host.GetThis();
IXService2 otherService = service.DoThat();
...
host.Dispose();
// get another host, consume it, dispose it, etc
...
// 3. app shutdown
xClient.Shutdown();
provider.Dispose();
I tried to follow Mark Seemann's suggestions to implement this, but I'm not sure if they apply to a class library too. The client provider is the composition root, which is the only place where the IoC container is used. The composition root follows the RRR pattern:
the container is created on new XClientProvider() and configured
the container resolves IXClient when calling GetClient()
the container is disposed on provider.Dispose()
Things get complicated when the container is asked to resolve IXHost. Its implementation is:
internal class XHost : IXHost
public XHost(Uri uri, IXService1 service1)
The client is supposed to create XHost instances, so its implementation needs to know how to create IXService1:
internal class XClient : IXClient
public XClient(Func<IXService1> xService1DelegateFactory)
Invoking the delegate factory reaches the container which creates a IXService1. Also, let's say that in this graph there is a class XComponent7 which requires the exact IXService1 instance which was used to create the host:
internal class XComponent7 : IXService7
public XComponent7(Func<IXService1> service1DelegateFactory)
I have to use Func to deal with the circular dependency. The container should be configured such that once a IXService1 was resolved, it will provide the same instance whenever asked to resolve IXService1.
Now it gets really complicated. I want to restrict this behavior "per host resolve", meaning once a host is created the container should create a IXService1 and cache it and provide it to whatever component needs it, as long as the component is part of the object graph of the host. I also need a way to dispose all components when a host is disposed.
I was thinking I can do it using child containers. I can create one when users call ConnectToHost, ask it to resolve the host and dispose it on host disposal. The main container is still alive and won't be disposed until they call Dispose on the provider.
Problem is, I think it breaks the RRR pattern. So I wonder how RRR works when child container are involved... Maybe the IXHost is another "root" which can be directly resolved by the composition root? Or maybe there's a really smart Unity lifetime manager which can do what I need?
#Suiden So my understanding is: Your client is something that lets you lookup hosts (like a registry). Hosts offer services implemented by components. Every application has exactly one instance of your lookup/client. Your components not only implement services but might need other services to do their job. You want to resolve all parts of that object graph exactly once and when you dispose your client throw all of it away.
A couple of thoughts:
Circular references between dependencies (or services) is something you should try to avoid. If these services need each other that indicates they should be one service. That's what high cohesion, low coupling is about.
Unity does not clean up after itself. That means that even if you dispose a container that will not dispose the objects created by that container. The cleanup feature is on the wish list for Unity vNext
If you want to resolve an instance of some service and cache that instance inside your client/host wherever you should have a look at Lazy. It takes a Func to create an instance of T and evaluates that Func the first time the value is requested. So you can inject the Func into your classes or teach Unity to inject Lazy instances directly.
Child containers are a feature I find less than usefull. You can scope registration information and object lifetimes. But to make use of these scopes you would have to reference the appropriate child container. That means you are dropping dependency injection in favor of the ServiceLocator anti-pattern.