I want to build an application server with Dart. The httpServer in the dart:io library is certainly a good starting point for that. But I struggle with the task to "deploy" an application without restarting the server process.
To be more precise: I want to have something like a servlet container in Java, like Tomcat, into which I can easily deploy or redeploy an application without restarting the container. I thought I could utilize the mirror system, which allows me in principle to load a library and its contained classes from the filesystem. But unfortunately it seems that I cannot re-load the library. When I add for example a new class to the library, or change the coding of an existing class, a new reflection of the library without restarting the dart process, does not reflect the changes. Only when I stop the process and restart it again, the changes are visible.
So: is there a way to scrub the mirror system and let it load the library and its classes again, within the same Dart process?
I think isolates are a good fit for this requirement.
I haven't used them myself much yet but as far as I know you can load and unload them dynamically.
The documentation is not very extensive yet.
A few things I found:
https://api.dartlang.org/apidocs/channels/stable/dartdoc-viewer/dart-isolate
Recent documentation about Dart Isolates
https://www.youtube.com/watch?v=TQJ1qnrbTwk
https://www.youtube.com/watch?v=4GlK-Ln7HAc
So, yes, it is possible in Dart to dynamically (re-)load dart-files at runtime. Every new isolate has its own MirrorSystem. If you want to reload a dart-file you must create a new isolate and use the MirrorSystem of this isolate to iterate over the contents in the libraries known to this MirrorSystem. If your dart-file is part of a library known to the MirrorSystem, all functions and classes contained in this file are loaded and reflected anew.
This solution has some drawbacks: First, it is quite heavyweight. The programming of inter-isolate communication is cumbersome. Also it is to be seen whether the memory consumption increases with each reload. Second, the solution is not really dynamic: Isolates load only libraries that are "known" at design-time. They must be directly or indirectly imported into the dart file that contains the static function, which is called when the isolate is created.
Two ideas how the situation could be improved:
1. It would help if the spawn and the spawnUri methods of Isolate could get a list of additional libraries as parameter, which are included in the MirrorSystem of the isolate.
2. The classloaders in Java are independent of processes and threads. They just load classes. Why isn't this possible in Dart?
Related
I have a long-lived object in my Dart app, created using the output of several computationally expensive functions. Often, I change these expensive functions - but because their output is cached, I cannot see the results of my changes without restarting the app.
Is there a way (perhaps using a Dart VM API) to respond to a Hot Reload and recreate my object accordingly?
Note: I am looking for a solution that does not depend on the Flutter framework, as the relevant code is in a standalone Dart package.
I started using project reactor. Does anyone know how can I pass thread local variables from one thread to another? I saw some methods on Hooks.java but could not figure out what is the recommended way of doing this. Can someone point me to some documentation or with a code snippet on how to do it. Thanks.
I have a working example in this github repository based on the spring-cloud-sleuth's implementation: https://github.com/gumartinm/JavaForFun/tree/master/SpringJava/WebReactive/spring-webreactive-reactor-context-enrich
The key classes are: ContextCoreSubscriber.java, SubscriberContext.java, ThreadContextEnrichmentAutoConfiguration.java and UsernameFilter.java
ContextCoreSubscriber.java:
Enables you to fill the Mapped Diagnostic Context: MDC
SubscriberContext.java:
Helper class for inserting data in the Reactor's Context.
ThreadContextEnrichmentAutoConfiguration.java:
In charge of configuring the Reactor's Hooks: Hooks.onEachOperator
UsernameFilter.java:
Example where we want to register the username information based on some HTTP header.
Reactor doesn't guarantee that the processing done by a Flux or Mono chain of operators will stick executing on a single thread. On the contrary, it performs work-stealing and lets the user switch execution context.
As such, using ThreadLocal is not very adapted to Reactor.
There is currently some work done in 3.1.0 towards providing an equivalent, at least for library authors that use Reactor, but nothing definite in place yet.
Keep your eyes peeled for 3.1.0, that should be the main theme of that release (and will probably be the focus of the second upcoming milestone, M2).
I'm writing something that I want to release as both a chrome extension and a firefox add-on.
The chrome extension is already available on github. I've factored my code into several modules using a module load format similar to what requirejs uses; I did this to separate the parts that were chrome-specific from the parts I hoped to re-use in the firefox add-on.
Specifically, I split up not only the backend work, but also the content scripts.
In chrome, when my content script needs to load another module, it sends a message to the background page saying "please load this module"; the script on the background page then does:
function onLoadLibrary(request, sender, sendResponse) {
var allFrames = request.allFrames || false;
chrome.tabs.executeScript(
sender.tab.id, {file: request.library.toLowerCase() + '.js',
allFrames: allFrames},
function () {
sendResponse({});
});
return true;
}
That is, I'm able to load additional javascript into the same sandbox as the content script that asked for that code. This is necessary to make module dependencies work.
In firefox, I can't figure out how to do this. I'll attach my initial content scripts through pageMods and by calling tab.attach from the "ready" event of tabs. That seems straightforward, but then if that content script needs to load more code I can't see how to do it.
There doesn't seem to be a way to access the sandbox my content script is running in from the main.js file so that I might inject more code into it. Even if I somehow kept a reference to the relevant tab instance (which only lets me inject into the top frame in any case), it appears that each new call to tab.attach puts injected code into a new sandbox. The object tab that's passed to my ready event handle isn't a real XUL tab that I could pass to require("tabs/util").getBrowserForTab; if it were, then I think I can follow through enough of the sdk code to create my own sandbox, though I'd worry about leaving accidental memory leaks behind.
I considered passing the code back to the content script through a "eval-this-code" message, but I really don't want to use eval in my extension because of security concerns; I also worry that using eval would make it difficult to impossible to get my firefox add-on approved for AMO. (Also, how would that interact when my add-on runs on sites with a Content Security Policy?)
The usage of traits to define the add-on API seems to close off access to objects such that I can't reach inside a Worker to get a reference to the sandbox my content script is executing in. At this point, it appears that I'd need to include nearly a full copy of the sdk in my add-on just to expose one method on WorkerSandbox.
Note: I'm using the Add-On sdk (the project formerly known as JetPack). I'm willing to use Components.utils.import if someone can tell me how to use that from inside an Add-On SDK-managed content script.
Content-scripts do not expose a public API to attach more scripts to a content-script sandbox after it was initialized. You should probably file an enhancement bug and state your use case, if there isn't one filed already (search first), and/or even come up with some patches yourself.
In cases where there is a DOM that your add-on own (widget), then it's just a matter of attaching another script tag.
For things like page-mods where there is no DOM you own, you're left with a couple of options, none of which is really satisfying. As you already found out yourself, the use of traits prohibits you from accessing "private" properties/methods.
Fork page-mod/tab/content-worker to provide the functionality you need. That would require creating your own copies of the modules and expose the necessary APIs to inject scripts into existing workers.
This is has a steep learning curve (but given that you already figured out details such as traits, should be doable for you), but more importantly hard to maintain as you need to make sure you keep up with the upstream. And AMO editors will not like you very much for it :p
On the plus side, you could try to get your stuff committed upstream, fixing this problem for everybody and become a hero to many authors using the Add-on SDK.
The eval method you propose. Not only is this eval a major source for security issues, but it also may be a performance killer, as right now IIRC evaled code will not use the JIT. And, of course, it will make us AMO editors cringe, even if used "correctly".
Do not use lazy loading at all, and specify all content scripts from the very beginning. This is what add-ons usually do (I'm almost inclined to say "always"). However, this conflicts with your current design, and depending on your add-on may pose a serious performance penalty for loading stuff you didn't really need in the end.
You could use the require mechanism to have most scripts as SDK module and not content-scripts. This is not always feasible, of course, e.g. when dealing with code that would normally modify the DOM in your content-script, but might work for some other stuff.
Replace page-mod, etc with your own Greasemonkey-like, enhanced API. This means lots of work, it is error-prone, security-sensitive and has to be maintained. So, it's not really a viable solution, IMO...
Components.utils.import does not help you. It isn't available to content-scripts anyway.
I just played around a bit with Lua and tried the Koneki eclipse plugin, which is quite nice. Problem is that when I make changes in a function I'm debugging at the moment the changes do not become effective when saving the changes. So I'm forced to restart the application. Would be so nice if I could make changes in the debugger and they would become effective on the fly as for example with Smalltalk or to some extend as in hot code replacement in Java. Anybody has a clue whether this is possible?
It is possible to some degree with some limitations. I've been developing an IDE/debugger that provides this functionality. It gives you access to a remote console to execute commands in the context/environment of your running application. The IDE also supports live coding, which reloads modified code as you make changes to it; see demos here.
The main limitation is that you can't modify a currently running function (at least without changes to Lua VM). This means that the effect of your changes to the currently running function will only be seen after you exit and re-enter that function. It works well for environments that call the same function repeatedly (for example a game engine calling draw), but may not work in your case.
Another challenge is dealing with upvalues (values that are created outside of your function and are referenced inside it). There are methods to "read" current upvalues and re-create them when the (new) function is created, but it requires some code analysis to find what functions will be recreated to query them for upvalues, to get the current values, and then to create a new environment with those upvalue and assign proper values to them. My current implementation doesn't do this, which means you need to use global variables as a workaround.
There was also relevant discussion just the other day on the Lua mailing list.
I'm currently attempting to integrate a DLL (FooEmulation) into an existing project.
The DLL assumes that it will only be used to emulate one Foo at a time, and uses a lot of static globals as a result.
However, I want to be able to manage thousands of Foo instances at once.
I have the source to the original DLL, so I could convert all of the static globals into parameters that would be passed in (whether directly or via a handle), but the DLL is being maintained separately and I'd like to avoid forking/merging if at all possible.
One technique I found was to load multiple dynamically generated copies of the DLL, but that is too resource-heavy for the scale I need.
I also can't afford to create a process or thread for each Foo.
Is it possible to keep multiple copies of the DLL's static memory and restore it per use of the DLL?
How do I locate it? Am I even allowed to touch it?
When you load the DLL multiple times into the same process all the static data is shared, period. You'll have to redesign the library so that all those objects can be created dynamically as you need them during the runtime.
I am assuming you're on windows since there's nothing telling me otherwise..
Take a look here, which is the documentation for DLLMain in Windows.
DLLMain has a parameter that tells you if
A process is attaching (loading your DLL)
A process is detaching (unloading your DLL)
A thread is attaching (loading the per thread parts of your DLL)
A thread is detaching (unloading the per thread parts of your DLL)
If you catch the process or thread events and allocate (attach) / free (detach) a new instance of your statics, I think this would solve your problem. It's a little hacky, but it would work...
You have to be careful what you do in DLLMain as well. Look in the docs for the warnings about blocking in any way in DLLMain.