nunit large project integration tests, out of memory exception - memory

I'm using nunit for running integration tests (which each test interacts with the database via nhibernate) for a large project with 600+ tests. problem is after 10-15 minutes minutes nunit throws out of memory exception. I used redgate ants memory profiler to see why nunit is not releasing memory between the tests. seems it tries to recreate permission objects per test and memory keeps growing and eventually it throws out of memory exception.
I took snapshots on one integration test class but you can see the memory keeps growing after awhile. I didn't find any call or setting for nunit to force memory release or solve this trust issue.
I really appreciate any help.
<NamedPermissionSets><PermissionSet class=\\"System.Security.NamedPermissionSet\\"version=\\"1\\" Unrestricted=\\"true\\" Name=\\"Full ...
ANTS Snapshots:
http://www.tinyuploads.com/images/IXwb8Q.jpg
http://www.tinyuploads.com/images/3R2VbB.png

I can't tell which version you're using, but your problem might have been fixed in NUnit 2.6.3. See here for the bug report.

Related

Not sure how to resolve outofmemory issue on jenkins server?

My Jenkins server keeps crashing, so I generated a heap dump which I then put through visualVM. It shows most of the memory is being used up by the class java.util.concurrent.concurrenhashmapnode.
My understanding is loads objects are being referenced, which are unable to be GC'd. As a result, most the memory is being used up by this. Any idea how to resolve this? I'm new to system admin stuff, so not the most technically proficient sorry.
TIA
I've recently came across OutOfMemoryError which crashed my Jenkins every 2 days. It was due to Ldap bug in old version of java: Ldap Error and java fixed versions matrix
In my case updating java fixed the problem.
Anyway to investigate OutOfMemoryError I did as follows:
restarted jenkins after crash,
took incremental thread dumps every half an hour, can be taken from <jenkinsUrl>/threadDump,
comparing thread dumps pointed me to memory leak on ldap threads.
In general I'd also suggest to:
update: java, Jenkins and it's plugins, other problematic tools,
investigate Jenkins logs, dumps, profile heap (what you already did).

Mule 3.8.1 CE - Memory leak issue

In the past few months our company's Mule was down twice - it happened when there were lots of traffic. To investigate, we did a load test to simulate a large number of users. I ran this test on my local machine with 512m application memory size, used JMeter to send request to Mule (Number of threads: 1000, ramp-up period: 10sec). And I use Visual VM to analyze.
Here are my observations:
I see "java.lang.OutOfMemoryError: GC overhead limit exceeded" in Anypoint Studio's console.
No error in mule log.
I did a heap dump - it showed that the combined size of char array objects is really big.
We see a very high number of AsyncLogger related classes: we now suspect we have a memory leak when writing logs.
When I changed all log level to Error, in order to remove all normal logs in log4j2.xml, the Garbage Collection performed correctly.
Then I tried to use Mule Runtime 3.9.0 Community Version and reverted the log level change, I can see Garbage Collection was performing and number of threads go down when I stopped sending request to Mule.
So why is there memory leak problem in 3.8.1? I checked 3.9.0 release notes - Resolved Issue checklist, I don't see any issue related to this issue. What is the root cause of the problem?
Log4j was updated in Mule 3.8.5/3.9.0, that could be the reason. I suggest you take a look at all release notes in between 3.8.1 and 3.9.0 since the 3.9.0 release notes are based on the latest 3.8.x version at that point. In fact, you can find the update information in the 3.8.5 release notes. HTH

Speeding up Dart2JS compilation

Is there any way to get the Dart2JS compiler to minimize time to output in any significant way? For example, is it possible to get it to compile only files which have been changed since the last compilation?
For context, I am experimenting with Dartangular. Compiling even a trivial hello world example takes 15-20 seconds, which in the long run becomes untenable to work with.
Recently dart2js had an issue which caused it to frequently run out of memory when to many tasks were run in parallel. To fix this issue they limited the number of tasks that can be run in parallel. This greatly slowed down the compilation speed but fixed the memory issue.
Development is usually done only in Dart and there is no compilation to JavaScript involved. This is mostly a deployment task. Of course it's also necessary to test the built result, so it is still important that it is fast.
As far as I know the Dart team is still working on it.
There is one option to increase heap size. and this will fix the run out of memory issue #Günter Zöchbauer mentioned,
--old_gen_heap_size=MEGABYTE_SIZE
For instance, --old_gen_heap_size=2048.
But i'm not quite sure this will speed up compilzation.

Rubymine 5.4.3.2.1 freezes sometimes

I am using Rubymine to work on a suite of rails projects. There are quite a bit of gems and source files, but I think rubymine should be able to handle everything despite the size...
My sources files (and gems) are shared in a Vagrant box (see https://stackoverflow.com/a/19662310/311744).
Sometimes, when I am editing, Rubymine freezes. (about 5-6 seconds).
Usually, 3-4 minutes after I restart the IDE, everything is fine. Then I have one big 5 seconds freeze, and then from that point every 15-20 seconds I have a new freeze. During a freeze, Rubymine takes 100% (or a big part) of the CPU.
I am running on MacOS 10.8.5 on a recent MBPr.
I don't want to give up Rubymine. How can I troubleshoot that ?
In case of the performance issues you should start from the troubleshooting guides on the official support site:
thread dumps
CPU snapshots
idea.log
Check that all the files are local, try to increase the heap size in case the problem is caused by the JVM garbage collection.
If nothing helps, provide all the collected data to the technical support team or to the issue tracker.
Trying RubyMine 6.0 will definitely not hurt as noted in the comment above =)
This guide is for all the users having performance issues with RubyMine or any other IntelliJ IDEA platform based IDE.

Techniques and tools for debugging problems on remote machines?

Users have been reporting problems/crashes/bugs that I can't reproduce on my machine. I'm finding these problems difficult to fix.
I've started using EurekaLog (fantastic!) and SmartInspect. Both these tools have helped greatly but I'm still finding it difficult to catch some problems.
I've just purchased Debugging by David Agans (and waiting for it to arrive).
Are there any other tools or techniques specific to Delphi that will help with catching these hard to find remote problems? The kinds of problems I'm finding difficult to track down are those that don't raise exceptions or have a clear cause. EurekaLog catches exceptions and SmartInspect is pretty good once I have a theory to check. But in some cases it is a seemingly random crash and there are several thousand lines of code that may may be at fault. How to narrow down to the root cause?
MadExcept is what I use, and it is fabulous. I have also used EurekaLog and find the functionality almost exactly identical, except that I have more experience and time using MadExcept. it's free for non-commercial use, and reasonably priced for commercial use.
Update: MadExcept 4 is now out and even supports 64 bit Delphi XE2 apps, and has memory-leak checking too.
When nothing blows up, I rely on heavy use of trace logging. I have a TraceMessage(integer,string) function which I call throughout all my apps, and when someone has problems I get them to click a menu item that turns up the debug trace level to the most verbose level; it gives me a complete history of everything my application did, and this has helped me even more than madExcept, to solve problems at customer sites. Customers get a crash, and that crash report sent by madexcept contains a log file (created by my app) that is attached automatically. I believe you can do this equally well with madExcept and EurekaLog. If you need a logging system you could download Log4D, or you could write your own, it's pretty simple.
For always-free, try JclDebug, which requires more work to set up, but which has worked fabulously for me, also.
For help with heap problems, learn more about fastMM (full version) debug options.
And you shouldn't forget that Delphi itself supports Remote debugging, if you can reproduce a crash on machines in your office that don't have delphi installed, use remote debug across the office network instead of installing a complete RAD Studio installation on that other machine at your work. You could also use remote debug to connect to a client PC computer across the internet, but I have not tried remote debug across the internet yet, so I can't say whether it works great over the internet or not. I do know that since remote debug doesn't support automatic deploy of the EXE file you built (you have to do that part yourself), remote debug over internet, to a client PC is more work.
You might also find lots of your problems by fixing all your hints and warnings, and then going through with CodeHealer or Pascal Analyzer (PAL) from Peganza. These static analysis tools can help you find real code problems.
If performance and memory usage are your problems, get the full version of AQTime, and use it to profile and watch your system operate. It will help you fix your memory leaks, and understand your app's runtime behaviour and memory usage, not just leaks but bottlenecks for memory and CPU usage. Some of those bottlenecks can also be the source of some odd problems. I have even used AQTime to help me find deadlocks, since it can generate traces of execution, that can help me figure out what code is running, and locate deadlocks. Update: AQTime is not installable on machines other than your main dev machine, without violating the newly modified license terms for AQTime. These terms were never this restrictive in the good old days.
If you gave more exact idea of what your problems are, I'm sure other people could give you some more ideas that are specific, but all of the above are general techniques that have served me well.
One of the best way is to use the Remote Debugger that comes with Delphi, so you can debug directly the application running on the remote machine. THe remote debugger is somewhat buggy in some Delphi releases, and requires to follow the instructions carefully to make it working, but when needed it's a tool to consider. Also check if there are updates available for your version, they could come in a separate installer for deployment on "remote" systems. Otherwise first install the remote debugger, than check if the files installed has newer versions in your local installation, and the copy tehm on the remote machine.
CodeSite has helped me a lot in these situations. Since XE it is bundled with Delphi.
Logging is the key, in this matter.
Take a look at our TSynLog class available in our Open Source SynCommons library.
It does have the JCL Debug / MadExcept features, with some additional (like customer-side profiling, and logging):
logging with a set of levels;
fast, low execution overhead;
can load .map file symbols to be used in logging;
compression of .map into binary .mab (900 KB -> 70 KB);
inclusion of the .map/.mab into the .exe;
reading of an external .map to add unit names and line numbers to a log file without .map available information at execution;
exception logging (Delphi or low-level exceptions) with unit names and line numbers;
optional stack trace with units and line numbers;
methods or procedure recursive tracing, with Enter and auto-Leave using interfaces;
high resolution time stamps, for customer-side profiling of the application execution;
set / enumerates / TList / TPersistent / TObjectList / TContainer / dynamic array JSON serialization;
per-thread or global logging;
multiple log files on the same process;
integrated log archival (in zip or any other format);
Open Source, works from Delphi 5 up to XE.

Resources