We're currently preparing our Azure role (standard Web Role) for an expected massive load, and we need to know how much memory the current setup consumes. To accomplish this, we're using load tests while measuring the consumed memory with GC.GetTotalMemory.
The page http://technet.microsoft.com/en-us/cloud/gg663909.aspx lists the Compute Instance Guaranteed Memory for each instance size (for example, 0.768 GB for the Extra-Small Instance and 3.5 GB for the Medium Instance).
Are the values of GC.GetTotalMemory comparable to the values in these list? In other words, if GC.GetTotalMemory is staying significantly below the listed limit, can we be sure that there won't be any sudden perfomance loss due to memory swapping?
If we hit the limit, is our assumption correct that there will be some memory swapping (writing memory content to the virtual harddisk), or will there be more severe implications like repeated App Pool recycling?
(the last question comes up because most shared hosters will recycle your App Pool if you hit some memory limit, but frankly we don't expect anything like this from Windows Azure)
This method will only give you the currently allocated bytes by your process. The 0.768 GB includes the memory availble to the operating system, and there can be virtual memory as well.
system.gc.gettotalmemory
To get the total system memory you can use:
Add a Reference to System.Management.
private static void DisplayTotalRam()
{
string Query = "SELECT MaxCapacity FROM Win32_PhysicalMemoryArray";
ManagementObjectSearcher searcher = new ManagementObjectSearcher(Query);
foreach (ManagementObject WniPART in searcher.Get())
{
UInt32 SizeinKB = Convert.ToUInt32(WniPART.Properties["MaxCapacity"].Value);
UInt32 SizeinMB = SizeinKB / 1024;
UInt32 SizeinGB = SizeinMB / 1024;
Console.WriteLine("Size in KB: {0}, Size in MB: {1}, Size in GB: {2}", SizeinKB, SizeinMB, SizeinGB);
}
}
Source for code
To answer your last question, Windows Azure will stay out of the way, and paging will happen like on any Windows server.
Whether IIS recycles your app pool probably depends on your IIS settings, but those are under your control. (You can, for example, run appcmd in a startup task if you want to change a default.)
Related
Note that I've also posted an issue on GitHub on the repo: https://github.com/apache/lucenenet/issues/784
I'm running the latest Lucene .NET versions:
Lucene.Net 4.8.0-beta00016
Lucene.Net.Analysis.Common 4.8.0-beta00016
I'm doing the following:
using var analyzer = new KeywordAnalyzer();
using var directory = FSDirectory.Open(IndexPath);
var config = new IndexWriterConfig(LuceneVersion, analyzer)
{
OpenMode = OpenMode.CREATE, // Use OpenMode.CREATE to overwrite, or OpenMode.APPEND to just open
RAMPerThreadHardLimitMB = 100,
RAMBufferSizeMB = 100,
};
using var writer = new IndexWriter(directory, config);
// Write index to disk
writer.AddDocuments(productDocuments);
writer.AddDocuments(productCategoryDocuments);
writer.AddDocuments(productTypeDocuments);
writer.AddDocuments(productLineDocuments);
writer.Commit();
As soon as writer.AddDocuments() is called, the memory consumption grows a lot.
You can see the sudden increase in the Diagnostic Tool in visual studio:
Running it multiple times keeps increasing the memory usage up to 3GB (on my machine), at which point it doesn't grow any longer. And these 3GB are never released again. The program doesn't crash, and it stops acquiring memory.
I want to limit how much memory Lucene can use, but using RAMBufferSizeMB and RAMPerThreadHardLimitMB seems to have no effect at all.
I tried flushing as well, which had no effect, and I tried calling Dispose() which also had no effect.
I've also tried setting MaxBufferedDocs to 1000, still no limit to RAM usage.
Have I missed something in the documentation? Is there a way to limit RAM usage of the Lucene .NET IndexWriter?
To future developers having memory consumption issues with Lucene.Net:
Try writing one document at a time with writer.AddDocument() rather than writer.AddDocuments().
writer.AddDocument() handles memory much better than writer.AddDocuments().
Also calling writer.Commit() more will flush memory more often, and the Garbage Collector can keep the memory consumption lower.
My experiments with Lucene in a Console Application can be seen here: https://github.com/apache/lucenenet/issues/784
I'm playing heaps of videos at the same time with AVPlayer. To reduce loading times, I'm storing the corresponding views in a NSCache.
This works fine until reaching a certain number of videos, from which the videos simply stop playing, or even appearing.
There's no error, log or memory warning. In particular, I'm listening to UIApplicationDidReceiveMemoryWarningNotification to clear the cache but this is never received.
If I remove the cache, all the videos play at expense of worse performance.
This makes me suspect that AVPlayer is using memory from a different process (which one?). And when that memory reaches a certain limit, new players cease to work.
Is this correct?
If so, is there a way to be notified when this magic limit is reached to take the appropriate measures (e.g., clear the cache) to ensure playback of other media?
Good news and bad news - good is you can probably fix the problem, bad is it takes work and is somewhat complex.
Root Problem
The reason you don't get notified early happens because iOS does not find out that your app has exceeded its memory budget til its almost too late, then it immediately kills it. The problem has to do with the way iOS (and OS X) manage the file system cache. Normally, when files get opened, as you read the data, the file data gets transferred into a buffer in the Unified Buffer Cache (a term you can google for more info) - I'll call it UBC from now on.
So suppose you have 10 open files, and you have read every file to the end, but have not closed the files. Well, all that data is sitting in the UBC. Now, if you close the files, the buffers are all freed. And technically, the OS can purge this buffers too - only it seems by the time it realizes that memory is tight, it chooses to blow the app away first (and there may be valid reasons for it to do this). So imagine that you app is showing videos, and the way the videos get loaded is through the file system, the number of free buffers starts dropping. At some point iOS notices this, tracks down who most belong to (your app), and wham, kills your app ASAP.
I hit this problem myself in an open source project I support, PhotoScrollerNetwork. Users started complaining that their project was getting terminated by the system, like you, without any notification. I tried in vain to monitor the UBC (there are APIs on OS X to do so, but not on iOS). In the end I found a solution using an heuristic - monitor all your memory usage including the UBC, and don't exceed 50% of the total available iOS memory pool.
So (you might ask) - what is the Apple approved way to solve this problem? Well, there is none. How do I know that? Because I had a half hour long discussion at WWDC 2012 with the Director of Core iOS in one of the labs (after getting ping ponged around by others who had no idea what I was talking about). In the end, after I explained the above heuristic, he told me directly that the solution was probably as good as any he could think of. Without an API to directly monitor the UBC, you can only approximate its usage and adjust accordingly.
But you say, I'm using the NSCache - why doesn't the system account for the AVPlayer memory there? There reason is undoubtedly the UBC - an AVPlayer instance probably only consumes a few thousand K of memory itself - its the open file to the video that is not accounted for by iOS.
Possible Solutions
1) If you can load the videos directly into a NSData object, and keep that in the NSCache, you can most likely totally avoid the UBC issues mentioned above. [I don't know enough about the AV system to know if you can do this.] In this case the system should be capable of purging memory when it needs to.
2) Continue using your original code, but add memory management to it. That is, when you get create an AVPlayer instance, you will need to account for the size of the video in bytes, and keep a running tally of all this memory. When you approach 50% of total device free memory, then start purging old AVPlayers.
Code
For completeness, I've provided the relevant code from PhotoScrollerNetwork below. If you want more details you can peruse the project - however its quite complex so expect to spend some time (its doing JPEG decoding on the fly for massive images and writing tiles to the file system as the decode proceeds).
// Data Structure
typedef struct {
size_t freeMemory;
size_t usedMemory;
size_t totlMemory;
size_t resident_size;
size_t virtual_size;
} freeMemory;
Early on in your app:
// ubc_threshold_ratio defaults to 0.5f
// Take a big chunk of either free memory or all memory
freeMemory fm = [self freeMemory:#"Initialize"];
float freeThresh = (float)fm.freeMemory*ubc_threshold_ratio;
float totalThresh = (float)fm.totlMemory*ubc_threshold_ratio;
size_t ubc_threshold = lrintf(MAX(freeThresh, totalThresh));
size_t ubc_usage = 0;
// Method on some class to monitor the memory pool
- (freeMemory)freeMemory:(NSString *)msg
{
// http://stackoverflow.com/questions/5012886
mach_port_t host_port;
mach_msg_type_number_t host_size;
vm_size_t pagesize;
freeMemory fm = { 0, 0, 0, 0, 0 };
host_port = mach_host_self();
host_size = sizeof(vm_statistics_data_t) / sizeof(integer_t);
host_page_size(host_port, &pagesize);
vm_statistics_data_t vm_stat;
if (host_statistics(host_port, HOST_VM_INFO, (host_info_t)&vm_stat, &host_size) != KERN_SUCCESS) {
LOG(#"Failed to fetch vm statistics");
} else {
/* Stats in bytes */
natural_t mem_used = (vm_stat.active_count +
vm_stat.inactive_count +
vm_stat.wire_count) * pagesize;
natural_t mem_free = vm_stat.free_count * pagesize;
natural_t mem_total = mem_used + mem_free;
fm.freeMemory = (size_t)mem_free;
fm.usedMemory = (size_t)mem_used;
fm.totlMemory = (size_t)mem_total;
struct task_basic_info info;
if(dump_memory_usage(&info)) {
fm.resident_size = (size_t)info.resident_size;
fm.virtual_size = (size_t)info.virtual_size;
}
#if MEMORY_DEBUGGING == 1
LOG(#"%#: "
"total: %u "
"used: %u "
"FREE: %u "
" [resident=%u virtual=%u]",
msg,
(unsigned int)mem_total,
(unsigned int)mem_used,
(unsigned int)mem_free,
(unsigned int)fm.resident_size,
(unsigned int)fm.virtual_size
);
#endif
}
return fm;
}
When you open a video, add the size to ubc_usage, and when you close one decrement it. When you want to open a new video, test ubc_usage against ubc_threadhold, and its it exceeds the value you have to close something first.
PS: you can try calling that freeMemory method at other times, and see, but in my case it hardly changes at all when files get opened - the system seems to consider the whole UBC as "free", since it could purge it if it needed to (I guess).
If you're throwing all of these videos in a NSCache, you have to be prepared for the cache to throw away items when it feels like they are consuming too much memory. From the NSCache documentation:
The NSCache class incorporates various auto-removal policies, which
ensure that it does not use too much of the system’s memory. The
system automatically carries out these policies if memory is needed by
other applications. When invoked, these policies remove some items
from the cache, minimizing its memory footprint.
Check to see if you're getting nils back from the cache, and if you are, you'll have to reconstruct your objects.
Edit:
It is also worth mentioning that objc.io #7 advises against storing large objects in a NSCache:
The eviction method of NSCache is non-deterministic and not
documented. It’s not a good idea to put in super-large objects like
images that might fill up your cache faster than it can evict itself.
I have a TCP/IP DataSnap server running as a service [Session based LifeCycle] that continuously chews up memory and never comes back to the starting memory size even when there are no connections to it.
In order to eliminate My code as the culprit, I have modeled up a basic TCP/IP DataSnap server running as VCL [Session based LifeCycle] that serves a Server Method class [TDSServerModule] which only contains basic mathematical functions using native data types [no objects to create or free].
When I connect to said DataSnap server with a very thin client I get the same results.
The Memory Usage continuously grows with each connection and sporadically grows when executing the server side methods from the client. Once the connections are closed the DataSnap Server never reduces its Memory Usage [even when left running without connections for 8 hrs].
Any suggestions as to why this occurs or more importantly how to curtail it?
I am using RAD Studio XE2 Update 4 HotFix 1.
Let me quote a "must read" article about DataSnap. This is about XE3 but i hope the code here would work for XE2 as well.
Memory consumption
One of the issues that I observed was related to memory consumption. Why Datasnap server consumes so much memory if the method called does absolutely nothing?
Maybe I don’t know how to explain exactly but i will try. Basically the DataSnap creates a session for each HTTP connection that it receives. This session will be destroyed after 20 minutes, in other words, on the first 20 minutes of the test the memory consumption will only go up, after that it has the tendency of stabilize itself. I really have no idea why Datasnap does this. In a REST application I don’t see much sense in these sessions with a default configuration. Of course, sessions can be helpful, but i can’t understand why it’s a default configuration. Indeed, DataSnap doesn’t have a configuration for that. It appears like you just have to use this session control, without being able to choose otherwise (There is no documentation). The MORMot framework has a session control too but it’s configurable and doesn’t consumes so much memory.
Anyway, there is a way around this problem. Daniele Teti wrote an article on his blog, take a look. The solution that I will show here was placed by him on his blog. Thanks Daniele.
uses System.StrUtils, DataSnap.DSSession, Data.DBXPlatform;
function TServerMethods1.HelloWorld: String;
begin
Result := 'Hello World';
GetInvocationMetaData.CloseSession := True;
end;
After running this method the session will close and memory consumption will be lower. Of course still exists an overhead for creating and destroying this session.
So it seems the best course for you is ending every server method with explicit memory cleanup, if that was possible in XE2. Then you'd better read thosee articles again and prepare for future scalability challenges.
http://www.danieleteti.it/2012/12/15/datasnap-concurrency-problems-and-update1/
http://robertocschneiders.wordpress.com/2013/01/09/datasnap-analysis-based-on-speed-stability-tests-part-2/
I added the below method and called it from the "TWebModule1::WebModuleBeforeDispatch" event. It eliminated the memory consumption and actually allow the idle REST service to return to a state of no-session memory. DataSnap definitely needs to work on this issue.
// ---------------------------------------------------------------------------
/// <summary> Memory Restoration. DataSnap opens a session for each call
/// even when the service is set for invocation.
/// Sessions are building up consuming memory and seem not to be freed.
/// See: https://stackoverflow.com/questions/17748300/how-to-release-datasnap-memory-once-connections-are-closed
/// </summary>
/// <remarks> Iterates session in the session manager and closes then terminates
/// any session that has been idle for over 10 seconds.
/// </remarks>
/// <returns> void
/// </returns>
// ---------------------------------------------------------------------------
void TWebModule1::CloseIdleSessions()
{
TDSSessionManager* sessMgr = TDSSessionManager::Instance;
int sessCount = sessMgr->GetSessionCount();
WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "Session Count: " + IntToStr(sessCount));
TStringList* sessKeys = new TStringList;
sessMgr->GetOpenSessionKeys(sessKeys);
WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "Session Keys Count: " + IntToStr(sessKeys->Count));
TDSSession* sess = NULL;
for(int index = 0; index < sessKeys->Count; index++)
{
String sessKey = sessKeys->Strings[index];
sess = sessMgr->Session[sessKey];
unsigned elapsed = (int)sess->ElapsedSinceLastActvity();
if(elapsed > 10000)
{
WriteLogEntry(LogEntryTypeDebug, "TWebModule1::CloseIdleSessions", "CloseSession TerminateSession Key: " + sessKey);
sessMgr->CloseSession(sessKey);
sessMgr->TerminateSession(sessKey);
}
sess = NULL;
}
delete sessKeys;
sessMgr = NULL;
}
You should check the Lifecycle property on TDSServerclass component on your servercontainer. It provides a way to determine the way the session is handled. It defaults to session. Setting it to invokation will free the session after each call (invokation). This will of course mean you have no state. This would be oke in a typical REST server though.
if you still have memory consumption growing. put the following line in your dpr unit.
ReportMemoryLeaksOnShutdown := True;
your application will then show you the memory leaks it has on closing of your datasnap server.
all
My program maybe have a memory issue, so I try to find information about memory usage provided by various tools. In order to find the cause, I do simple experiments as well.
In release mode, I add the following code,
pChar = new char[((1<<30)/2)];
for(int i; i < ((1<<30)/2); i++)
{
pChar[i] = i % 256;
}
When the code is executed, the available physical memory in Windows task manager doesn't change. In my view, the compiler may remove the code to boost performance. I declare the variable as one global variable. It doesn't work. But in debug mode, the available physical memory in Windows task manager changes as expected. I can't understand that.
I have another question. Will the new operation allocate memory from virtual memory if the physical memory runs out. Or one exception will be thrown?
It's indeed quite possible that the compiler detects a "write-only" variable. Since it's non-volatile, the writes can be safely eliminated, and then there's no need for the OS to actually allocate RAM.
new just allocates address space, on modern systems. Physical RAM is allocated when needed. Typically this happens when the ctor runs, as it initializes the members. But in new char there's of course no ctor.
My Delphi XE application is based on a single EXE using a local server DLL created by RemObjects and uses a lot of memory for a specific operation until it generates an exception saying there are not enough memory. So I'm trying to understand why and where this is happening so I placed various steps throughout my code where I report on memory usage. The problem is that I'm getting very different information based on the method used to get memory usage information:
If I use the method explained here which asks FastMM directly for both the Client EXE and Server DLL, here is what I get:
STEP 1: [client] = 36664572 - [server] = 3274976
STEP 2: [client] = 62641230 - [server] = 44430224
STEP 3: [client] = 66665630 - [server] = 44430224
Now if I use the method explained here which uses GetProcessMemoryInfo, I get far more memory usage:
STEP 1: [process] = 133722112
STEP 2: [process] = 1072115712
STEP 3: [process] = 1075818496
It looks like second method is the right based on my memory problems but how could the FastMM method be so "low" ? And what can explain the difference ?
GetProcessMemoryInfo also reports memory that is not managed by FastMM, like memory that is allocated by the various non Delphi dlls you might call (like winapi).
Also FastMM can allocate more memory from Windows that your application actually uses for internal structures, fragmentation and pooling.
And as last, with GetProcessMemoryInfo you measuring the Workingset size. That is what part of the application's memory currenctly in RAM instead if in the page file. It includes more than just data structures and is definately not comparable to the total memory the application has allocated. PagefileUsage would be more comparable. Workingset size almost never is what you are looking for. See here for a better explanation.
So they both give different results because they both measure different things.