Consider I have a case where i have executed a program and created an instance of a class,
MyClass mClass = new MyClass()
After the execution the Reference Space will be stored in the Heap Space. Now I want to write a Program which can access the Heap Space to retrieve the data from the Previously created instance of another program.
Can I do it?
Thank you
As far as I know, in practice, no. If you give the other program admin privileges, you can read the memory of another program, but as far as I know there is no way to know for sure where the heap of that program resides. (There are probably hacky ways to achieve this, but it's not going to be pretty or reliable).
However, it is possible for a process to establish a region of shared memory that another process can read, or use sockets, but this requires co-operation between the processes. Also, it still doesn't give a process direct access to another process' heap - your program can only see what the other process lets it see.
Note that, while you can't change the behavior of new (as far as I know), there is nothing preventing you from writing code to manage your heap manually; in that sense it would be possible to place the heap directly in the shared memory region. Whether that would be wise or not is another question and, obviously, highly dependent on the context.
In order to fully understand how operating systems manage memory, you'll have to understand virtual memory and memory management hardware (you'll probably want to go deeper than Wikipedia, though) .
Well i think you can use sockets for accessing the Heap Space. I hope this works.
Related
I am little confused with term mapping, for example, when we say mapping memory for database, it means that we assigning specific amount of memory at some memory location to that database?
Also is allocating memory synonym for reserving memory?
Very often I encounter these two terms, and they aren't so clear to me.
If someone can clarify these two terms, I will be very thankful.
This might be a question better asked to the software community at stackoverflow. However, I am a CS.
I would say that terms aren't always used accurately and precisely.
In general allocating memory is making memory available to a program for an active purpose, such as allocating memory for buffers to hold a file or in in-memory structure now.
Reserving memory is often used to mean the same thing. However, it is sometimes more passive. For example reserving memory in case their is a future requirement, or protecting against too much memory allocation for a different purpose.
Often when the term 'mapping' is used, it is for a file. It may mean exactly the same as allocating. Or it means more; mapping may be using an underlying mechanism provided by virtual memory management systems, where part of virtual memory is 'mapped' to the file, without actually reading the file into physical memory. The trick is, as the memory-mapped file is accessed, the block/page being accessed is read in 'invisibly' to the process when necessary. This uses a mechanism called demand paging. It's benefit is a program can access the file as if it is all read into memory, but only the parts actually accessed are retrieved from the persistent storage system (disk, flash, whatever), which can be a huge win if only small parts of the file are needed.
Further, it simplifies the program, which can be written as if the whole file is in memory. Instead of the application developer trying to keep track of which parts of the file have been loaded into memory, the operating system does that instead.
Even better, the Operating system can be asked to track which blocks/pages have their contents changed, and it can be asked to periodically write that back out to persistent storage. This can even further simplify the application program.
This is popular with some databases.
Mapping basically means assigning. Except we often want a 1 to 1 mapping in the case of functions. If you define the function of an object, physical or just logical, and define it's relationships and how it changes under transformation then you have mapped it.
Say for instance I write a program which allocates a bunch of large objects when it is initialized. Then the program runs for awhile, perhaps indefinitely, and when it's time to terminate, each of the large initialized objects are freed.
So my question is, will it take longer to manually deallocate each block of memory separately at the end of the program's life or would it be better to let the system unload the program and deallocate all of the virtual memory given to the program by the system at the same time.
Would it be safe and/or faster? Also, if it is safe, does the compiler do this when set to optimise anyway?
1) Not all systems will free a memory for you when application terminates. Of course most of the modern desktop systems will do this, so if you are going to run your program only on Linux or Mac(or Windows), you can leave the deallocation to the system.
2) Often it is needed to make some operations with the data on termination, not just to free the memory. So if you are going to develop such program design that makes it hard to deallocate objects at the end manually, then it can happen that later you will need to perform some code before exiting and you will face up with hard problem.
2') Sometimes even if you think that your program will need some objects all the way until dead, later you may want to make a library from you program or change a project to load and unload you big objects and the poor design of your program will make this hard or impossible.
3) Moreover, the program deallocation performance depends on the implementation of the allocator you are going to use in your program. The system deallocation depends on the system memory management and even for a single system there can be several implementations. So if you face with allocation/deallocation performance problems - you would like to develop better allocator rather then hope on the system.
4) So my opinion is: When you deallocate memory manually at the end - you are always on a right way. When you don't do this, perhaps you can get some ambiguous benefits in several cases, but likely you will just face with the problems sooner or later.
Well most OS will free the memory at exit if the program, but the bigger question is why would you want it to have to?
Is it faster? Hard to say with memory sometimes. I would guess not really and definitely not worth breaking good coding practices anyway.
Is it safe? Define safe... Will your OS crash? Probably not. Will your code be susceptible to memory leaks or other problems? Absolutely, it will. In fact you are basically telling it you want memory leaks.
Best practice is to always free your memory when you are done with it. With C and C++, every malloced or new block of memory should have a corresponding free or delete.
It is a bad idea to rely on the OS to free your memory because it not only makes your code look bad and makes it less portable, but if the program was ever integrated into another program, then you will likely be tracking down memory leaks for hours.
So, short answer, always do it manually.
Programs with a short maintenance life time are good candidates for memory deallocation by "exit() and let the kernel sort them out." However, if the program will last more than a few months you have to consider the maintenance burden.
For instance, consider that someone may realize that a subsequent stage is required in the program, and some of the data is not needed, or not needed in memory. They now have to go and find out how to deallocate the memory, properly removing stale references, etc.
I was reading an article on memory fragmentation when I recalled that there are several examples of software that claim to defragment memory. I got curious, how does it work? Does it work at all?
EDIT:
xappymah gave a good argument against memory defragmentation in that a process might be very surprised to learn that its memory layout suddenly changed. But as I see it there's still the possibility of the OS providing some sort of API for global memory control. It does seem a bit unlikely however since it would give rise to the possibility of using it in malicious intent, if badly designed. Does anyone know if there is an OS out there that supports something of the sort?
The real memory defragmentation on a process level is possible only in managed environments such as, for example, Java VMs when you have some kind of an access to objects allocated in memory and can manage them.
But if we are talking about the unmanaged applications then there is no possibility to control their memory with third-party tools because every process (both the tool and the application) runs in its own address space and doesn't have access to another's one, at least without help from OS.
However even if you get access to another process's memory (by hacking your OS or else) and start modifying it I think the target application would be very "surprised".
Just imagine, you allocated a chunk of memory, got it's starting address and on the next second this chunk of memory is moved somewhere else because of "VeryCoolMemoryDefragmenter" :)
In my opinion memory it's a kind of Flash Drive, and this chip don't get fragmented because there aren't turning disks pins recording and playing information, in a random way, like a lie detector. This is the way that Hard Disk Fragmentation it's done. That's why SSD drives are so fast, effective, reliable and maintenance free. SSD it's a BIG piece of memory and it kind of look alike.
I had an issue recently (see my last question) that led me to take a closer look at the memory management in my Delphi application. After my first exploration, I have two questions.
I've started playing with the FastMMUsageTracker, and noticed the following. When I open a file to be used by the app (which also creates a form etc...), there is a significant discrepancy between the variation in available virtual memory for the app, and the variation in "FastMM4 allocated" memory.
First off, I'm a little confused by the terminology: why is there some FastMM-allocated memory and some "System-allocated" (and reserved) memory? Since FastMM is the memory manager, why is the system in charge of allocating some of the memory?
Also, how can I get more details on what objects/structures have been allocated that memory? The VM chart is only useful in showing the amount of memory that is "system allocated", "system reserved", or "FastMM allocated", but there is no link to the actual objects requiring that memory. Is it possible for example to get a report, mid-execution, similar to what FastMM generates upon closing the application? FastMM obviously stores that information somewhere.
As a bonus for me, if people can recommend a good reference (book, website) on the subject, it would also be much appreciated. There are tons of info on the net, but it's usually very case-specific and experts-oriented.
Thanks!
PS: This is not about finding leaks, no problem there, just trying to understand memory management better and be pre-emptive for the future, as our application uses more and more memory.
Some of your questions are easy. Well, one of them anyway!
Why is there some FastMM-allocated
memory and some "System-allocated"
(and reserved) memory? Since FastMM is
the memory manager, why is the system
in charge of allocating some of the
memory?
The code that you write in Delphi is only part of what runs in your process. You use 3rd party libraries in the form of DLLs, most notably the Windows API. Anytime you create a Delphi form, for example, there are a lot of windows objects behind it that consume memory. This memory does not get allocated by FastMM and I presume is what is termed "system-allocated" in your question.
However, if you want to go any deeper then this very rapidly becomes an extremely complex topic. If you do want to go deeper into the implementation of Windows memory management then I think you need to consult a serious reference source. I suggest Windows Internals by Mark Russinovich, David Solomon and Alex Ionescu.
First off, I'm a little confused by the terminology: why is there some FastMM-allocated memory and some "System-allocated" (and reserved) memory? Since FastMM is the memory manager, why is the system in charge of allocating some of the memory?
Where do you suppose FastMM gets the memory to allocate? It comes from the system, of course.
When your app starts up, FastMM gets a block of memory from the system. When you ask for some memory to use (whether with GetMem, New, or TSomething.Create), FastMM tries to give it to you from that first initial block. If there's not enough there, FastMM asks for more (in one block if possible) from the system, and returns a chunk of that to you. When you free something, FastMM doesn't return that memory to the OS, because it figures you'll use it again. It just marks it as unused internally. It also tries to realign unused blocks so that they're as contiguous as possible, in order to try not to have to go back to the OS for more needlessly. (This realignment isn't always possible, though; that's where you end up with memory fragmentation from things like multiple resizing of dynamic arrays, lots of object creates and frees, and so forth.)
In addition to the memory FastMM manages in your app, the system sets aside room for the stack and heap. Each process gets a meg of stack space when it starts up, as room to put variables. This stack (and the heap) can grow dynamically as needed.
When your application exits, all of the memory it's allocated is released back to the OS. (It may not appear so immediately in Task Manager, but it is.)
Is it possible for example to get a report, mid-execution, similar to what FastMM generates upon closing the application?
Not as far as I can tell. Because FastMM stores it somewhere doesn't necessarily mean there's a way to access it during runtime from outside the memory manager. You can look at the source for FastMMUsageTracker to see how the information is retrieved (using GetMemoryManagerState and GetMemoryMap, in the RefreshSnapshot method). The source to FastMM4 is also available; you can look and see what public methods are available.
FastMM's own documentation (in the form of the readme files, FastMM4Options.inc comments, and the FastMM4_FAQ.txt file) is useful to some extent in explaining how it works and what debugging options (and information) is available.
For a detailed map of what memory a process is using, try VMMAP from www.sysinternals.com (also co-authored by Mark Russinovich, mentioned in David's answer). This also allows you to see what is stored in some of the locations (type control-T when a detail line is selected).
Warning: there is much more memory in use by your process than you might think. You may need to read the book first.
Is there a way to access (read or free) memory chunks that are outside the memory that is allocated for the program without getting access violation exceptions.
Well what I actually would like to understand apart from this, is how a memory cleaner (system garbage collector) works. I've always wanted to write such a program. (The language isn't an issue)
Thanks in advance :)
No.
Any modern operating system will prevent one process from accessing memory that belongs to another process.
In fact, it you understood virtual memory, you'd understand that this is impossible. Each process has its own virtual address space.
The simple answer (less I'm mistaken), no. Generally it's not a good idea for 2 reasons. First is because it causes a trust problem between your program and other programs (not to mention us humans won't trust your application either). second is if you were able to access another applications memory and make a change without the application knowing about it, you will cause the application to crash (also viruses do this).
A garbage collector is called from a runtime. The runtime "owns" the memory space and allows other applications to "live" within that memory space. This is why the garbage collector can exist. You will have to create a runtime that the OS allocates memory to, have the runtime execute the application under it's authority and use the GC under it's authority as well. You will need to allow some instrumentation or API that allows the application developer to "request" memory from your runtime (not the OS) and your runtime have a way to not only response to such a request but also keep track of the memory space it's allocating to that application. You will probably need to have a framework (set of DLL's) that makes these calls available to the application (the developer would use them to form the request inside their application).
You have to be sure that your garbage collector does not remove memory other then the memory that is used by the application being executed, as you may have more then 1 application running within your runtime at the same time.
Hope this helps.
Actually the right answer is YES.. there are some programs that does it (and if they exists.. it means it is possible...)
maybe you need to write a kernel drive to accomplish this, but it is possible.
Oh - and I have another example... Debugger attach command... here is one program that interacts with another program memory even though both started as a different process....
of course - messing with another program memory.. if you don't know what you're doing will probably make it crush...