As far as I know , the paging system do eliminate external fragment in physical address space, but what about fragment in virtual address space?
In modern OSes the virtual address space is used per process (the kernel has it's own dedicated virtual range), which means that the demands are much lower compared to the whole OS. The virtual address space is usually large enough (2-3 GB per process on x86 and multiple TB (8 on Windows) on x64 machines), so that fragmentation is not such a big issue as for the OS-wide physical address space. Still the issue can arise, especially for long running and memory hungry applications on x86 or other 32 bit architectures. For this the OS provides mechanisms, for example in form of the heap code. An application usually reserves one or more memory ranges as heap(s) when it starts and allocates the required chunks of memory from there later (e.g. malloc). There are a varity of implementations that handle fragmentation of the heap in different ways. Windows provides a special low-fragmentation heap implementation that can be used, if desired. Everything else is usually up to the application or it's libraries.
Let me add a qualification to your statement. Paging systems nearly eliminate fragmentation in the physical address space when the kernel is pageable.
On some systems, the user mode page tables are themselves pageable. On others, they are are physical locations that are not pageable. Then you can get fragmentation.
Fragmentation in the virtual address space tends to occur in heap allocation. The challenge of heap managers is to manage the space while minimizing fragmentation.
Related
I have been reading about demand paging and there are a few terminologies I don't understand.
What is a frame? I read that it is a block of physical memory which can at least fit in a page ( so a frame can fit one or more pages? ). But does this physical memory refer to the RAM or the disk storage?.
Which one of these is true:
The virtual address space ( which is 4 GiB in 32 bit systems ) is allocated for one application at a time, so that every application has 4 GiB virtual address to access to, and each time we switch application, the OS reconfigures the virtual address space to map to other other applications. Or the virtual address space is allocated to several processes? If so, how much virtual memory does each application get and what happen when it wants more virtual memory?
Do we have a page table for each application running, or a common page table for all applications?
Where does virtual memory fragmentation come from ?
I hope someone can clarify me.
A frame is a block of physical memory, RAM. I've not heard of frames being larger than pages, I've always understood them synonymous. However, a CPU may allow for frames/pages of different sizes to coexist simultaneously (e.g. large pages of 4MB/2MB/1GB size and regular 4KB pages on x86).
Whether there's a single address space shared by multiple applications or each has its own address space depends on the OS. Windows 3.xx and 9x/Me had a shared address space. Windows NT/2000/XP/etc had individual, per-app address spaces. Not all of the address space is available to an application / applications. A portion is reserved for the OS (kernel, drivers, their data).
Should be obvious now. One note though... Even with individual address spaces a portion of memory can still be made available in several different address spaces and it may be done by having a common page table in the respective processes. Also, it's very typical for the kernel portion of the address space to be managed by several page tables common to all processes.
Whether the address space is virtual or not, it can become fragmented. You may want to allocate a contiguous (in terms of the virtual addresses) buffer of, say, 8KB, but you may only have two non-adjacent 4KB regions available.
An operating system itself has resources it needs to access, like block I/O cache and process control blocks. Does it use virtual memory addresses or physical memory addresses?
I feel like it should be the former since it prevents the need to keep a large area of physical memory for a purpose, even when it is mostly empty. The mechanism of page tables/virtual memory would do a much better job at keeping those resources that the OS really needs.
So which is it?
10 randomly selected operating systems will do virtual memory management in 10 different ways. There's no answer that applies to all operating systems.
Some (e.g. MS-DOS) don't support or use virtual memory management for anything, some (e.g. Linux) just map all of physical memory into kernel space and don't bother using virtual memory management tricks for the kernel itself (it's almost as if the kernel is in physical memory even though it's technically both), and some may do any number of virtual memory tricks in kernel space.
I wonder why the application can't use 4GB memory on the 32-bit OS since we use the virtrual memory technique. I know some programs like OS which should use some memory, does it mean every application must treat these programs such OS as a part of them from the point of 4 GB addressable virtrual memory.
The OS or hardware divide the address space into special purposes. E.g., Kernel and User. The logical address range assigned to these other purposes cannot be used by the application.
System Parameters can limit the size of the address space.
Quotas can limit the logical address space for a specific user or process.
Page file space can limit the logical address space for everyone.
I have been reading about how the PCI subsystem gets configured from Bootup, BIOS involvement and mapping of device addresses i.e the BAR's into system Memory.
From the diagram above I am assuming that the address space is physical 4GB RAM with 4GB physical addresses. So, As can be seen above 3GB the device memory is mapped. What happens to this memory on 2GB physical RAM addresses.
If suppose my assumption is wrong and the above map shows virtual address for a 32 bit system. Then how is the device memory mapped to physical addresses for DMA. Is the mapping permanent (non swappable and changeable).
Please help me understand this concept.
If I understand your question, nothing different happens on a 2GB system. There will simply be a "hole" in the physical address space between 2GB and 3GB; that is, there simply won't be a hardware device decoding the addresses here. But otherwise there is no significant difference with respect to PCI devices: they will still be assigned space in the region above 3GB.
It's important to note that the map you show above (physical address space) doesn't necessarily stop at 4GB (since about 1995). Most modern x86 processors have more than 32 address bits. This is why you now often get systems with more than 4GB RAM. And there may be additional holes in the address space above 4GB too.
Actually using the RAM above 4GB requires either the processor's 64-bit mode or PAE (Physical Address Extension) which offers a way to address more than 4GB of physical space in 32-bit mode. [There is also PSE-36 {Page Size Extension} but that's much less commonly used.]
The map you're showing above is specific to physical address space. The x86 virtual address space, (when the processor is operating in 32-bit mode) is 4GB in size, but it does not have all the reserved areas in your diagram. Indeed, the layout of the virtual address space is totally dependent on and determined by the operating system. The usual way that it's configured in linux reserves the part of the virtual address space below the 3GB line for user-mode, and the area above 3GB for kernel-mode. However, this configuration can be changed via the kernel config.
Mapping of the physical address space into virtual address space is managed by the operating system kernel on a page by page basis. A virtual page may be directed either to system RAM or to a PCI device. And note that the page size can vary too, depending on how the processor and page tables are configured.
I thought that virtual address space was a section of RAM allocated to a specific process. But the book I'm reading says that 4 gbs is the standard limit of virtual address space. Isn't that the entire amount of RAM? If that is the case then I'm confused at what virtual address space is. Can anyone enlighten me?
That's the whole point of virtual addresses: The OS handles the physical memory, the process handles its own, virtual memory which is mapped to any memory the OS has available, not necessarily RAM.
On a 32 bit operating system the virtual address space (VAS) is, as you say, usually 4 GiB. 32 bits give you (2^32) addresses (0 ... (2^32)-1), each addressing one byte.
You could have more or less physical RAM and still have a 4-GiB-VAS for each and every process running. If you have less physical RAM, the OS would usually swap to harddrives.
The process doesn't need to know any of this, it can use the full VAS it is given by the OS and it's the OS' job to supply the physical memory.
(This is actually just a dumbed-down version of the Wikipedia article on VAS.)