I am new to Operating Systems and I was following this handbook which I am reading to get a better grasp on how computers really work. However having stumbled upon Memory Management I found it somewhat confusing when allocating pages and frames using page tables to virtual memory locations.
Say Page/Frame size is 4K and the system is 32-bit (it handles addresses 4byte long and page entries are also 4byte long).
If the same Process references two different memory locations for its needs say Virtual_MemoryLocation_A is 222 and Virtual_MemoryLocation_B is 999 will these memory requests be pinned to the same Page 0 (first Page in the Page table) since both do not extend beyond the frane size scope of 4096 bytes and be granted memory by the OS to the same Frame X?
If you have a page size of 4096, then
page number = address DIV 4096
page offset = address MOD 4096
Those two values uniquely identify the logical memory location.
Two addresses can be in the same frame. If that were not the case, there would be no point in having pages. All memory locations from 0 .. 4095 are in the same, zeroth page.
Related
I am given a system with 64-bit virtual address space. with page size of 2KB.
Also it is given that the physical memory is of the size 16GB.
I need to calculate the following parameters:
number of page entries (number of lines in the page table), how many bits are needed for the page offset, how many bits are needed for the virtual page number (VPN), and how many bits are needed for the physical page number (PPN).
So, first I concluded that the size of the virtual memory is 2^64 bytes, and that means there are 2^53 entries in the page table.
From the size of a page I concluded that 11 bits are needed for the page offset.
From here I'm not so sure.
Since each virtual address is of the size 64 it, then the VPN is of the size 64 - 11 = 53 bits.
Since the physical memory is of the size 2^34 bytes, then a physical address if of 34 bits. Which means the PPN is of the size 34 - 11 = 23 bits.
Are my calculations correct? and also is my thinking correct?
Help would be appreciated
Some of your results are correct. PPN is 23 bits, VPN is 53 bits.
But all the stuff concerning the page tables is wrong.
A page table contains a set of physical page adresses. Hence as a PPN is 23 bits, one needs 4 bytes (the power of 2 above 23) to describe a PP. If pages are 2k bytes, you can store 2^9 PP adresses par page.
As VPN are 53bits, and each table can resolve 9 bits, the translation can be done by 6 consecutive tables.
If you are not familiar with multilevel pages, there are many good tuturials. See for instance https://en.wikipedia.org/wiki/Page_table
What is certain is that the PT size is NOT 2^53!! First because 2^55 is an insane amount of memory (~10^16). And second, because the total number of PP is 2^23, so why use a table 1 billion times larger... (and this is why we use multilevel page tables)
Please explain it nicely. Don't just write definition. Also explain what it does and how is it different from segmentation.
Fragmentation needs to be considered with memory allocation techniques. Paging is basically not a memory allocation technique, but rather a means of providing virtual address spaces.
Considering the comparison with segmentation, what you're probably asking about is the difference between a memory allocation technique using fixed size blocks (like the pages of paging, assuming 4KB page size here) and a technique using variable size blocks (like the segments used for segmentation).
Now, assume that you directly use the page allocation interface to implement memory management, that is you have two functions for dealing with memory:
alloc_page, which allocates a single page and returns a pointer to the beginning of the newly available address space, and
free_page, which frees a single, allocated page.
Now suppose all of your currently available virtual memory is used, but you need to store 1 additional byte. You call alloc_page and get a 4KB block of memory. You only use 1 byte of that huge block, but also the other 4095 bytes are, from the perspective of the allocator, used. If this happens multiple times eventually all pages will be allocated, so further calls to alloc_page will fail. Even if you just need another additional byte (which could be one of the 4095 that got wasted above) the allocator will tell you that you're out of memory. This is internal fragmentation.
If, on the other hand, you would use variable sized blocks (like in segmentation), then you're vulnerable to external fragmentation: Suppose you manage 6 bytes of memory (F means "free"):
FFFFFF
You first allocate 3 bytes for a, then 1 for b and finally 2 bytes for c:
aaabcc
Now you free both a and c, leaving only b allocated:
FFFbFF
You now have 5 bytes of unused memory, but if you try to allocate a block of 4 bytes (which is less than the available memory) the allocation will fail due to the unfavorable placement of the memory for b. This is external fragmentation.
Now, if you extend your page allocator to be able to allocate multiple pages and add alloc_multiple_pages, you have to deal with both internal and external fragmentation.
There is no external fragmentation in paging but internal fragmentation exists.
First, we need to understand what is external fragmentation. External fragmentation occurs when we have a memory to accommodate a process but it's not continuous.
How does it not occur in paging?
Paging divides virtual memory or all processes into equal-sized pages and physical memory into fixed size frames. So you are typically fixing equal size blocks called pages into equal block shaped spaces called frames! Try to visualize and conclude that there can never be external fragmentation.
In the case of segmentation, we divide virtual addresses into different sized blocks that is why there may be the case some blocks in main memory must stick together or compact to make space for the new process! I hope it helps!
When a process is divided into fix sized pages, there is generally some leftover space in the last page(internal fragmentation). When there are many processes, each of their last page's unused area could add up to be greater than or equal to size of one page. Now even if you have to total free size of one page or more but you cannot load a new page because a page has to be continuous. External fragmentation has happened. So, I don't think external fragmentation is completely zero in paging.
EDIT: It is all about how External Fragmentation is defined. The collection of internal fragmentation do not contribute to external fragmentation. External fragmentation is contributed by the empty space which is EXTERNAL to partition(or page). So if suppose there are only two frames in main memory ,say of size 16B, each occupied by only 1B data. The internal fragmentation in each frame is 15B. The total unused space is 30B. Now if you want to load one new page of some process, you will see that you do not have any frame available. You are unable to load a new page eventhough you have 30B unused space. Will you call this as external fragmentation? Answer is no. Because these 15B unused space are INTERNAL to the pages. So in paging, internal fragmentation is possible but not external fragmentation.
Paging allows a process to be allocated physical memory in non-contiguous fashion. I will answer that why external fragmentation can't occur in paging.
External frag occurs when a process, which was allocated contiguous memory , is unloaded from physical memory, which creates a hole (free space ) in the memory.
Now if a new process comes, which requires more memory than this hole, then we won't be able to allocate contiguous memory to that process due to non contiguous nature of free memory, this is called external fragmentation.
Now, the problem above originated due to the constraint of allocating contiguous memory to the process. This is what paging solved by allowing process to get non contiguous physical memory.
In paging, the probability of having external fragmentation is very low although internal fragmentation may occur.
In paging scheme, the whole main memory and the virtual memory is divided into some fixed size slots which are called pages (in case of virtual memory) and page frames (in case of main memory or RAM or physical memory). So, whenever a process is executed in main memory, it occupies the entire space of a page frame. Let us say, the main memory has 4096 page frames with each page frame having a size of 4096 bytes. Suppose, there is a process P1 which requires 3000 bytes of space for its execution in main memory. So, in order to execute P1, it is brought from virtual memory to main memory and placed in a page frame (F1) but P1 requires only 3000 bytes of space for its execution and as a result of which (4096 - 3000 = 1096 bytes) of space in the page frame F1 is wasted. In other words, this denotes the case of internal fragmentation in the page frame F1.
Again, external fragmentation may occur if some space of the main memory could not be included in a page frame. But this case is very rare as usually the size of a main memory, the size of a page frame as well as the total no. of page frames in main memory can be expressed in terms of power of 2.
As far as I've understood, I would answer your question like so:
Why is there internal fragmentation with paging?
Because a page has fixed size, but processes may request more or less space. Say a page is 32 units, and a process requests 20 units. Then when a page is given to the requesting process, that page is no longer useable despite having 12 units of free "internal" space.
Why is there no external fragmentation with paging?
Because in paging, a process is allowed to be allocated spaces that are non-contiguous in the physical memory. Meanwhile, the logical representation of those blocks will be contiguous in the virtual memory. This is what I mean:
A process requires 128 units of space. This is 4 pages as in the previous example. Unregardless of the actual page numbers (formally frame numbers) in the physical memory, you give those pages the numbers 0, 1, 2, and 3. This is the virtual representation that is the defining characteristic of paging itself. Those pages may be 21, 213, 23, 234 in the actual physical memory. But they can really be anything, contiguous or non-contiguous. Therefore, even if paging leaves small free spaces in between used spaces, those small free spaces can still be used together as if they were one contiguous block of space. That's why external fragmentation won't happen.
Frames are allocated as units. If the memory requirements of a process do not happen to coincide with page boundaries, the last frame allocated may not be completely full.
For example, if the page size is 2,048 bytes, a process of 72,766 bytes will need 35 pages plus 1,086 bytes. It will be allocated 36 frames, resulting in internal fragmentation of 2,048 - 1,086 = 962 bytes. In the worst case, a process would need 11 pages plus 1 byte. It would be allocated 11 + 1 frames, resulting in internal fragmentation of almost an entire frame.
Excluding the valid, dirty and reference bits, considering only actual "mapping" from virtual address space to physical address space, why is it said that the size of a page table entry is determined by the number of bits required to reference a page from the main memory (Like here: https://stackoverflow.com/a/14770650/3684931)
My argument is that since a physical address can be in secondary storage too (which is the point of using virtual memory), the size of a page table entry should simply be equal to the number of bits to required to reference any page among all the pages in the virtual memory.
To give an example, if virtual address space is addressable by 64 bits, main memory is addressable by 48 bits and page size is 16KB (addressable by 14 bits), a page table should map (64 - 14) 50 bit addresses to (64 - 14) 50 bit addresses and not (48 - 14) 34 bit addresses.
It could map to 34 bit addresses if the page exists in main memory, but otherwise, the upperbound should be 50 bits which should be considered while calculating the size of a page table.
Am I missing something here?
Page table entries are sized to support the virtual address size and the maximum amount of physical memory supported. They are not sized based on any aspect of the secondary storage.
In your example, the page table has to support mapping 2^50 virtual pages to a possible 2^34 physical pages. Thus a page table entry will use 34 bits to hold the physical page number.
If a page is not present in memory, and it was previously paged out to secondary storage, then a data structure (like a hash table) can be used to locate where in the paging file the page is located. You don't need to use the page table structure to do this.
The size of a page table must account for:
1. Control bits
2. Access restriction bits
3. The page size
4. The number of bits requires to access the required number of pages.
The page table entries correspond to the virtual memory. The number of entries times the page size is the virtual address size.
The entries themselves only need to address physical memory.
I was reading up on Virtual Memory and from what I understand is that each process has its own VM table that maps VM addresses to Physical Addresses in real memory. So if a process allocated objects continuously they can potentially be stored in completely different places in Physical Memory. My question is that if I allocate and array which is supposed to be stored in a contiguous block of memory and if the size of the array requires more space than one page can provide, from what I understand is that array will be stored contiguously in VM but possibly in completely different location in PM. Is this correct? please correct me if I misunderstood how VM works. And if it is correct does that mean we are only concerned whether allocation is contiguous in VM?
Whether or not something that overlaps a page boundary is actually contiguous in Physical Memory is never really knowable with modern memory handlers. Memory glue logic essentially treats all addressable memory pages as an unordered set, and the ordering is essentially associated with a process; there's no guarantee that for different processes that end up getting assigned the same two physical memory pages (at different points in time) that the expressed relationship between those physical pages will be the same. Effectively, there's a translation layer between the CPU and the memory that handles this stuff.
That's right. Arrays must only looks contiguous for your application, but may be physically scattered on memory.
I just wanted to add/make it clear that from a user space program's point of view, a chunk of allocated memory always appears contiguous. The operating system in conjunction with the CPU's Memory Management Unit (MMU) handles all virtual to physical memory mappings and the programmer never needs to worry about how this mapping is handled (unless, of course, said programmer is writing an operating system).
A compiler (or one who writes code in assembly) can treat a program's addresses as starting from 0 and going up until the largest address needed for that particular program. The operating system then creates a page table for each process and uses this table to partially decode a physical address for each virtual memory location. The OS treats an address in a program as two separate parts, the page address and the offset into that page. Then, the MMU translates a page address into a physical frame address. Note that a physical memory "frame" is analogous to the conceptual "page" from the standpoint of the OS; these two are of the same size (eg 4096 bytes).
Since physical memory is divided into equally sized frames, and page size is the same as frame size you can know how much of your virtual address is used as a page location and how much is an offset into that page. For instance, if your OS "allocates" 4 gigabytes to each process (as is the case in Linux), and your page/frame size is 4096 bytes, you can know that 20 bits (4,294,967,296 bytes / 4096 bytes = 2 ^ 20 = 1,048,576 pages/page addresses) of a 32 bit address are used as a page address, which will then be converted to a physical frame address by the MMU, and the remaining 12 bits are used as an offset to determine the location of the address starting from the beginning of the page/frame.
VM (user pace) address --> page + offset (OS) --> frame + offset (MMU) = physical address
Consider a virtual memory system with a 38-bit virtual byte address, 1KB pages and 512 MB of physical memory. What is the total size of the page table for each process on this machine, assuming that the valid, protection, dirty and use bits take a total of 4 bits, and that all the virtual pages are in use? (assume that disk addresses are not stored in the page table.)
Well, if the question is simply "what is the size of the page table?" irrespective of whether it will fit into physical memory, the answer can be calculated thus:
First physical memory. There are 512K pages of physical memory (512M / 1K). This requires 19 bits to represent each page. Add that to the 4 bits of accounting information and you get 23 bits.
Now virtual memory. With a 38-bit address space and a 10-bit (1K) page size, you need 228 entries in your page table.
Therefore 228 page table entries at 23 bits each is 6,174,015,488 bits or 736M.
That's the maximum size needed for a single-level VM subsystem for each process.
Now obviously that's not going to work if you only have 512M of physical RAM so you have a couple of options.
You can reduce the number of physical pages. For example, only allow half of the memory to be subject to paging, keeping the other half resident at all time. This will save one bit per entry, not really enough to make a difference.
Increase the page size, if possible. A 1K page on a 38-bit address space is the reason for the very chunky page tables. For example, I think the '386, with its 32-bit address space, uses 4K pages. That would result in a million page table entries, far less than the 260 million required here.
Go multi-level. A bit more advanced but it basically means that the page tables themselves are subject to paging. You have to keep the first level of page tables resident in physical memory but the second level can go in and out as needed. This will greatly reduce the physical requirements but at the cost of speed, since two levels of page faults may occur to get at an actual process page (one for the secondary paging tables then one for the process page).
Let's look a little closer at option 3.
If we allow 32M for the primary paging table and give each entry 4 bytes (32 bits: only 23 are needed but we can round up for efficiency here), this will allow 8,388,608 pages for the secondary page table.
Since each of those secondary page table pages is 1K long (allowing us to store 256 secondary page table entries at 4 bytes each), we can address a total of 2,147,483,648 virtual pages.
This would allow 8,192 fully-loaded (i.e., using their entire 28-bit address space) processes to run side by side, assuming you have a fair chunk of disk space to store the non-resident pages.
Now obviously the primary paging table (and the VM subsystem, and probably a fair chunk of the rest of the OS) has to stay resident at all times. You cannot be allowed to page out one of the primary pages since you may well need that page in order to bring it back in :-)
But that's a resident cost of only 32M of the 512M for the primary paging table, much better than the (at a minimum, for one fully-loaded process) of 736M.
size of the page table= total no of page table entries*size of the page table entry
STEP 1:FINDING THE NO OF ENTRIES IN PAGE TABLE:
no of page table entries=virtual address space/page size
=2^38/2^10=2^28
so there are 2^28 entries in the page table
STEP2:NO OF FRAMES IN PHYSICAL MEMORY:
no of frames in the physical memory=(512*1024*1024)/(1*1024)=524288=2^19
so we need 19 bits and additional 4 bits for valid, protection, dirty and use bits
totally 23 bits=2.875 bytes
size of the page table=(2^28)*2.875=771751936B=736MB
1KB pages = 2^10, 512MB = 2^29 => Offset = 29 - 10 = 19 bit.
virtual includes two part: page frame + offset => page frame + dirty bit = 38 - 19 = 29 bit.
29 bit includes 4 bit dirty (above) => 25 bit for real page frame, each page frame has 10 bit long.
So, page table size: 2^25 * 10 = 320M.
Hope this correct.