I'm reading "Operating System Principles and Practices".
It says that there are hardware virtual pages on top of hardware physical pages. And there is also operating system's virtual pages on top of hardware virtual pages, mapping os virual pages to hardware virtual pages.
So, my question is - if address translation happens when processor accesses memory address, how can processor know about os page mapping to be able to translate addresses correctly?
The operating system has to maintain page tables that defines the mapping from logical pages to physical page frames. The location of the page tables is defined using system registers.
Related
I understand that every process in a computer has an address space, which contains addresses for all the instructions of the process. I also understand that this address is a virtual one - meaning it maps to the physical RAM addresses, and doesn't actually exist.
When a process is to be executed, the CPU sends the MMU the virtual address of the instruction (the page I guess?) that it needs, and then MMU can provide the physical address of it in RAM, which is what the CPU fetches to execute the instruction.
What I don't get is who assigns that virtual address to the process in the first place - how does the CPU know the virtual address? I get that these virtual addresses are generic (I read: Difference between logical addresses, and physical addresses?, and it was very helpful in that aspect), but I don't get who assigns that to begin with. Where even are the page tables stored, and why is the swap space needed?
I've been trying to understand memory mapping for a few days now, mostly through youtube videos, but if anyone knows any good sources I'd really appreciate it!
On older time with virtual address, you have a fixed layout: some data at beginning, the program, the program data, libraries, and than stack. Sometime Kernel had the last bytes reserved. Note: ometime with different layout, but usually same structure on the same OS/version. We had also the possibility to prelink libraries, so you can have quick loading time. Virtual address allow this.
Now it is the kernel that randomize the addresses (to reduce the success rate of buffer overflow attack). So you may have libraries and programs in random places. It is not the MMU, the kernel choose the virtual addresses randomly.
So now you have the virtual address. Now the kernel should assign it to something. If you clone, the virtual address point to the parent process, or when you execute the program, your virtual address point to disk (where the binary reside), and some (the rest, and maybe some guard) are just unassigned, so that program will get a segmentation fault (really a protection exception of CPU). This "physical pointer" could change. Because of swapping the physical address could change, or the pointer could point to different part of disk, but also you may point to old process memory, and then to your modified memory, or to different part of disk.
So virtual addresses are defined by kernel and kernel will handle it (either having a physical address, or an exception, which kernel may check a different source [other process memory, disk]). If the page (virtual address block) has linked a physical address, it is the MMU which do the works (and updating also access count of the page), else it is the kernel (and so much slower).
I was reading the textbook:Computer Systems A Programmer’s Perspective, in chapter 9.7.2:Linux Virtual Memory System (third edition) that talks about virtual memory.
I was a bit confused by the structure of virtual memory for linux process as shown below:
My question is: does kernel virtual memory preserve for kernel to run
and rest of the virtual memory preserve for user process? What does kernel code and data do? And what does the physical memory in kernel virtual memory?
does kernel virtual memory preserve for kernel to run and rest of the virtual memory preserve for user process?
Yes, there is a part of virtual memory that is always reserved for the kernel and another part that is left available to userspace processes. Every single process has its own virtual memory, but the kernel is always mapped in the higher part (higher addresses) of virtual memory. Whether or not this mapping is visible to the process depends on Kernel Page Table Isolation.
See also: Do the virtual address spaces of all the processes have the same content in their “Kernel” parts?
What does kernel code and data do?
Part of the high virtual memory is a direct mapping of the actual kernel image. That is, the kernel executable and all its data. You can see it in more detail here in this page of the kernel documentation, marked as "kernel text mapping, mapped to physical address 0".
See also: What's the use of having a kernel part in the virtual memory space of Linux processes?
And what does the physical memory in kernel virtual memory?
That part of the image is totally misleading. I don't know precisely what information the authors of the book were trying to convey, but physical memory is definitely not a part of kernel virtual memory. They were probably trying to address the fact that there is a direct mapping of all physical memory in the kernel virtual memory, which can be seen again on the same page of the kernel documentation, marked as "direct mapping of all physical memory".
Physical memory refers to the real memory of the system (i.e. the RAM). Each region of virtual memory is mapped to some region of physical memory. This virtual-to-physical mapping is totally transparent to processes and is managed by the kernel. For example, two executables that have the same file open in read-only mode are usually sharing the same physical memory region, while seeing two different virtual address.
This is a more accurate depiction of the relationship between virtual and physical memory:
Source: https://computationstructures.org/lectures/vm/vm.html
cited from the CSAPP book, 3rd version, section 9.7.2, where the picture is shown.
Interestingly, Linux also maps a set of contiguous virtual pages (equal in size to the total amount of DRAM in the system) to the corresponding set of contiguous physical pages. This provides the kernel with a convenient way to access any specific location in physical memory—for example, when it needs to access page tables or to perform memory-mapped I/O operations on devices that are mapped to particular physical memory locations.
I think the Physical memory in the picture just reflects what's described above: a virtual memory area that maps to the entire physical memory.
Who divides the main memory into page frames? From my current understanding it's OS that maintains the frame table, so it's not MMU for this job?
The MMU is just a hardware thing. Today, it is a part of the CPU chip. Details about the MMU are instruction set architecture specific (so different on x86 and on PowerPc or ARM or RISC-V).
The MMU needs to be configured, to implement suitable virtual address to physical address translation.
That MMU configuration (a privileged operation) is done by the operating system kernel which manages the page table. From the user-space point of view, the OS would provide system calls (such as mmap(2) on Linux) to change the virtual address space of the process.
Read Operating Systems: Three Easy Pieces for more (freely downloadable). There are several chapters (12 to 24) explaining all this.
PS. If you have a Linux system, try the cat /proc/$$/maps command (it "shows" the virtual address space of your shell process, since your shell would expand $$ to its pid), and read more about proc(5).
Who divides the main memory into page frames?
The system hardware divides the main memory into physical page frames.
From my current understanding it's OS that maintains the frame table, so it's not MMU for this job?
No. The operating system maps a process's logical pages to the physical page frames. The underlying hardware divides the memory into page frames. This may or may not be done by the MMU. It would all depend upon how the chipset is designed.
I have been reading about demand paging and there are a few terminologies I don't understand.
What is a frame? I read that it is a block of physical memory which can at least fit in a page ( so a frame can fit one or more pages? ). But does this physical memory refer to the RAM or the disk storage?.
Which one of these is true:
The virtual address space ( which is 4 GiB in 32 bit systems ) is allocated for one application at a time, so that every application has 4 GiB virtual address to access to, and each time we switch application, the OS reconfigures the virtual address space to map to other other applications. Or the virtual address space is allocated to several processes? If so, how much virtual memory does each application get and what happen when it wants more virtual memory?
Do we have a page table for each application running, or a common page table for all applications?
Where does virtual memory fragmentation come from ?
I hope someone can clarify me.
A frame is a block of physical memory, RAM. I've not heard of frames being larger than pages, I've always understood them synonymous. However, a CPU may allow for frames/pages of different sizes to coexist simultaneously (e.g. large pages of 4MB/2MB/1GB size and regular 4KB pages on x86).
Whether there's a single address space shared by multiple applications or each has its own address space depends on the OS. Windows 3.xx and 9x/Me had a shared address space. Windows NT/2000/XP/etc had individual, per-app address spaces. Not all of the address space is available to an application / applications. A portion is reserved for the OS (kernel, drivers, their data).
Should be obvious now. One note though... Even with individual address spaces a portion of memory can still be made available in several different address spaces and it may be done by having a common page table in the respective processes. Also, it's very typical for the kernel portion of the address space to be managed by several page tables common to all processes.
Whether the address space is virtual or not, it can become fragmented. You may want to allocate a contiguous (in terms of the virtual addresses) buffer of, say, 8KB, but you may only have two non-adjacent 4KB regions available.
In a modern PC, where will
MOV [0x0000], 7
put a 7? Is it the first byte of my RAM, or is it the first byte of the process's address space? Assuming it triggers a memory violation.
You mean assuming it doesn't trigger an access violation? Every process has it's own virtual address space. The first 64kiB are normally kept unmapped, so NULL-pointer accesses actually fault noisily, instead of letting programs silently do Bad Things.
In a user-space process on a typical OS, an absolute address of 0 does refer to the first byte of your process's virtual address space.
With paging enabled, there's no way even for the kernel to use physical addresses directly. To write to a given physical address, would have to create a page table entry mapping that physical page to a virtual page (or find an existing mapping), invlpg to make sure the TLB isn't caching a stale entry, and then use that virtual address.
it depends on the system architecture. Every architecture provides an instruction set and a memory layout. Furthermore it depends on the operating system you use. E.g. Real Time Operating systems often do not provide Virtual Memory.
greets