Assignment 3: Virtual Memory
Contents
- Due Dates and Mark Distribution
- Introduction
- Setting Up
- Tutorial Exercises
- Coding Assignment
- Submission of Simplified Assignment
- Assignment Submission
- Advanced Assignment
1. Due Dates and Mark Distribution
-
Due Dates:
- Simplified Version: 8am, Monday October 25th
- Full Version: 8am, Monday November 1st
-
Marks: Worth 35 marks (of the 100 available for the class mark component of the course)
-
10% of the marks are allocated for submitting a simplified version of the assignment at least one week early. This is explained further below. We won't automark the simplified version, and use a very crude marking scheme: full marks if the code at least looks reasonable (even if it does not compile), 50% (i.e. 5% of overall) if only parts of the simplified assignment are
implemented or code is marginal, 0 marks if nothing reasonable is submitted.
This is basically an easy a way for you to get marks as long as you take the time and seriously think about the problem well before the final deadline: if you are stuck somewhere, add comments to describe the problem. You can also add pseudo-code in comments in case you know only roughly what should happen is a certain function.
- The 10% bonus for one week early is available for the basic assignment. Deadline: 8am, Monday October 25th.
- The extra 5% bonus for a submitted, working assignment within 48-ish hours of release, is also available for the basic assignment. We are stretching this because it's a more difficult assignment. Deadline: 8am, Monday October 11th. See course intro for exact details. Note the demonstration requirement.
- Up to a 20% bonus is available for the advanced part of the assignment To attempt the advanced part, you must finish the standard assignment at least one week early, and get permission from the lecturer in charge, Gabriele Keller. Any bonus awarded for the advanced part can be used to make up for lost marks in any class mark component of the course.
- The familiarisation questions contained herein or the subject of your Week 11 tutorial. Please answer the questions and bring them to your tutorial.
2. Introduction
-
In this assignment you will implement the virtual memory sub-system of OS/161. The existing VM implementation in OS/161, dumbvm, is a minimal implementation with a number of shortcomings. In this assignment you will adapt OS/161 to take full advantage of the simulated hardware by implementing management of the MIPS software-managed Translation Lookaside Buffer (TLB). You will write the code to manage this TLB. You will also write code to manage system memory.
If you attempt the advanced portion of this assignment, you will implement paging, the mechanism by which memory pages of an active process can be sent to disk when memory is needed, and restored to memory when required by the program.
- global: 1 bit; if set, ignore the PID bits in the TLB.
- valid: 1 bit; set if the TLB entry contains a valid translation.
- dirty: 1 bit; enables writing to the page referenced by the entry; if this bit is 0, the page is only accessible for reading.
- nocache: 1 bit; unused in System/161. In a real processor, indicates that the hardware cache will be disabled when accessing this page.
- pid: 6 bits; a context or address space ID that can be used to allow entries to remain in the TLB after a context switch.
- kseg2, TLB-mapped cacheable kernel space
- kseg1, direct-mapped uncached kernel space
- kseg0, direct-mapped cached kernel space
- kuseg, TLB-mapped cacheable user space
The System/161 TLB
-
In the System/161 machine, each TLB entry includes a 20-bit virtual page number and a 20-bit physical page number as well as the following five fields:
The System/161 Virtual Address Space Map
-
The MIPS divides its address space into several regions that have
hardwired properties. These are:
Address | Segment | Special properties |
---|---|---|
0xffffffff | kseg2 | |
0xc0000000 | ||
0xbfffffff | kseg1 | |
0xbfc00180 | Exception address if BEV set. | |
0xbfc00100 | UTLB exception address if BEV set. | |
0xbfc00000 | Execution begins here after processor reset. | |
0xa0000000 | ||
0x9fffffff | kseg0 | |
0x80000080 | Exception address if BEV not set. | |
0x80000000 | UTLB exception address if BEV not set. | |
0x7fffffff | kuseg | |
0x00000000 |
3. Setting Up
-
Make sure you have plenty of disk quota available before you start.
The OS/161 distribution can be copied from the class account into your home direcory. Assuming you've set up your account as describe earlier, use this command:
% cp -r ~cs3231/assigns/asst3/src ~/cs3231/srcYou should now have a src directory to work on. You will need to increase the amount of physical memory to run some of the provided tests. Update your sys161.conf so that the ramsize is as follows
31 busctl ramsize=16777216If you do not have a sys161.conf file you can download this one. Compile and run the kernel. You should get to the menu prompt. If you try and run a program, the kernel should panic with a message about vm_fault being unimplemented.
You are now ready to start the assignment.
4. Tutorial Exercises
-
Please answer the following questions and bring them to your tutorial
in week 11. You should be familiar enough with navigating the kernel
source that you can find the answers to the below questions by yourself
(Hint: use the grep utility). You may also find the
MIPS r3000 reference useful.
- What is the difference between the different MIPS address space segments? What is the use of each segment?
- What functions exist to help you manage the TLB? Describe their use. (Hint: look in kern/arch/mips/include/tlb.h)
- What macros are used to convert from a physical address to a kernel virtual address?
- What address should the initial user stack pointer be?
- What are the entryhi and entrylo co-processor registers? Describe their contents.
- What do the as_* functions do? Why do we need as_prepare_load() and as_finish_load()?
- What does vm_fault do? When is it called?
-
Assuming a 2-level hierarchical page table (4k pages), show for the following virtual addresses:
- The page number and offset;
- the translated address (after any page allocation); and
- the contents of the page table after the TLB miss.
- 0x100008
- 0x101008
- 0x1000f0
- 0x41000
- 0x41b00
5. Coding Assignment
-
Due to the difficulty of this assignment, and the historical behaviour of students attempting to do this assignment at the last minute, we are providing an incentive scheme to encourage early work. It involves submitting a simplified version of the assignment a week early. This is described later in the Hints Section, and instructions on how to submit this are found here. You do not have to implement the simple version if you're confident of completing the full version, but you must submit some version a week early in order to gain the marks.
This assignment involves designing and implementing a number of data-structures. Before you start, you should work out what data you need to keep track of, and what operations are required.
- What information do you need to store for each page?
- How does the page table get populated?
Memory Management
-
This assignment requires you to keep track of physical memory. The
current memory management implementation in dumbvm never frees memory;
your implementation should handle both allocation and freeing of
frames.
In the basic assignment you will need to keep track of whether a frame is used or not - you don't need an extra data structure for this, as you can use the pointer of the collision chain to connect all the free frames in the IPT. In the advanced part, however, you will need to keep track of reference statistics and other information about the frames.
The functions that deal with memory are described in kern/include/vm.h. You may assume that only one page will be allocated at a time --- designing a page allocator that can allocate multiple pages at a time is surprisingly tricky. However, make sure that you never allocate memory (through kmalloc) that is larger than a page!
Note that alloc_kpages() should return the virtual address of
the page, i.e., an address in kseg0.
Warning: alloc_kpages() can be called before vm_bootstrap(). The means that your implementation of alloc_kpages() must work before your free frame information is initialised. You should just call ram_stealmem() if it hasn't been initialised.
Address Space Management
-
OS/161 has an address space abstraction, the struct addrspace. To enable OS/161 to interact with your VM implementation, you will need to fill in the functions in kern/vm/addrspace.c, The semantics of these functions is documented in kern/include/addrspace.h.
You may use a fixed-size stack region (say 16 pages) for each process.
Address Translation
-
The main goal for this assignment is to provide virtual memory translation for user programs. To do this, you will need to implement a TLB refill handler. You will also need to implement a page table. For this assignment, you will implement an inverted page table (IPT).
Hashing functions for TLBs tend to be optimised for speed rather than uniform coverage, and are therefore very simple. An appropriate choice of the hash is the least significant bits of the page number value.
The following questions may assist you in designing the contents of your page table
Testing and Debugging Your Assignment
-
To test this assignment, you should run a process that requires more virtual memory than the TLB can map at any one time. You should also ensure that touching memory not in a valid region will raise an exception. The huge and faulter tests in testbin may be useful.
Apart from GDB, you may also find the trace161 command useful. trace161 will run the simulator with tracing, for example
% trace161 -t t -f outfile kernelwill record all TLB accesses in outfile.
Hints
-
Have a close look at the dumbvm implementation, especially vm_fault(). Although it is simple, you should get an idea on how to approach the rest of the assignment.
We suggest you implement the assignment in the following order:
- Understand how a page table works, and its relationship with the TLB.
- Understand the specification and the supplied code.
-
Work out a basic design for you implementation.
Start simple: assume a small address space. This means you can use a straight-forward static array as the page table (without collision chain), and can keep you code and data structures simple. - Implement the TLB exception handlers in vm.c using this simplified page table. Note, your final solution should use an IPT!
- Implement the functions in kern/vm/addrspace.c that are required for basic functionality (e.g. as_create(), as_prepare_load(), etc.). Allocating user pages in as_define_region() may also simplify your assignment.
- Test and debug this. Use the debugger! Once you're happy with it submit this to asst2_simple as described below. You can submit your full version to asst2_simple if you'd rather not implement the simple version.
- Understand how the inverted page table works.
- Decide exactly what data structures you need.
- Work out the design for the proper solution, using the IPT.
- Modify your implementation to include the IPT.
- Write routines in kern/vm/frametable.c to manage free frames and copy pages. Modify kern/arch/mips/mips/vm.c to create and delete page table entries, and keep the TLB consistent with the page table.
- Use these routines to finish the functions in kern/vm/addrspace.c.
- Test and debug.
6. Submission of Simplified Assignment
-
Once your assignment works, you now need to generate a diff of the changes you made to OS/161. Firstly, you need to clean the source tree.
% cd ~/cs3231/src % make clean % cd kern/compile/ASST3 % make cleanYou will be submitting a diff of your changes to the original tree. So generate a file containing this diff.
% cd ~/cs3231 % diff -N -r -u -X ~cs3231/assigns/asst3/src/.diffex ~cs3231/assigns/asst3/src src > ~/asst3_simple.diffThe arguments to diff are as follows
- -N: Include any new files you add to diff's output.
- -r: Recursively compare directories
- -u: Generate unified diff output.
- -X: Exclude files listed in .diffex from being included in the diff.
Testing Your Submission
-
Look here for information on testing and resubmitting your assignment.
Submitting Your Assignment
-
Now submit the diff as your assignment.
% cd ~ % give cs3231 asst3_simple asst3_simple.diffYou're now done.
7. Assignment Submission
-
Once you've completed and tested the assignment, you now need to generate a diff of the changes you made to OS/161. Again, you need to clean the source tree.
% cd ~/cs3231/src % make clean % cd kern/compile/ASST3 % make cleanYou will be submitting a diff of your changes to the original tree. So generate a file containing this diff.
% cd ~/cs3231 % diff -N -r -u -X ~cs3231/assigns/asst3/src/.diffex ~cs3231/assigns/asst3/src src > ~/asst3.diffNow submit the diff as your assignment.
% cd ~ % give cs3231 asst3 asst3.diffYou're now done.
8. Advanced Assignment
-
Students who wish to attempt the advanced part of this assignment may gain up to an extra 7 marks (20%) for one (or multiple) of the following:
- (easy) Shared pages and copy-on-write.
- (hard) Implement mmap() and related system calls.
- (hard) Implement demand-loading. You should load pages only when they are referenced by the user process, as opposed to at process creation.
- (harder) Implement paging. You should implement some page replacement algorithm and demonstrate your solution running under memory pressure.
Advanced Assignment Submission
-
Submission for the advanced assignment is similar to the basic assignment, except the advance component is given to a distinguished assignment name: asst3_adv.
Once again, you need to clean the source tree.
% cd ~/cs3231/src % make clean % cd kern/compile/ASST2 % make cleanGenerate a file containing these differences.
% cd ~/cs3231 % diff -N -r -u -X ~cs3231/assigns/asst3/src/.diffex ~cs3231/assigns/asst3/src src > ~/asst3_adv.diffTo check your diff file, see the assignment Submission Guidelines Now submit the diff as your assignment. Be careful to submit to the asst3_adv assignment, and not accidently re-submit over your basic assignment submission.
% cd ~ % give cs3231 asst3_adv asst3_adv.diffYou're now done.