1
00:00:00,210 --> 00:00:06,450
Hi, in the following lectures we will delve into the essential part of the memory manager. We have collected the free memory 

2
00:00:06,450 --> 00:00:09,080
and saved them in the linked list.

3
00:00:09,630 --> 00:00:15,180
In this video, we are going to remap our kernel to the same address space, but we will use 2m pages 

4
00:00:15,180 --> 00:00:20,040
instead of 1g page which is used by the kernel so far.

5
00:00:20,730 --> 00:00:26,430
To do the kernel remap, we need to set up the corresponding table entries.  We have learned 

6
00:00:26,430 --> 00:00:28,410
the structure of the translation tables. 

7
00:00:29,190 --> 00:00:31,980
So let's implement paging using these tables.

8
00:00:33,490 --> 00:00:38,230
The tables used in this example are pml4 table, 

9
00:00:39,250 --> 00:00:43,360
page directory pointer table and page directory table. 

10
00:00:44,270 --> 00:00:48,590
The entry in the page directory table will point to 2m physical page.

11
00:00:50,320 --> 00:00:51,820
OK, let's open the project.

12
00:00:53,750 --> 00:00:55,490
We open the memory.c file.

13
00:00:58,490 --> 00:01:02,300
First off, let’s see what functions we will implement in this file. 

14
00:01:05,870 --> 00:01:13,970
The first function is find pml4 table entry, as its name implies, find the specific entry according to the virtual address

15
00:01:13,970 --> 00:01:14,920
.

16
00:01:15,500 --> 00:01:19,190
The entry we get is pointing to page directory pointer table. 

17
00:01:20,550 --> 00:01:28,140
Then function find pdpt entry is used to find the entry in the directory pointer table which points to 

18
00:01:28,140 --> 00:01:29,610
page directory table.

19
00:01:30,590 --> 00:01:36,140
And in the page directory table, we can set the corresponding entry to map the addresses to the physical page. 

20
00:01:37,390 --> 00:01:39,400
So we use map pages 

21
00:01:41,740 --> 00:01:44,680
to do the mapping.

22
00:01:45,720 --> 00:01:52,050
The function setup kernel virtual memory is used to remap our kernel using 2m pages 

23
00:01:52,050 --> 00:01:58,050
and switch vm is used to load cr3 register with the new translation table to make the mapping work. 

24
00:01:59,270 --> 00:02:06,410
Ok we start from function initialize kvm which then call setup kvm. Because we have collected the physical memory

25
00:02:06,410 --> 00:02:12,320
when we call this function, we can allocate a new free memory page using function 

26
00:02:12,320 --> 00:02:12,860
kalloc. 

27
00:02:13,970 --> 00:02:20,390
This page is actually the new page map level 4 table.  Since what we do here is initializing kernel vm, 

28
00:02:20,390 --> 00:02:24,220
if the allocation failed we simply stop the system.

29
00:02:24,680 --> 00:02:27,440
So here we use assert to test the condition. 

30
00:02:29,230 --> 00:02:34,620
If the check pass, we zero the page using memset function because the page could include random values 

31
00:02:34,630 --> 00:02:38,260
when they get the page from function kalloc.

32
00:02:39,850 --> 00:02:42,700
The function map pages take a few parameters. 

33
00:02:43,500 --> 00:02:49,850
The first one is pml4 table. Next two parameters are the start and end address of memory region we want to map.  

34
00:02:49,890 --> 00:02:56,350
The arguments we pass in this example are the base of the kernel 

35
00:02:56,350 --> 00:02:59,360
and the end of free memory, both of which are virtual addresses.

36
00:03:00,070 --> 00:03:02,280
The next one is the start of physical page we want to map into. 

37
00:03:02,310 --> 00:03:08,680
Because we want to map the kernel to the same physical address, 

38
00:03:08,680 --> 00:03:10,570
we pass physical address of kernel base in this case. 

39
00:03:12,020 --> 00:03:18,110
The last one holds the attribute of the page table or physical page.  If the mapping failed, it returns false 

40
00:03:18,110 --> 00:03:18,550
.

41
00:03:19,750 --> 00:03:22,360
Here we simply add assert to test the conditions. 

42
00:03:23,680 --> 00:03:25,360
Because we used Boolean type, 

43
00:03:28,830 --> 00:03:31,050
we add the header file stdbool. 

44
00:03:32,140 --> 00:03:32,590
OK.

45
00:03:36,000 --> 00:03:39,890
These are macros we defined in the header file. So let’s take a look.

46
00:03:45,600 --> 00:03:49,650
As you see the kernel base is the address we have seen in the last lecture. 

47
00:03:50,790 --> 00:03:56,280
The p w u and entry are the attributes of the table entries. 

48
00:03:57,550 --> 00:03:59,920
They are in decimal or hexadecimal formats.

49
00:04:02,090 --> 00:04:07,190
If you convert them to binary forms, you can see they just set the corresponding bits in the table entries 

50
00:04:07,190 --> 00:04:08,030
.

51
00:04:10,600 --> 00:04:17,470
The pde address and pte address are used to retrieve the address of the next level page or physical page. 

52
00:04:18,920 --> 00:04:25,910
The entries in page map level 4 table and page directory pointer table have attributes within the lower 12 bits. 

53
00:04:28,240 --> 00:04:32,500
So we clear the lower 12 bits of the entry to get the correct address. 

54
00:04:34,950 --> 00:04:38,430
The page table entry has attributes within the 21 bits. 

55
00:04:39,790 --> 00:04:43,390
So we clear 21 bits of the address to get the physical address.

56
00:04:44,230 --> 00:04:45,360
Alright, let's move on.

57
00:04:47,220 --> 00:04:53,820
These three customized data types are used when finding the translation tables. 

58
00:04:53,820 --> 00:04:56,070
The pdptr points to page directory 

59
00:04:56,070 --> 00:04:58,970
and page directory points to page directory entry, etc. 

60
00:05:00,050 --> 00:05:01,790
Ok let’s back to c file. 

61
00:05:04,760 --> 00:05:11,360
So here we specify that the kernel memory is readable, writeable and not accessible by the user applications. 

62
00:05:12,450 --> 00:05:13,380
Let's move to

63
00:05:15,220 --> 00:05:23,110
map pages function. In the function, we define two variables vstart and vend 

64
00:05:23,110 --> 00:05:24,040
which save the aligned virtual addresses. 

65
00:05:25,190 --> 00:05:31,060
For example, if we get unaligned start virtual address, we need to map the page where the start address is located

66
00:05:31,070 --> 00:05:32,120
.

67
00:05:32,780 --> 00:05:38,420
So here we align the address to the previous 2m page so that we can include the start address. 

68
00:05:39,120 --> 00:05:40,640
The same thing happens with end address

69
00:05:40,640 --> 00:05:41,030
.

70
00:05:41,450 --> 00:05:47,130
In this case, we align the address to the next 2m page to include the page where the end address is located

71
00:05:47,180 --> 00:05:48,250
.

72
00:05:49,810 --> 00:05:57,610
Then we define variable pd which is used to set the page directory entry.  The index is used to locate the specific entry in the table. 

73
00:05:57,610 --> 00:06:01,570
In this example, we also perform some checks. 

74
00:06:02,610 --> 00:06:08,580
The first one checks the start and end of the region. The next one is to make sure 

75
00:06:08,580 --> 00:06:11,420
the physical address we want to map into is page aligned. 

76
00:06:12,000 --> 00:06:17,400
The last one checks to see if the end of physical address is outside the range of 1g memory 

77
00:06:17,400 --> 00:06:17,880
.

78
00:06:18,830 --> 00:06:20,630
OK, now we do the mapping.

79
00:06:21,830 --> 00:06:26,630
The first thing we are going to do is we are going to find the page directory pointer table entry 

80
00:06:26,930 --> 00:06:33,640
which points to page directory table by calling function find pdpt entry.  We will see this function in a minute 

81
00:06:33,640 --> 00:06:34,210
.

82
00:06:34,790 --> 00:06:39,320
If it returns null, it means that it failed and we simply return false. 

83
00:06:40,830 --> 00:06:47,700
If it returns a valid table, we use the index to locate the correct page entry according to the virtual address. 

84
00:06:49,100 --> 00:06:54,860
We have learned that the virtual address is divide into several parts to locate the different table entries.

85
00:06:55,730 --> 00:06:59,050
Here the table entry we want is page directory entry, 

86
00:06:59,510 --> 00:07:04,280
the index value is 9 bits in total starting from bits 21. 

87
00:07:04,690 --> 00:07:10,280
So we shrift right 21 bits and clear other bits except the lower 9 bits of the result.

88
00:07:11,720 --> 00:07:16,010
The value in the index is used to find the entry in the page directory table. 

89
00:07:17,850 --> 00:07:23,850
Here we check the present bit of the entry. If the present bit is set, it means that we remap to the used page. 

90
00:07:23,850 --> 00:07:27,540
We don’t allow it to happen in our system.  

91
00:07:27,960 --> 00:07:29,700
So we use assert to check that.

92
00:07:31,190 --> 00:07:36,710
If the check pass, what we are going to do next is we are going to set the entry 

93
00:07:36,710 --> 00:07:37,920
with the physical address and attributes. 

94
00:07:38,750 --> 00:07:44,360
Don't forget to add the attribute page entry to indicate that this is 2m page translation.

95
00:07:45,260 --> 00:07:47,450
Ok now we have mapped one page. 

96
00:07:48,760 --> 00:07:53,560
Then we move to the next page by adding the page size to the virtual address and physical address. 

97
00:07:54,770 --> 00:08:00,140
If the virtual address is still within the memory region, we continue the process until we reach 

98
00:08:00,140 --> 00:08:00,920
the end of the region.

99
00:08:02,910 --> 00:08:06,360
Ok let’s see the function find pdpt entry. 

100
00:08:10,240 --> 00:08:17,680
The parameters it takes are the page map level 4 table, virtual address and alloc indicating 

101
00:08:17,680 --> 00:08:20,220
whether or not we will create a page if it does exist. The last one is attribute.

102
00:08:21,390 --> 00:08:22,770
The last one is attribute.

103
00:08:25,920 --> 00:08:32,710
You can see we pass the value 1 to the function. So we will create a page if it does exist. 

104
00:08:33,559 --> 00:08:35,640
Ok, in the function find pdpt entry, 

105
00:08:37,130 --> 00:08:45,460
we define the variable pdptr pd and index. Since we find the pdpt entry in this case, the index value 

106
00:08:45,470 --> 00:08:51,290
is located from the bits 30 of the virtual address and is also 9 bits in total. 

107
00:08:51,950 --> 00:08:54,980
So we preserve the low 9 bits of the result.

108
00:08:56,780 --> 00:09:03,680
Then we call function find pml4 table entry to get the page directory pointer table and we check if it returns null. 

109
00:09:03,680 --> 00:09:07,490
If it returns a valid table, 

110
00:09:07,520 --> 00:09:10,790
we will find the correct entry in the table using index value. 

111
00:09:11,730 --> 00:09:17,250
If the present attribute is 1, then it means that the value in the entry points to the next level table 

112
00:09:17,760 --> 00:09:19,580
which is page directory table.  

113
00:09:20,280 --> 00:09:25,860
We use macro pde address to clear the attribute bits to get the address. 

114
00:09:26,430 --> 00:09:32,700
Note that here we use p2v to covert the physical address to virtual address because the addresses in the table entries 

115
00:09:32,700 --> 00:09:34,960
are physical addresses. 

116
00:09:35,580 --> 00:09:36,690
OK, let's continue.

117
00:09:39,790 --> 00:09:46,600
If the present bit is cleared, then it’s an unused entry. We will check the value of alloc.

118
00:09:47,050 --> 00:09:53,310
If it’s equal to 1, we will allocate a new page and set the entry to make it point to this page.

119
00:09:54,250 --> 00:10:00,490
Also, we use v2p to convert the virtual address to physical address and save it to the entry. 

120
00:10:01,720 --> 00:10:05,140
In the end we return the address of page directory table.

121
00:10:06,200 --> 00:10:09,470
The function find pml4 table entry 

122
00:10:10,930 --> 00:10:14,110
has the same parameters as find pdpt entry. 

123
00:10:15,670 --> 00:10:20,960
The pml4 table holds the entries which point to the page directory pointer tables. 

124
00:10:21,670 --> 00:10:22,960
So we define the variable map entry 

125
00:10:22,960 --> 00:10:28,530
which is a pointer to type pdptr and variable pdptr. 

126
00:10:29,860 --> 00:10:36,790
Since we find the entry in pml4 table, the index is located from bits 39 of the virtual address

127
00:10:36,790 --> 00:10:37,200
.

128
00:10:38,870 --> 00:10:43,060
Then we locate the correct entry and check to see if the present bit is set.

129
00:10:44,140 --> 00:10:47,620
If it is set, we copy the address to pdpt. 

130
00:10:49,040 --> 00:10:55,190
otherwise we will check the alloc. If it is 1, we will allocate a new page and set the entry 

131
00:10:55,190 --> 00:10:56,070
to point to the page. 

132
00:10:57,080 --> 00:10:59,870
This process is pretty much the same as in function 

133
00:11:00,830 --> 00:11:02,420
find pdpt entry. 

134
00:11:06,230 --> 00:11:09,230
At the end of the function, we return pdptr.

135
00:11:10,390 --> 00:11:12,700
OK, the last function we implement

136
00:11:16,200 --> 00:11:20,390
is switch vm. With the translation table prepared, 

137
00:11:20,440 --> 00:11:23,450
we need to load cr3 register with the new pml4 table

138
00:11:23,460 --> 00:11:24,510
.

139
00:11:25,480 --> 00:11:26,060
Here we convert 

140
00:11:26,080 --> 00:11:31,360
the virtual address to physical address because cr3 register stores the physical address of the table  

141
00:11:31,360 --> 00:11:31,840
.

142
00:11:32,710 --> 00:11:36,220
We haven’t written load cr3 function. Let’s do it now. 

143
00:11:38,600 --> 00:11:40,940
We add it in the trap assembly file.  

144
00:11:42,200 --> 00:11:44,210
So we opened trap assembly file.

145
00:11:48,350 --> 00:11:55,190
Here is load cr3 function we use in the file. In this function, we just load the address to cr3 register and return

146
00:11:55,190 --> 00:11:55,850
.

147
00:11:56,960 --> 00:12:00,950
Also, we declare it global to make it accessible in the c file.

148
00:12:03,800 --> 00:12:10,460
Since it is defined in the assembly file and we will use it in the c file.

149
00:12:10,460 --> 00:12:13,820
we add the declaration of the load cr3 function in the memory header file

150
00:12:15,260 --> 00:12:23,360
Alright, let's go back to memory.c file. After we set up kvm, we call switch vm to 

151
00:12:23,360 --> 00:12:24,760
use the new paging setup in the kernel. 

152
00:12:25,490 --> 00:12:27,680
We also add print function to print memory manager is working now

153
00:12:27,680 --> 00:12:28,940
.

154
00:12:30,040 --> 00:12:31,480
OK, let's build the project.

155
00:12:41,200 --> 00:12:47,150
As you can see, we print the addresses on the screen. But there is no memory manager is working on the screen. 

156
00:12:48,430 --> 00:12:53,350
So in this example, the last message printed on the screen is the memory end

157
00:12:54,780 --> 00:13:00,390
which means after we switch vm, the printk function is not working as we expect.

158
00:13:01,860 --> 00:13:07,260
The reason is that we use the new paging setup. Remember we don’t remap the lower virtual address space 

159
00:13:07,450 --> 00:13:07,950
.

160
00:13:09,730 --> 00:13:11,470
But in the print.c file,

161
00:13:14,880 --> 00:13:18,720
you can see the screen buffer is still pointing to the address b8000. 

162
00:13:19,950 --> 00:13:25,740
In order to access the physical address b8000, we have to convert it to virtual address using macro 

163
00:13:26,130 --> 00:13:26,910
p2v. 

164
00:13:32,980 --> 00:13:35,370
So we need to add memory.h

165
00:13:40,500 --> 00:13:42,720
We should see the message on the screen this time. 

166
00:13:43,690 --> 00:13:44,790
So let's check it out.

167
00:13:55,570 --> 00:13:58,120
OK, you can see memory manager is working now.

168
00:13:59,540 --> 00:14:01,300
Alright, that's it for this lecture.