Ch 11 Memory Management
28
1 Ch 11 Memory Management 제 47 제 : Memory Management
description
제 47 강 : Memory Management. Ch 11 Memory Management. Background. Pages basic unit of memory management Each architecture supports its own page size eg Most 32-bit architectures have 4KB pages The kernel represents every physical page with a struct page.
Transcript of Ch 11 Memory Management
1
Ch 11 Memory Management
제 47 강 : Memory Management
2
Background• Pages
– basic unit of memory management– Each architecture supports its own page size
eg Most 32-bit architectures have 4KB pages
– The kernel represents every physical page with a struct page
struct page {unsigned long flags;atomoc_t count;struct list_head list;struct list_head lru;struct address_space *mapping;unsigned long index;;union {
struct pte_chain *chain;pte_addr_t direct;
} pte;unsigned long private;void *virtual;
};
status of the pageusage count of the page
page’s virtual address
<include/linux/mm.h
inode list, file list, free listLRU list for replacement
3
struct page {unsigned long flags;atomoc_t count;struct list_head list;struct list_head lru;struct address_space *mapping;unsigned long index;;union {
struct pte_chain *chain;pte_addr_t direct;
} pte;unsigned long private;void *virtual;
};
usage count of the page
<include/linux/mm.h
inode list, file list, free listLRU list for replacement
page
page
page
inodeA
page
page
page
inodeB
page
page
taskA
page
page
IPCA
page
page page page page free
LRU
Page CacheSet of data structuresthat contains pages
backed by files, devices, …
4
Background• Zones
– kernel cannot treat all pages as identical.• because some pages cannot be used for certain task• eg early DMA devices could only access 1st 16MB of memory
– kernel divides pages into different zones– 3 memory zones in Linux
– Each zone is represented by struct zone, – which is defined in <include/linux/mmzone.h>
Zone DescriptionZONE_DMA DMA_able pagesZONE_NORMAL Normally addressable pagesZONE_HIGHMEM Dynamically mapped pages
5
Smarter Space ManagementStatic Dynamic Allocation
• Before: static allocation (allocation size at coding time)
• Now:space allocated dynamically on-demandkernel manages space pool
[Lions] [Linux]
proc[N] file[M] inode[I]
- static allocation- arrays- not flexible
proc fileCentral
Space Pool(Heap Storage)
- dynamic allocation (kernel)- linked list- flexible
inode
Allocated Not Allocated
6
Heap Storage
7
Heap - Variable size allocation
Heap Storageallocator
deallocator
malloc()
free()
Allows to allocate/free any size block,
in any order
8
struct task_struct *t;struct inode *i;
process_create() { /* create a new process. Need a new space for task */ /* allocate a block of task_struct from heap storage */ t = (task_struct) malloc(sizeof(task_struct)) /* link this struct to task list */}
More malloc(sizeof(task_struct) …
malloc/free exampleHeap Storageallocator
deallocator
malloc()
free()
Kernel
inode
Heap Storageallocator
deallocator
malloc()
free()
Kernel
inode
task
Heap Storageallocator
deallocator
malloc()
free()
Kernel
file
inode
sizeof(t)
sizeof(inode)
task
9
process_remove() { /* deleting this process. Do not need the task_struct space return t to available memory pool */ free(t)}
inode_remove() { /* deleting this file. Do not need the inode space return f to available memory pool */ free(i)}
malloc/free example
Heap Storageallocator
deallocator
malloc()
free()
Kernel
file
inode
Heap Storageallocator
deallocator
malloc()
free()
Kernel
file
inode
task
task
10
Heap Storage
space pool
proc file
inode
Allocated
variable sizeany order
Not Allocated
variablesize
11
Problems of HeapExternal Fragmentation
Enough aggregate heap storage. But no single free block of size 3 is available
use (3) use (2) free(2)
free(2)
free(2) use (4)
use (3) free(2) free(2) use (4)
mfree( )
malloc(3)
use (3)
12
Problems of HeapCoalesce
3 independent free (small) blocks?
use (3) use (2) free(2) free(2) use (4)
use (3) free(2) free(2) free(2) use (4)
use (3) free(2) free(2) free(2) free(4)
13
Problems of HeapCoalesce
coalesced large free block
Join with next (previous) block if they are free & adjacent
Make it into single (large) free block
Complex data structure & algorithm
use (3) use (2) free(2) free(2) use (4)
use (3) free(2) free(2) free(2) use (4)
use (3) free(2) free(2) free(2) free(4)
use (3) free(2)
14
Compaction
shift many wordsVery big overhead though
use (3) free(2) free(2) free(2) use (4)
use (3) free(2) free(2) free(2) use (4)
15
Performance Enhancement of Heap storage
• External fragmentation problem– Coalesce– Compaction– Large overhead for solving external
fragmentation!
• All SMP CPU’s access, every kernel data structure– Must lock/unlock on every access– Central Bottleneck
Heap Storage
CPU 0 CPU 1 CPU 2 CPU 3
inode file task inode
16
Smarter Space ManagementStatic Dynamic Allocation
[Method 1-Array]
proc[N] file[M] inode[I]
- static allocation- not flexible
proc fileCentral
Space Pool(Heap Storage)
-dynamic allocation (kernel)-Heap becomes a bottleneck-External Fragmentation Overhead
inode
Allocated Not Allocated
[Method 2-Heap]
17
[Method 3][Allocate in Slab units -- Big Size, Fixed Size]
Allocated Not Allocated Allocated
inode fileHeap
malloc (inode)malloc (inode)malloc (inode)
malloc (file)malloc (file)malloc (file)
slabslabslabunit
slabunit
slab
18
Requests for free space
Distributed among Many Caches
Allocated Not Allocated
inode file
slabunit
<inode Cache>
slab
inodeobject
free list
slabslabunit
Allocated
<file Cache>
fileobjec
t free
list
slabslab
Fixed Size
Simple Space Management
inode object, inode cache
Slab Layer
19
Requests for free space
Distributed among Many Caches
Allocated Not Allocated
inode file
slabunit
<inode Cache>
slab
inodeobject
free list
slabslabunit
Allocated
<file Cache>
fileobjec
t free
list
slabslab
Fixed Size
Simple Space Management
partial
full
empty
emptypartial/full slabfree inode list
Slab Layer
20
Terminologies: “inode“ “page” “inode-table”
inode-object slab inode-cache
space poolslabunit
Allocated Not Allocated
<inode Cache>
partial full empty
slab
slabs
inodeobject
free space
for inode free
list
Terminology
Slab Layer
21
Terminology Inode Cache has to manage its private free-list “Allocated” to inode cache
“Deallocated” back to central storage pool “Occupied” used for file A’s inode “Free” allocated, but not used for any
space poolslabunit
Allocated Not Allocated
<inode Cache>
free list
allocate/deallocate
occupy/free
allocated but free
slab
22
Slab layer – how it works• request space for a new inode struct
– <case> free list “has” space• there is space in partial (or empty) slabs• allocate from unused struct from slab
– <case> free list is empty • does not exist --- any empty or partial slab• invokes page(slab) allocation function
• release -- finished using inode struct • marks object as free
• If system-wide memory is low– deallocate empty slabs
Allocated
<inode Cache>
free list
slab
23
Slab Layer• Cache Data Structure
– each cache is represented by a kmem_cache_s structure
– defined in <mm/slab.c>– eg) inode_cachep
struct kmem_cache_s {…struct kmem_list3
lists; …
} struct kmem_list3 {
struct list_head
slabs_partial
struct list_head slabs_full
struct list_head slabs_free
…
}
• each list contains
partial slabs,
full slabs,
free slabs
Allocated
<inode Cache>
free list
slab
24
Slab Layer
• Slab Data Structure– each slab is represented by slab
structure– defined in <mm/slab.c>
struct slab { struct list_head list; unsigned long colouroff; void *s_mem; unsigned int inuse; kmem_bufctl_t free;};
Not Allocated
slab slab
Slab Layer
slabunit
slabunit
25
Slab Layer
• Create / Destroy Cache– kmem_cache_t * kmem_cache_create (const char
*name, size_t size, size_t offset, unsigned long flags, void (*ctor)(void*, kmem_cache_t *, unsigned long), void (*dtor)(void*, kmem_cache_t *, unsigned long))• create new cache
– int kmem_cache_destroy (kmem_cache_t *cachep)• destroy a cache• must ensure that all slabs in the cache are empty• must ensure that no one access the cache during a call
to kmem_cache_destroy()
Allocated
<inode Cache>
free list
slab
26
Slab Layer
• Object in a Cache– void * kmem_cache_alloc (kmem_cache_t *cachep,
int flags)• returns a pointer to an object for the given
cache
– void kmem_cache_free (kmem_cache_t *cachep, void *objp)•marks the object in cache as free
Allocated
<inode Cache>
free list
slab
27
• Allocating a Slab to a Cache– allocate slabs only when – no partial/empty slabs exits in cache
kmem_cache_alloc kmem_cache_grow()
kmem_getpages()
cache_slabmgmt()
cache_init_objs()
(1) there are no free object in cache (2) obtain page
frame(s)
(3) get a newslab descriptor
(4) adds slab descriptor at the endof free slab list
Allocated
<inode Cache>
free list
slab
28
• Releasing a Slab from CacheSlab is released only when (1) available memory grows low (& free some memory) (2) or when a cache is explicitly destroyed
– static void kmem_freepages (kmem_cache_t *cachep, …) • returns all the contiguous page frames used by the slab
– static void slab_destroy (kmem_cache_t *cachep, ..) • checks whether this cache has destructor method• for its objects
Allocated
<inode Cache>
free list
slab
