07-memory

5/22/2018 07-memory

1/35

Crash Dump Analysis

Memory management issues

Jakub Jerm

Martin Dck

5/22/2018 07-memory

2/35

Crash Dump Analysis MFF UK Memory management issues 2

Overview

Slab allocator

Kernel memory allocator

Internal debugging features

Support in mdb

Memory management errors

Library libumem

5/22/2018 07-memory

3/35


References

Jeff Bonwick: The Slab Allocator: An Object-

Caching Kernel Memory Allocator

http://www.usenix.org/publications/library/proceedings/bos94/bonwick.html

Jeff Bonwick, Jonathan Adams: Magazines andVmem: Extending the Slab Allocator to Many

CPUs and Arbitrary Resources

http://www.usenix.org/event/usenix01/bonwick.html
http://www.usenix.org/publications/library/proceedings/bos94/bonwick.htmlhttp://www.usenix.org/event/usenix01/bonwick.htmlhttp://www.usenix.org/event/usenix01/bonwick.htmlhttp://www.usenix.org/publications/library/proceedings/bos94/bonwick.html

5/22/2018 07-memory

4/35


References (2)

Man pages

libumem(3LIB), umem_debug(3MALLOC)

Identifying Memory Management Bugs ...

http://developers.sun.com/solaris/articles/libumem_library.html
http://developers.sun.com/solaris/articles/libumem_library.htmlhttp://developers.sun.com/solaris/articles/libumem_library.html

5/22/2018 07-memory

5/35


Slab allocator

Object caching allocator

API for dealing with objectswhich are frequently

allocated / deallocated

Avoid full initialization after each allocation

Also for non-caching allocations

Kernel version of malloc/free

kmem_alloc(size) allocate a buffer kmem_free(bufp, size) release it

5/22/2018 07-memory

6/35


Slab allocator (2)

Cache

Contains objectsof same type/size

Several caches for different purposes

Object caches: process_cache, thread_cache, ...

Used for types such as proc_t, kthread_t, ...

Non-caching allocations use caches internally:

kmem_alloc_8, kmem_alloc_16, ...

Suffix _8 or _16 is maximum allocation size

Best fit: kmem_alloc(14)allocates a bufferfrom thekmem_alloc_16cache (2 bytes wasted)

Cache consists of slabs

5/22/2018 07-memory

7/35


Slab allocator (3)

Slab

Allocation unit for a cache(internally)

One or more pages

Buffer (buf)

Raw memory area

Non-cached data

Object

Bufctl

Buffer's metadata

kmem_slab

kmem_

bufctl

kmem_

bufctl

kmem_

bufctl

bufbuf buf unused

slab

Metadata

about slab

5/22/2018 07-memory

8/35


Slab allocator (4)

Small objects

Size

5/22/2018 07-memory

9/35


Buffer layout

Buftag is appended to each buffer depending

on value of kmem_flagsvariable

Redzone pattern

Metadata (bufctl back pointer, stats)

Redzone byte marks end of allocated user area

It's used in case of non-caching allocations

dataredzone

byteunused

redzone

pattern

general buffer

metadata

buftag

5/22/2018 07-memory

10/35


Vmem allocator

Backing store for Slab allocator

General-purpose large allocations

One or multiple pages

5/22/2018 07-memory

11/35


Using kmem_flags

Debugging features

Kernel variable

Add set kmem_flags = 0x0f to /etc/system

0x0f is used by DEBUG kernels

Bits for particular features

See defines in /usr/include/sys/kmem_impl.h

defi!e "#$_%&' 0x00000001 *+ tra!sactio! auditi!g +*

defi!e "#$_'%'-$ 0x0000000. *+ deadbeef checki!g +*

defi!e "#$_/'2 0x00000004 *+ redzo!e checki!g +*

defi!e "#$_322 0x00000008 *+ freed5buffer co!te!t loggi!g +*

defi!e "#$_2#%%2 0x000000.0 *+ disable per5cpu magazi!es +*

defi!e "#$_$/7% 0x00000040 *+ put all bufs before u!mapped pages +*

5/22/2018 07-memory

12/35


Debugging patterns

Memory patterns are human-readable strings

Free pattern deadbeef

Uninitialized pattern baddcafe

Redzone pattern feedface

Defined in /usr/include/sys/kmem_impl.h

defi!e "##_$/_9%/2 0xdeadbeefdeadbeef&

defi!e "##_&22%'_9%/2 0xbaddcafebaddcafe&defi!e "##_/'2_9%/2 0xfeedfacefeedface&

defi!e "##_/'2_-: 0xbb

5/22/2018 07-memory

13/35


Memory management errors

Multiple free

Buffer overrun

Use of uninitialized memory Use after free

Memory leaks

Memory allocator specific

Freeing buffer to wrong cache

Freeing invalid pointer

5/22/2018 07-memory

14/35


Multiple free

Impact

May corrupt heap

May free someone else's bufferker!el memor; allocator10 cache< kmem_alloc_.

pre?ious tra!sactio! o! buffer ffffff008ada088 slab=ffffff008ace88A8 cache< kmem_alloc_.

kmem_cache_free_debugB1.f

kmem_cache_freeBC

kmem_freeB1f>

barB.A

fooBe

taskD_threadB16

thread_startB8

pa!icEcpu0F*thread=ffffff0001.fac808 cache< kmem_alloc_8

pre?ious tra!sactio! o! buffer ffffff00ba0fa660

taskD_threadB16

thread_startB8

pa!icEcpu0F*thread=ffffff00014.fc80c80

07-memory

Documents

Transcript of 07-memory