Asymmetric NUMA: - Multiple-memory … NUMA: Multiple-memory management for the rest of us ......

Asymmetric NUMA:Multiple-memory management for the rest of us

Paul Mundtpaul.mundt@renesas.com

Magnus Dammdamm@igel.co.jp

Renesas Technology

ELC-Europe 2007

Magnus Damm (damm@igel.co.jp) Asymmetric NUMA ELC-Europe 2007 1 / 30

Outline

Introduction to NUMAUniprocessor and BeyondFrom UMA to NUMASymmetric and Asymmetric NUMANUMA for Embedded Systems

Linux Kernel Memory ManagementSingle Node BasicsMultiple NodesAsymmetric NUMA

Application Node ControlMemory Policiestmpfs and cpusets

Introduction to NUMA

Outline

Introduction to NUMA Uniprocessor and Beyond

Uniprocessor and Beyond

Uniprocessor System (UP):I A single processor has all memory bus bandwidth to itself.

Symmetric Multi-Processor System (SMP):I Multiple processors share memory bus bandwidth.

Memory bus bandwidth becomes a bottleneck for large systems.

Introduction to NUMA Uniprocessor and Beyond

Uniprocessor and Beyond

Uniprocessor System (UP):I A single processor has all memory bus bandwidth to itself.

Symmetric Multi-Processor System (SMP):I Multiple processors share memory bus bandwidth.

Memory bus bandwidth becomes a bottleneck for large systems.

Introduction to NUMA From UMA to NUMA

From UMA to NUMA

Uniform Memory Access (UMA)

I Uniprocessor or Symmetric Multi-Processor configurations.I Memory is accessed through a high-speed local bus.

Memory access speed is constant for all processors and memories.

From UMA to NUMA

Uniform Memory Access (UMA)

I Uniprocessor or Symmetric Multi-Processor configurations.I Memory is accessed through a high-speed local bus.

Memory access speed is constant for all processors and memories.

Non-Uniform Memory Access (NUMA):

I A NUMA system is divided into multiple nodes.I Each node is equipped with zero or more processors.I Local memory may exist on the same node as the processor.I Remote memory access is provided through interconnects.

With NUMA the access speed for memory varies with the distancebetween processors and memory regions.

Non-Uniform Memory Access (NUMA):

I A NUMA system is divided into multiple nodes.I Each node is equipped with zero or more processors.I Local memory may exist on the same node as the processor.I Remote memory access is provided through interconnects.

With NUMA the access speed for memory varies with the distancebetween processors and memory regions.

Introduction to NUMA Symmetric and Asymmetric NUMA

Symmetric and Asymmetric NUMA

Symmetric NUMA:I Equal amounts of memory is assigned to each node.

Asymmetric NUMA:I The amount of memory for each node may vary greatly.I Nodes may be equipped with memory but without processors.I Memoryless nodes - processor-only configurations.

Introduction to NUMA NUMA for Embedded Systems

NUMA for Embedded Systems

The existing NUMA interfaces in Linux are. . .I architecture-independent.I well-established.

. . . and they allow us to:I Select node memory to back file data with.I Select node for process memory ranges.

The existing NUMA interfaces in Linux are. . .I architecture-independent.I well-established.

. . . and they allow us to:I Select node memory to back file data with.I Select node for process memory ranges.

“...superh is starting to use NUMA now, due tovarying access times of various sorts of memory.one can envisage other embedded setups doing that”

- Andrew Morton, on linux-mm

Linux Kernel Memory Management

Outline

Linux Kernel Memory Management Single Node Basics

Physical Memory

struct page mem_map[]

Physical Memory

PFN - Page Frame Number

Memory Models

One contiguous range of memory: CONFIG_FLATMEM

Memory Models

More than one contiguous range of memory:CONFIG_SPARSEMEM or CONFIG_DISCONTIGMEM

Physically Contiguous Allocators

Physical Page Allocator

Binary Buddy Allocator Algorithm

struct page *alloc_pages(gfp_t, unsigned int)void __free_pages(struct page *, unsigned int)

Order N allocations:

I Order 0 -> 1 page.I Order 1 -> 2 pages.I Order 2 -> 4 pages.I . . .I Order N -> 2 ^N pages.

SLAB Allocator

A caching object-based allocator.

Create and destroy a cache of objects:

struct kmem_cache *kmem_cache_create(...)void kmem_cache_destroy(struct kmem_cache *)

Allocate and free objects using the cache:

void *kmem_cache_alloc(struct kmem_cache *, gfp_t))void kmem_cache_free(struct kmem_cache *, void *)

/proc/slabinfo provides statistics.

SLAB Allocator

A caching object-based allocator.

Create and destroy a cache of objects:

struct kmem_cache *kmem_cache_create(...)void kmem_cache_destroy(struct kmem_cache *)

Allocate and free objects using the cache:

void *kmem_cache_alloc(struct kmem_cache *, gfp_t))void kmem_cache_free(struct kmem_cache *, void *)

/proc/slabinfo provides statistics.

SLAB Allocators (2.6.23)SLAB (1996)

I linux/mm/slab.c, ~4500 lines, CONFIG_SLABI Default allocator, per-cpu and per-node data.

SLOB (2003)

I linux/mm/slob.c, ~600 lines, CONFIG_SLOBI Simple and small, but with single free list.

SLUB (2007)

I linux/mm/slub.c, ~4100 lines, CONFIG_SLUBI Unqueued allocator, minimizes cache line usage.

kmalloc and kfree

A thin multi-purpose layer on top of the SLAB allocator.

void *kmalloc(size_t, gfp_t)void kfree(const void *)

Linux Kernel Memory Management Multiple Nodes

Multiple Nodes (CONFIG_NUMA=y)

CONFIG_SPARSEMEM or CONFIG_DISCONTIGMEM required.

Linux Kernel Memory Management Asymmetric NUMA

Asymmetric NUMAPhysical Page Allocator

I Kernel allocates from all nodes during initialization.I 16MB cut-off added.

I Kernel defaults to node-local allocations during run time.I Use node 0 for System RAM.

SLAB Allocators

I SLABI SLUB

I Requires patches to exclude small nodes.I SLOB

I Low overhead, preferred SLAB Allocator for Asymmetric NUMA.I Primitive locking - possible performance issues for SMP systems.

Asymmetric NUMA

Asymmetric NUMA - Overhead

/ # cat /sys/devices/system/node/node1/meminfo

Node 1 MemTotal: 128 kBNode 1 MemFree: 72 kBNode 1 MemUsed: 56 kBNode 1 Active: 0 kB...

Application Node Control

Outline

Application Node Control Memory Policies

What are Memory Policies?

Memory Policies control the behavior of the memory allocator.

They allow us to adjust. . .I Per-process memory allocation policy.I Memory allocation policies for ranges of process memory.I Memory Policies for file systems such as tmpfs.

Using Memory Policies we can. . .I Select which nodes to allocate from.I Chose between optimizing for latency or bandwidth.I Allow or disallow fallback allocation from other nodes.

Memory Policies - ModesMPOL_DEFAULT

I Prioritize local node over remote ones.

MPOL_BIND

I Allocate from specified nodes only, one by one.

MPOL_INTERLEAVE

I Spread out allocations over specified nodes.

MPOL_PREFERRED

I Allocate from specified nodes, one by one.

Memory Policies

Per-process control of memory allocations:

int set_mempolicy(int mode, unsigned long *nodemask,unsigned long maxnode)

Control ranges of process memory:

int mbind(void *start, unsigned long len, int mode,unsigned long *nodemask, unsigned long maxnode,unsigned flags)

man 2 set_mempolicyman 2 mbindman 3 numa - libnuma by Andi Kleen

Memory Policies

Per-process control of memory allocations:

int set_mempolicy(int mode, unsigned long *nodemask,unsigned long maxnode)

Control ranges of process memory:

int mbind(void *start, unsigned long len, int mode,unsigned long *nodemask, unsigned long maxnode,unsigned flags)

man 2 set_mempolicyman 2 mbindman 3 numa - libnuma by Andi Kleen

Application Node Control tmpfs and cpusets

tmpfs and cpusets

tmpfs is a file system which keeps all files in memory. Mount optionscan be used to select memory policy.

mount -t tmpfs -o mpol=bind:0,2 tmpfs /mytmpfs

See linux/Documentation/filesystems/tmpfs.txt for more information.

cpusets is kernel mechanism that assigns processes to a subset of allavailable processors and memory nodes.

More information available in linux/Documentation/cpusets.txt

Application Node Control tmpfs and cpusets

tmpfs and cpusets

tmpfs is a file system which keeps all files in memory. Mount optionscan be used to select memory policy.

mount -t tmpfs -o mpol=bind:0,2 tmpfs /mytmpfs

See linux/Documentation/filesystems/tmpfs.txt for more information.

cpusets is kernel mechanism that assigns processes to a subset of allavailable processors and memory nodes.

More information available in linux/Documentation/cpusets.txt

Summary

I Asymmetric NUMA brings NUMA to the embedded space.I Well-established interfaces outweights the added overhead.

Asymmetric NUMA: - Multiple-memory … NUMA: Multiple-memory management for the rest of us ......

Documents

Transcript of Asymmetric NUMA: - Multiple-memory … NUMA: Multiple-memory management for the rest of us ......

The Art of Efﬁcient In-memory Query Processing on NUMA ...sray/papers/ICDE2020_NUMA_Main_Memory_Query.pdfConsequently, remote memory access is slower. In addition to remote memory

NUMA aware heap memory manager

ERIS: A NUMA-AWARE IN-MEMORY S E T -S A W - adms … · Tim Kiefer ADMS 2014 Benjamin Schlegel Hangzhou, ... 1000 local remote 0 10 20 30 40 ... ERIS: A NUMA-Aware In-Memory Storage

Kernel level memory management - GitHub Pages...Kernel level memory management 1. The very base on boot vs memory management 2. Memory ‘Nodes’ (UMA vs NUMA) 3. x86 paging support

Thread and Memory Placement on NUMA Systems: …uniform memory access times (NUMA), meaning that the latency of data access depends on where (which CPU-cache or memory node) the data

Compact NUMA-aware Locks - arXiv · locks can be implemented using a single memory word (or even a bit), virtually all NUMA-aware locks are hierarchical in their nature, built of

Reducing Memory Access Latency with Asymmetric DRAM …prof.icc.skku.ac.kr/~jaewlee/pubs/isca13_charm.pdfReducing Memory Access Latency with Asymmetric DRAM Bank Organizations Young

Linux NUMA support for HP ProLiant servers...HP incorporates the NUMA architecture in its servers primarily because of the architecture’s capability for providing scalable memory

Matching Memory Access Patterns and Data Placement for NUMA Systems

NUMA Time Warp - CORE · Uniform-Memory-Access (NUMA) platforms. The relevance of the NUMA architectural paradigm lies in that systems continuously increase their number of processors

Memory System Performance in a NUMA Multicore Multiprocessor

NUMA Memory Architectures and the Linux Memory … Memory Architectures and the Linux Memory System Patrick Ladd Technical Account Manager pladd@redhat.com / pmladd@gmail.com ACM Poughkeepsie

Preparing Applications for Persistent · PDF file · 2017-08-24Preparing Applications for Persistent Memory ... SSD NVMe Flash Persistent Memory Context Switch NUMA Min, ... Use today’s

NUMA-Aware Shared-Memory Collective Communication for MPI · collective operations on multicore architectures. 3. We establish performance models based on memory access latency and

Computer Science | Western Michigan Universityelise/courses/cs626/s09/abdullah/… · Web view2) Non-uniform memory access (NUMA): More than one memory but some memory is closer

Unfair Scheduling Patterns in NUMA Architectures · protocols and non-uniform memory access (NUMA). In this paper, we thoroughly investigate concurrent scheduling on real hardware.

Non-uniform memory access (NUMA)echow/ipcc/hpc-course/HPC...Non-uniform memory access (NUMA) •Memory access between processor core to main memory is not uniform. •Memory resides

Scale-Free Graph Processing on a NUMA Machinematei/papers/ia3-tanuj.pdf · 2018-10-27 · Non-Uniform Memory Access (NUMA, a.k.a. distributed shared-memory) architecture machines

(Mis) Understanding the NUMA Memory System Performance of Multithreaded Workloads

Introduction to Parallel Programming€¦ · Introduction to Shared Memory Parallel Programming Parallel Architectures Parallel ArchitecturesI cc-NUMA I cc-NUMA (cache-coherent non-uniform