Renaissance of sparc UKOUG 2014

download Renaissance of sparc UKOUG 2014

of 46

  • date post

    12-Jul-2015
  • Category

    Software

  • view

    551
  • download

    0

Embed Size (px)

Transcript of Renaissance of sparc UKOUG 2014

  • 1

  • The Renaissance of SPARC Bjrn Rost / Philippe FierensStories from the field

    2

  • Not so safe harbour ;-)

    3

  • About Bjrn

    Founder, manager and consultant President RAC SIG ACE Director

  • About Phil

    Philippe Fierens Independent

    Oracle DBA (not solaris admin) Architect

    and Oracle ACE Associate

    and most of all DAD & husband ;-)

    AND

    5

    whoami

  • BBL

    6

  • AgendaHistoric overview

    SPARC features

    Comparing performance : stories from the field

    Licensing in a multi core multi socket world

    Sparc Super Cluster Exadata ++

    Solaris : well ahead of comet

    The near (R)evolution

    7

  • How it all began

    1982

    Sun M

    icrosy

    stems

    Inc F

    ound

    ed

    1985

    Scala

    ble Pr

    ocesso

    r ARC

    hitect

    ure se

    es the

    light

    1987

    32 bit

    SPAR

    C V7 s

    ystem

    1994

    64 bit

    SPAR

    C V9 a

    rchite

    cture

    1989

    SPAR

    C Inte

    rnatio

    nal h

    ttp://

    sparc

    .org

    2010

    Oracl

    e Acqu

    ires S

    un

    Solar

    is 11

    1997

    64 CP

    U En

    terpri

    se 100

    0 Syst

    em

    2000

    Solar

    is most

    used

    unix

    in the

    worl

    d

    2004 2005

    Dual

    core U

    ltra I

    VT1

    relea

    se Ni

    agara

    Solar

    is 10

    2007

    M ser

    ies la

    unch

    Ultr

    a VI

    8

  • 2010 2011 2012 2013

    9

  • M7M7#Processor# 32#SPARC#S4#Cores# #Dynamic#Threading#1#to#8#threads#/#core#

    New#Cache#Organiza8ons# Shared#L2#Data#and#Instr#Cache# 64#MB#Shared#and#Par@@oned#L3#Cache#

    DDR4#DRAM# Upto#2TB#Physical#Mem#/#Processor# 2x#F3x#beHer#Mem#Bandwidth#over#Last#gen#

    Applica8on#Accelera8on# Real@me#Applica@on#Data#Integrity# Concurrent#Memory#Migra@on## VA#masking# DB#Query#Ooad#Engine#

    SMP#1#to#32#Processors# Technology#20nm#

    2015

    more about this later

    10

  • SPARC FEATURES

    11

  • RAS

    Reliability

    Availability

    Serviceability

    12

  • RAS Redundant hardware in combination with software (FMA)

    Fault Management Architecturepredictive self-healing, offline resources

    with increased correctable errors

    Reducing components

    Reduce connectors to ASICS boards

    Instruction level retry and ECC Hot-swapping / plug-ability

    PCI cards hard disks power supplies

    13

  • SPARC Server Availability M6-32 / M5-32: Highest Availability in the SPARC Server Portfolio

    RAS

    T4#4$

    T5#8$T5#4$T5#2$

    T4#2$T4#1$ M6#32$$

    M5#32$

    VM Secure Live Migration Redundant / Hot-swap Fans, Power, Disks

    ECC, Predictive Self Healing

    Physical Partitions / Redundant Clock / Redundant & Hot-Plug SP Hot Plug PCIe Lane Sparing (L3$/Memory/Interconnect) + Redundant Interconnect

    14

  • 64 bit support (1994)64 bit support in HW and Solaris

    15

    1991 1992 1994 1995 2001

    MIP

    S R40

    00DE

    C PR

    ISM 64

    bit

    Intel

    anno

    unce

    s Itan

    ium IA

    64

    ULTR

    A SP

    ARC

    1997

    IBM

    POW

    ER 64

    bitIn

    tel re

    lease

    s Itan

    ium

  • large memory support

    64 bit allows large address spaceLarge pages support by defaultSPARC 2GB large pages

    % pagesize -a8192

  • memory per CPU

    x4170 M2 (nehalem)

    X3-2 (sandy bridge)

    X4-2 (sandy bridge)

    3-channel Intel E5

    T4 (S3)

    T5 (S3)

    M6 (S3)

    M7 (S4)

    0 512 1024 1536 2048

    2048GB

    1024GB

    512GB

    512GB

    384GB

    256GB

    256GB

    72GB

    memory per CPU

    17

  • Solaris 10 (2005)

    18

    Supports SPARC and x86 Scalable kernel ZFS Dtrace Zones lightweight virtualization >12 years of premier support +3 years of extended support

  • OVM for SPARC (2007)

    19

    Logical Domains Hypervisor built in hardware Zero overhead Strict isolation of CPU, mem, IO

    Flexible vswitch stuff PCIe dio for HBAs Improved vdisk performance (sol11 SRU19)

    Supported for hard partitioning

  • OVM types/roles Control domain :

    primary domain used to connect to HV and manage resources

    I/O domain :Direct access to phys I/O devices

    PCI root complex PCI card Regular non virtualized driversLimited number of I/O domains depending on platform T5-2

    vs T5-4 Service domain :

    Provides virtual I/O to guestsVirtual networkVirtual Disk

    Guest domain => consumer of virtual I/O 20

  • SPARC capping

    21

    SPARC offers lots of options to CAP CPU PDOMS LDOMS Zones

  • PDOMS

    22

    Full hardware and software fault isolation Divide large server into smaller one Physical segregation

    CPU memory I/O resources

  • ZonesGlobal zone

    owns the hardware and controls the resources

    Non Global zonesConfigured and run in global zone Exclusive access and Shared network access possible, NO Exclusive access to HBA Share same kernel as global zone Isolated ( != kernel zones)

    Supported to CAP CPU (licensing)

    23

  • domain isolation

    24

  • domain isolation

    128GB

    64GB

    128GB

    64GB

    M9000 with 4 domains

    PRODRAC

    STAGINGRAC

    ACCEPTANCERAC

    PRODRAC

    STAGINGRAC

    ACCEPTANCERAC

    ZONES ZONES

    25

  • subserver licensing

    68.000

    1.221.760

    SW license EE,RAC,PART,DIAG, TUNE + 3years supportHW 2*T4-1, 128GB RAM, 3 years support

    26

    Rule: Oracle SW costs will always outweigh GW Standard Edition: get as many cores per CPU as possible Enterprise Edition: get the smallest number of cores

    with most power per core

  • dark ages of SPARC

    Single thread Ultra systems EOL after 05 T1/T2 not really general purpose Slow single thread performance Mx000 the first successor

    27

  • CMT explained

    28

    psrinfo0 on-line since 12/07/2014 23:34:441 on-line since 12/07/2014 23:34:46

    snipped

    60 on-line since 12/07/2014 23:34:4661 on-line since 12/07/2014 23:34:4662 on-line since 12/07/2014 23:34:4663 on-line since 12/07/2014 23:34:46

  • perf benchmarkstim

    e

    0s

    125s

    250s

    375s

    500s

    threads per core

    0,5 1 2 4

    x5570 @2.93GHzT4 @2.85GHz29

  • results - latency

    30

    Very simple test w looped SQL Intel wins at very low utilization

    90s vs 119s (30%) this is intel turbo-boost effect

    @ 1 thread/core same response time SPARC performs better at over-

    saturation

  • more benchmarks

    http://kevinclosson.wordpress.com/2012/02/06/introducing-slob-the-silly-little-oracle-benchmark/31

    SLOB LIO test mode

    there are modes for PIO, writes, ... One suite of tools for multiple

    platforms Very simple (and open) logic Easy to set up, easy to compare

  • SLOB resultspeak SLOBops/s per core

    0

    1250

    2500

    3750

    5000

    x5570 E5-2640 E5-2690 SPARC64 T4 T5

    32

  • SLOB resultstime to complete 500k SLOB loops

    time

    0s

    400s

    800s

    1200s

    1600s

    threads per core

    0,25 0,5 1 2 4 8

    E5-2640 T4 T533

  • Solaris 11 (2010)

    34

    Network stack re-written ipadm/dladm virtual networking

    Enhanced support for zones kernel zones immutable zones Vnics

    New IPS packaging system GNU tools by default (feels like linux) Puppet integration Openstack integration ZFS enhancements

    deduplication

  • golden ages of sparc

    T4 T5 M7

    35

  • 2010 2011 2012 2013

    36

  • M7M7#Processor# 32#SPARC#S4#Cores# #Dynamic#Threading#1#to#8#threads#/#core#

    New#Cache#Organiza8ons# Shared#L2#Data#and#Instr#Cache# 64#MB#Shared#and#Par@@oned#L3#Cache#

    DDR4#DRAM# Upto#2TB#Physical#Mem#/#Processor# 2x#F3x#beHer#Mem#Bandwidth#over#Last#gen#

    Applica8on#Accelera8on# Real@me#Applica@on#Data#Integrity# Concurrent#Memory#Migra@on## VA#masking# DB#Query#Ooad#Engine#

    SMP#1#to#32#Processors# Technology#20nm#

    2015

    37

  • M7 in a nutshell

    Applica'on Data Integrity Virtual Address (VA) Masking Database Query Accelerator Engine

    In memory accelera'on Value Range Comparisons

    Fine Grain Memory Migra'on On-Chip Decompression

    2TB memory per chip32 cores on 1 chip

    SOFTWARE IN SILICON

    38

  • Test drive the M7

    39

    https://swisdev.oracle.com

  • SSC line-up

    M6-32T5-8T4-440

  • SSC aka Exadata ++

    Same exadata storage cells as in ExadataFlexibility

    2 Types :

    A Technical Overview of the Oracle SPARC SuperCluster T4-4

    Page 13

    Application Domain configuration.

    Figure 9. Domains mapped to a half rack system.

    Notice how the CPU and memory is configured separately from the Root Complex. When SPARC SuperCluster is installed, allocation granularity is 25% (8 CPU cores and 256GB of memory) per domain. This means the number and types of domains, number of Root Complexes, cores and memory must be decided upon at installation time. The customer can reassign CPU and memory after the installation using a standalone tool, setcoremem. The granularity is 4 CPU cores and 32GB of memory.

    Configurations

    SPARC SuperCluster is