EXPERIENCES FROM OCP GEAR DEPLOYMENTS WITH TELCOS

Mika Hatanpää

Head of AirFrame data center solutions R&D

Nokia

© 2018 Nokia3

Topics

• Telco transformation to NFV and typical telco deployments

• Nokia Open Rack solution and Open Rack benefits

• Experiences from Open Rack solution deployments

• Summary

© 2018 Nokia4

Telco transformation

to NFV and typical

telco deployments

5 © Nokia 2018

Cloud based solutions required for scalability and agility

Service providers fast having to adapt to demands of digitalized business

Digitalization of business

10Mbps

100ms

10B

$1000

Months

10Gbps

1ms

100B+

$1

Days

People

Devices

People

Devices

Things

5G

4GMore connections

Faster speeds

Shorter cyclesCloud based network needed to increase scale and agility

Evolution path based on cloud required to be 5G ready

6 © Nokia 2018

• Cloud at network edge for radio and mobile edge computing apps

• Exploiting cloud infrastructure synergies between distributed and centralized sites

• Cloud for highly scalable workloads, no compromise to high availability

• Design for performance with lowest TCO

Cloud infrastructure for telco needs to deliver high performanceEfficient, real-time, low latency, centralized and distributed

4G 4G evolution 5G

Drive latency reduction to enable new use cases

Address fast growing

capacity demand

Centralized Data Centers, efficient capacity

Distributed Data Centers, low latency

Cloud infra that incorporates webscale best practices

Efficient Scalability

Robust performance

Automated operations

Cloud infra that delivers low latency

7 © Nokia 2018

From bare metal hardware to cloud data centers

Network functions today run on purpose built bare metal hardware huge effort to

operate and maintain networks where almost every box runs on a different platform.

Network functions

(Network

elements) today

have very different

kinds of

requirements

related to real time

behavior (latency,

jitter etc.),

computing

performance,

storage, network

connectivity, traffic

throughput, high

availability, etc.

Big challenge for

the telco cloud.

© 2018 Nokia8

Scalable data center architecture from hyper-scale DC to the local edge

Multi-layer cloud architecture driven by performance and facility requirements

Centralized data centerRegional

data centersLocal

data centers

De

ma

nd

100’s of sites, low power budget, limited physical dimensions (D=600mm), NEBS

10’s of sites, medium power budget, medium physical dimensions

Few sites, high power budget, flexible physical dimensions (D=1200mm)

© 2018 Nokia9

Nokia Open Rack

solution and Open

Rack benefits

10 © Nokia 2018

Rackmount systems

• Easy downward scalability for distributed datacenters

• 19” rack-mount form factor

• Full-width 1U/2U server, or 2U storage servers

Combined innovation from telecom, IT and Open Source HW

OCP hyperscale systems

• Higher density and lower energy consumption

• 3 servers in 2OU in OpenRackv2

• Superior serviceability: one person manages 25k+ servers

Enhancements for telcoEnhanced Open Rack v2 with

• EMI shielding, CE/FCC certification

• 48V and 400V DC power feed options

• Single-rack seismic tolerance

• Power capacity increase

• Extended lifetime

Nokia AirFrame releases provided in two form factorsServing differing operator use cases and needs

Intel® Xeon® Scalable Processor (Skylake)Intel® Xeon® Scalable Processor (Skylake)

11 © Nokia 2018

Nokia AirFrame Open Rack building blocks

1. Rack• 42 OU / 21” rack (external width 600mm,

similar to 19” rack)• Open Rack v2 specification• Two power zones with own power shelves• 12,5 VDC bus bar for power distribution

5. Power shelf• 6 PSU’s, 12,5 kW load capacityInput:• Dual 3P 208/415 VAC 50/60 Hz• Dual negative 40 – 72 VDCOutput:• 12,5 VDC busbar

6. Server shelf• 2OU (96mm) shelf with three

bays for server nodes• Open Rack V1 and V2 compatible

2. Server nodes• Dual-socket 2 OU and 1/3 shelf server node• Intel Xeon Scalable CPUs, up to 165W SKUs• 12x DDR4 RDIMM modules• OCP mezzanine slot for dual-port 1/10/25G NIC

Storage nodes

3. Storage nodes• 2 OU and full width unit• JBOD supporting up to 45 hot

swappable 3,5”/2,5’’ HDD drives• Fully redundant

4. Switches• AirFrame maintenance switch (48+6 port 1RU management switch)• AirFrame 10/25/40/50/100GbE 1RU high-density switch (32x100G /

32x40G / 128x10/25G ports)

1 3

4

6

52

© 2018 Nokia14

OCP design for serviceability

Top serviceability benefits of OCP based design:

1. 4x faster completion of required HW tasks

2. 65% more servers handled per operational person*

3. 61% less of productive employee time lost*

4. 38% less time needed to resolve unplanned downtime*

* Source:IDC OCP study

© 2018 Nokia15

Experiences from

Open Rack solution

deployments

© 2018 Nokia16

Typical Installations

• Number of racks in one installation is typically less than 10 per site.

• No greenfield installations, installations are done to existing brownfield sites.

• Requirement to adapt to existing environment.

• Everything is redundant in order to guarantee high availability for the service.

• Switches, NICs, cabling, HDDs/SSDs, …

• Telco requirements like NEBS are still mandatory in many cases.

• Installations are customer specific.

• No common blueprint, customized configurations are required.

• Installation use cases vary from bare metal applications to NFV telco cloud IaaS

platform.

© 2018 Nokia17

Transportation

• Rack and stack on site is typically not allowed.

• Pre-installed racks are preferred.

• Installation sites are all over the world and due to long transportation lead times sea

cargo is not possible.

• Air freight is used for pre-installed racks.

• Current Nokia Open Rack v2 height is 42 OU

• Height x width x depth: 2258 mm x 600 mm x 1067 mm.

• Palleted rack crate height is ~2500 mm which creates challenges for air freight.

• Transportation tolerance according to NEBS in required.

© 2018 Nokia18

Anything can happen in transportation

Before shipping Arrival

© 2018 Nokia19

Site Physical Limitations

• Fully equipped Open Rack v2 weight is >800 kg >1200 kg/m2

• Floor load capacity sometimes limits the configurations.

• Rack depth is limited in many locations

• Old telco central office sites can limit rack depth to 800 mm

• Edge sites are typically existing radio sites where rack depth is max 600 mm

• Old sites typically also have limitations due to

• Elevator capacity

• Delivery path height (door openings)

• Site surveys have to be done before shipping.

• Colo Facility Guidelines for OCP racks defined in OCP Data Center Facility project is providing

good framework.

© 2018 Nokia20

Power and Cooling

• Old telco sites typically have -48VDC power feed infrastructure with battery rooms

• Several AC power feed options for global use cases are needed, e.g.

• 110VAC, 208VAC, 230VAC, 380VAC, single phase, three phase, 50/60 Hz, different wattages,

different connectors, …

• Power cabling from top and bottom both need to be supported.

• Site power budgets quite often limit size of installations

• Limitation can be as low as 4 kW per rack

• Site cooling capacity often limits rack configurations

• Limitation can be as low as 4 kW per rack

• Integrated BBUs are not in use in telco installations

• Difficult to manage due to lithium (transportation limitations)

© 2018 Nokia21

Compliance

Standard telco equipment environmental requirements are still mandatory in most cases. For example:

• Safety: IEC 62368-1:2014, EN60950-1: 2006 + A2:2013 and IEC 60950-1 for safety, including national deviations, GR-1089-CORE.

• EMI/EMC: EN300386 (v1.6.1), CFR 47, FCC 15, class A, CISPR 22 Class A and CISPR 24, TEC/EMI/TEL-001/01/FEB-09 and TEC/IR/SWN-2MB/07/MAR-10, GR-1089-CORE

• Temperature tolerance: ETSI EN300 019-1-3 Class 3.1, ETSI EN300 019-1-3 Class 3.2, GR-63-CORE, section 4.1.

• Seismic tolerance: GR-63-CORE, section 4.4 Zone 4

• Transportation and storage: ETSI EN 300 019-1-2 v.2.2.1 class 2.2, EN 300 019-1-1 [20] Class 1.2, EN 300 019-1-2 [21] Class 2.3

• RoHS: EU RoHS directive 2011/65/EU Article 7b (EN 50581 (2012))

• WEEE: EU WEEE (Waste Electrical & Electronic Equipment) Directive 2002/96/EC and recast WEEE Directive 2012/19/EU

• REACH: EU REGULATION (EC) No 1907/2006

• Fire resistance: ANSI T1.307-2007 and the requirements specified in GR-63-CORE chapter 4.2.3, GR-63-CORE chapter 4.2.2.2 Shelf-Level Fire-Resistance Criteria.

• Energy efficiency: ATIS-0600015

• Acoustic noise: GR-63-CORE, section 4.6

Concerns:

• Busbar safety is raising concerns.

© 2018 Nokia22

Networking

• Installations typically have very complex networking

requirements.

• ONL based switch implementations do not have required

features related to redundancy and millisecond level recovery

times.

• All kinds of NICs and transceivers are in use

• Customer specific configurations required.

• 4 x 25GbE or 6 x 10GbE from servers requires a lot of

cables and cabling space.

• Open Rack does not have too much room for cable

management.

• Switches from different vendors need to be supported.

• Uplink cabling from top and bottom have be supported.

© 2018 Nokia23

Software Support

• Certifications for commercial software products are needed, e.g.

• Red Hat Linux

• Canonical Linux

• VMware

• Oracle

• Open source software is gaining interest, e.g.

• Open Stack, Kubernetes, OPNFV, etc.

• Onboarding of 3rd party applications is required in many cases.

© 2018 Nokia24

Security

• Security is taken very seriously by telcos

• A huge effort has been spent on security testing and hardening of the OCP gear.

• Main challenges are related to firmware and networking security

<Document ID: change ID in footer or remove> <Change information classification in footer>

© 2018 Nokia25

Acceleration

• In traditional telco many functionalities are done with special purpose HW using

• FPGAs

• DSPs

• Network / packet processors

• ASICs

• Acceleration is becoming a must for

• Radio baseband processing

• Packet processing

• Security

• AI/ML

• Etc.

© 2018 Nokia26

Data Center Management

• In open ecosystem support for multivendor environment is a mandatory requirement.

• This requires open APIs between different layers

• RSD defines a good framework for data center gear management architecture

• Server management interface standardization is needed

• IPMI is too low level and too much vendor specific extensions.

• DMTF Redfish is doing good work in this area but it is not yet ready.

• Switch management is typically done using SNMP and CLI

• No common way to manage switches today.

• BMC in switches simplifies HW management of switches.

• A lot of integration effort is needed to adapt to customer specific configurations.

© 2018 Nokia27

Summary

© 2018 Nokia28

Call to Action

Get Involved in the OCP Rack & Power Project Group to participate in the Central Office

applications of Open Rack.

http://www.opencompute.org/projects/rack-and-power/

Find out more about Nokia's OCP products via the OCP Marketplace or at Nokia.com.

<Document ID: change ID in footer or remove> <Change information classification in footer>

© 2018 Nokia29

Summary

• Nokia started to use OCP Open Rack compatible designs in telco NFV infrastructure

deployments in 2017.

• Many traditional telco requirements are still mandatory in NFV telco cloud environment.

• Experiences are showing that OCP Open Rack design can work very well in telco use

cases with enhancements that are needed to fulfill telco requirements.

© 2018 Nokia30

Thank You!

© 2018 Nokia31

Come and visit us at

Nokia booth B6World’s first telco grade Open Rack v2 solution