WAN MACsec Deployment White Paper - August 2016

48
CISCO WHITE PAPER WAN MACsec Deployment White Paper August 2016

Transcript of WAN MACsec Deployment White Paper - August 2016

Page 1: WAN MACsec Deployment White Paper - August 2016

CISCO WHITE PAPER

WAN MACsec Deployment White Paper

August 2016

Page 2: WAN MACsec Deployment White Paper - August 2016

Cisco White Paper

Table of ContentsIntroduction ....................................................................................................................................................................1

Technology Overview .....................................................................................................................................................2

MACsec Frame Format ....................................................................................................................................................3

Ethernet WAN Transition for Carrier Services ...................................................................................................................3

Layer 2 Adjacency with MACsec ......................................................................................................................................5

WAN MACsec and IPsec Comparison .............................................................................................................................6

Target Use Cases for MACsec ........................................................................................................................................8

WAN MACsec Deployment Models ...............................................................................................................................10

WAN MACsec Enhancements .......................................................................................................................................12

Clear Tag Option to Enable Point-to-Multipoint Deployments ....................................................................................... 12

Access Control Option for Smoother Migration ............................................................................................................. 12

Configurable EAPoL Destination Address ...................................................................................................................... 13

Configurable Replay-Protection-Window Size .............................................................................................................. 14

Configurable Key Lifetime and Hitless Key Rollover ....................................................................................................... 14

Configurable 128/256-bit Encryption and Hitless Rollover ............................................................................................ 15

CLI Syntax for Configuring Encryption Algorithm for Data Packets ................................................................................ 15

CLI Syntax to Configure Encryption Algorithm for MKA Control Packets ....................................................................... 15

Configuration Guidelines ...............................................................................................................................................16

Use Case 1: Point to Point E-Line Service .................................................................................................................... 17

Use Case 2: E-LAN Service (VPLS Service) ................................................................................................................. 22

Use Case 3: MACsec + SGT Transport ......................................................................................................................... 26

Configurable MKA, Key Chain & MACsec Parameters ...................................................................................................28

Performing Maintenance Tasks .....................................................................................................................................30

Performing Maintenance Tasks (without Impacting Traffic) ........................................................................................... 30

Performing Maintenance Tasks (Traffic Impacting) ........................................................................................................ 32

Scalability .....................................................................................................................................................................34

Monitoring and Troubleshooting ...................................................................................................................................35

Monitoring MACsec Sessions: Sample Output .............................................................................................................. 35

Monitoring MKA Sessions: Sample Output .................................................................................................................... 36

Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings ........................................38

WAN MACsec Best Practices .......................................................................................................................................42

Caveats and Restrictions ..............................................................................................................................................43

Feature Limitations ........................................................................................................................................................ 43

Glossary .......................................................................................................................................................................44

Page 3: WAN MACsec Deployment White Paper - August 2016

page 1Cisco White Paper

Introduction

IntroductionThis white paper provides a reference point for Cisco sales staff, support teams, and organizations to use when deploying media access control security (MACsec) in common use cases for WAN.

You can use WAN MACsec innovation from Cisco in order to optimize and simplify the overall deployment, func-tionality, and operation of networks that require high-speed encryption beyond what IPsec can deliver, without sacrificing performance and packet size agility. WAN MACsec provides a line-rate network encryption solution over Layer 2 Ethernet transport services. MACsec is no longer just a LAN technology and can be leveraged outside campus networks, whether it be over Metro Ethernet transport or Data Center Interconnect (DCI) links. MACsec also secures WAN connections that are leveraging Ethernet as the link-layer media.

This paper provides an overview of MACsec beyond the campus LAN. MACsec can be a formidable encryption solution for WAN and Metro Ethernet links. This document provides an overview of MACsec, compares MACsec with current IP-based encryption solutions, and highlights key WAN/Metro use cases.

The goal of this white paper is to provide the details necessary to configure WAN MACsec, a line-rate network encryption solution over Layer 2 Ethernet Transport Services in the Cisco ASR 1001-X platform. This document also describes the best practices and recommendations for Cisco WAN MACsec deployments on ASR 1001-X platforms. The solution has been thoroughly validated during the system test cycle, and some of the deployment recommendations are included.

Page 4: WAN MACsec Deployment White Paper - August 2016

page 2Cisco White Paper

Technology Overview

Technology OverviewFigure 1 Data encryption using MACsec for a WAN connection between routers

Encrypted Data

MACsec

Encrypt

Decrypt

Encrypted Data

EVCs

L2 Service ProviderNetwork

60

20

F

The WAN MACsec solution:

• Is an IEEE 802.1AE standards-based Layer 2 hop-by-hop encryption that provides data confidentiality and integrity for media access independent protocols.

• Encrypts all data except for the source and destination MAC addresses of an Ethernet packet.

• Mitigates packet eavesdropping, tampering, and injection.

• Supports:

◦ IEEE 802.1AE-2006 compliant GCM-AES-128 and IEEE 802.1AEbn-2011 compliant GCM-AES-256 MACsec, NIST-approved, 1/10 Gb line-rate encryption.

◦ Point-to-point and point-to-multipoint configurations.

◦ Extending from hop-to-hop service to peer-to-peer service.

◦ The option to use VLAN tag in clear.

◦ The option to modify Extensible Authentication Protocol over LAN (EAPoL) Destination MAC and EtherType to avoid MACsec key agreement (MKA) protocol data unit being consumed by intermediate devices.

◦ The option to change access-control for smoother migration.

• Is typically done through hardware (ASIC/PHY) and provides line-rate throughput.

Page 5: WAN MACsec Deployment White Paper - August 2016

page 3Cisco White Paper

Technology Overview

MACSEC FRAME FORMATThe MACsec frame format is similar to that of standard Ethernet but includes an additional 32-byte MACsec header, which includes a well-known EtherType field.

Figure 2 MACsec frame format

Encrypted

Authenticated

DMAC

MACsec EtherType TCI/AN SL Packet Number SCI (optional)

SMAC 802.1AE Header 802.1Q CMD ETYPE PAYLOAD ICV CRC

TrustSec Frame Format

MACsec Tag Format

0x88E5

60

21

F

As shown in the above figure, the frame format using MACsec leverages a tag format with a MACsec EtherType (0x88E5), while allowing the Ethernet source/destination MAC addresses to be left in the clear for Ethernet frame forwarding. Also note that MACsec encrypts all fields behind the source/destination MAC addresses, so unless the ability to offset the encrypted field exists, fields such as MPLS labels and 802.1Q tags are encrypted and not able to be used when the Ethernet frame traverses the underlying transport between encrypted stations. The Cisco implementation of WAN MACsec described in this white paper allows the 802.1Q header to be left in the clear, offering a vast amount of network design flexibility as it relates to MACsec frames traversing a public Ether-net transport.

ETHERNET WAN TRANSITION FOR CARRIER SERVICESWAN and metro service provider offerings are replacing existing T1, ATM/frame relay, and SONET options for their customers in favor of low-cost Ethernet transport, which offers:

• Highly flexible, granular and scalable bandwidth.

• Simple troubleshooting.

• Enterprise-maintained networking and routing decisions.

• Easily added new locations (to L2VPN).

• Ubiquitous use for router ports with Ethernet support.

MACsec, a hop-by-hop encryption technology (per link), offers several advantages over existing IPsec encryption solutions in certain designs.

Page 6: WAN MACsec Deployment White Paper - August 2016

page 4Cisco White Paper

Technology Overview

Figure 3 MACsec encryption process, applied per link

Enterprise WAN

802.1AE MACsec Secured Ethernet WAN Link

A

B

EC

D F

Per “Hop” LinkEncryption over

the WAN

EnterpriseLAN

EnterpriseLAN

EnterpriseLAN

EnterpriseLAN

60

23

F

As shown in the above figure, the MACsec encryption process is applied per link, so as the Ethernet frame enters the PHY layer encapsulation process, the MACsec encryption is applied as well.

The PHY layer processing for MACsec is performed as the last process prior to the frame being sent to the egress port to be put on the physical connection to the neighbor device. Leveraging the encryption process late in the frame-forwarding procedure offers several advantages, most notably:

• It has no impact on Ethernet frame markings (802.1p for QoS, 802.1Q tag, Q-in-Q tags).

• Certain implementations have the option to offset where MACsec encryption is applied, leaving chosen tags in the clear, which can be advantageous to the Ethernet transport provider and the services offered based on 802.1Q and/or 802.1p bits (example: class of service).

In the case where an IP packet traverses multiple router hops (example: multiple P routers in an MPLS backbone) to get from router A to router E, because the links are MACsec encrypted over the Ethernet link, each router hop must encrypt/decrypt the frame. Some designers may believe this is a disadvantage in using MACsec, but it is no different than networks where the source/destination of the packet traverses multiple router hops, because the encryption process is done in the hardware PHY of the MAC ASIC. This is case with the ASR 1001-X — there is no encryption processing penalty when you enable MACsec.

Page 7: WAN MACsec Deployment White Paper - August 2016

page 5Cisco White Paper

Technology Overview

LAYER 2 ADJACENCY WITH MACSECCharacteristics of CE-to-CE deployment packet format:

• There is Layer 2 adjacency between CEs via EoMPLS or VPLS cloud.

• MACsec is required on CEs only. The cloud doesn’t need to be MACsec-capable.

• The purpose is to secure the untrusted WAN link.

Figure 4 Layer 2 adjacency with MACsec

CEASR1K

Egress

PE

PE

PE

CEASR1K

CEASR1K

Remote Site 3with ASR1K at edge

Remote Site 2with ASR1K at edgeMACsec encryption between L2

adjacent CEs at remote site edges

MPLSCloud

EoMPLS/VPLS

Remote Site 1with ASR1K at edge

60

24

F

Page 8: WAN MACsec Deployment White Paper - August 2016

page 6Cisco White Paper

WAN MACsec and IPsec Comparison

WAN MACsec and IPsec ComparisonWhen considering encryption options over a WAN/MAN, IPsec has clearly been the preferred choice given it leverages the ubiquitous nature and broad footprint of IP. This allows a very broad set of transport options, from private Layer 3 VPN services to the public Internet.

Although IPsec solves most encryption requirements, one of its limitations has been performance, specifically as it relates high-speed links that require line-rate performance, regardless of the packet size and speed.

When comparing MACsec and IPsec, the more generic comparison is Layer 2 link encryption to Layer 3 IP en-cryption. Layer 2 encryption dictates that the encryption is being done at Layer 2 of the OSI model, so examples of Layer 2 encryption solutions available include ATM, serial, SONET, and Ethernet. With link encryption, the en-cryption is done at Layer 2 for any of the chosen technologies, and applications and protocols at the layers above Layer 2 (Layers 3-7) are transparent, and part of the payload being encrypted. Because Layer 2 encryption is link-layer aware, it is dependent on the Layer 2 transport chosen for the network design (example: SONET net-work for SONET, Ethernet transport for Ethernet encryption, etc.), thus leveraging the forwarding of the encrypted frame to be performed in the transport of choice.

Layer 3 encryption (example: IPsec), on the other hand, operates at the IP layer of the OSI model, thus encrypt-ing data packets within the IP protocol. IPsec, as described in RFC 4301, offers packet-based encryption, which drastically changes the encryption model to that of Layer 2 link encryption. With Layer 3, encryption between end-stations is not limited to a Layer 2 technology but instead uses IP, allowing the underlying IP transport to be leveraged for connectivity between those end-stations. Because IP networks span globally (example: the Inter-net), Layer 3 encryption is much more flexible because it can leverage any packet-based IP transport. Layer 2 encryption, in contrast, is limited to the specific Layer 2 transport and does not normally span as broad a network footprint. Layer 3 encryption is unaware of the Layer 2 transport networks/links the packet traverses, and when one thinks of scale and reachability, nothing can rival the ubiquitous nature of IP, which IPsec leverages.

Given this comparison, one may ask “why even consider Layer 2 encryption over Layer 3?” Although IP has dominated network designs, both solutions have advantages and disadvantages and can simplify the integration of encryption in certain network design use cases, and Layer 2 encryption does have some clear differentiators that offer the network designer options.

One of the key differentiators MACsec offers over IPsec (beyond a simpler implementation) is its abilities to offer an AES-128/AES-256 encrypted solution at 100Mbps and to remain transparent to the IPv4/v6 packet layer, as well as not impacting the use of MPLS label(s) appended to the Ethernet frame.

When evaluating the various encryption solutions that exist in network deployments, it is important to not to evalu-ate whether one technology is superior to the other. Instead, your focus should be which technology:

• Best meets the business objectives of the end-user and application experience.

• Simplifies operations.

• Offers the most cost-effective solution.

These technologies are never one-size-fits-all, so it is up to the network designer to understand the holistic view of the network so the proposed solution aligns with the business objectives and services the network aims to offer. This is a key component when evaluating encryption technologies or any network solution and transport overall.

Page 9: WAN MACsec Deployment White Paper - August 2016

page 7Cisco White Paper

WAN MACsec and IPsec Comparison

Table 1 Comparison of WAN MACsec and IPsec encryption solutions

Category WAN MACsec IPsec

Market positioning Aggregate deployments such as regional hubs

Large branches that require high throughput

Data center interconnects

Small branches

High scale deployments

Low throughput branches

Beyond MetroE (interna-tional) reach

Link requirement Requires dedicated MetroE EVC circuits for L2 connectivity between sites

Easily routable over many commonly available public networks

Encryption performance Per PHY link speed (1 Gb, 10 Gb)

Constrained by IPsec crypto engine perfor-mance

Services enablement No impact to encryption throughput

Impacts encryption throughput

Peers scale Limited by hardware resources Highly scalable

Throughput Up to line rate on each port (limited only by the forwarding capability)

Aggregate throughput (limited by the encryption throughput)

Configurability Simple configuration More complex configura-tion and policy choices

Layer 3 visibility for monitoring

No. Except Layer 2 headers (and optionally VLAN/MPLS labels). Everything else is en-crypted.

Visible. Layer 3 info can be used for monitoring and policy enforcement purposes

NAT environment L3 header is not accessible. Works with NAT environ-ment.

Page 10: WAN MACsec Deployment White Paper - August 2016

page 8Cisco White Paper

Target Use Cases for MACsec

Target Use Cases for MACsecAs discussed, both Layer 2 and Layer 3 encryption technologies have advantages and disadvantages. However, MACsec does offer several unique advantages, given the required transport to carry an Ethernet frame exists. Link-layer encryption, within itself, offers an advantage over IPsec in that it does not require an overlay transport technology (example: IP in IP). Secondly, for MPLS deployments, technologies that require the use of MPLS over IP for use with IPsec (UDP or GRE) are not required, because the MPLS label is imposed on the Ethernet frame prior to MACsec encryption. This offers several advantages, including:

• Eliminating the need to implement MPLS over IP or any over-the-top solutions when encryption is required, thus simplifying the implementation.

• Reducing the packet overhead (MACsec = 32 bytes) versus IPsec (GRE+Tunnel mode = ~74 bytes).

• Encrypting (that is, hiding) the label and/or 802.1Q tag, leaving only the source/destination MAC address in the clear between stations.

• Allowing customers the ability to leverage cost-effective Ethernet transport services.

• Encrypting at Ethernet speeds.

• Targeting peer-model network designs (versus flat multipoint Ethernet network transport).

Considering these advantages, there are several key situations when MACsec could be considered over IPsec in the design, including when:

• High encryption rate is required.

• The WAN/MAN transport service used is Ethernet.

• The backbone routers are interconnected with either fiber/wavelength or Ethernet, in a point-to-point and/or partial mesh topology design

• Per-link encryption is suitable for the given design for core/edge router connectivity within the backbone topology (that is, the core/edge routers leverage point-to-point Ethernet links for interconnection).

As the MACsec technology continues to evolve, consider the use cases that extend far beyond campus and data center designs. MACsec opens up a whole new paradigm as it relates to secure WAN and Metro Ethernet designs. The following WAN and Metro implementations require data encryption over untrusted networks and can benefit from Layer 2 MACsec encryption:

• Securing router core links (IP/MPLS, PE-P, P-P)—Secure high-speed backbone transport links (point-to-point today), such as the following:

◦ Optical transport hand-off is client optics from a DWDM system, which are easy to tap into and intercept

◦ Service provider hand-off is standard Ethernet, which is also easy to intercept

• Securing Metro Ethernet Services—Offer (as a service provider) or leverage secure 10/40/100GE Metro Ethernet services, each link leveraging 802.1AE for encryption

◦ Secure Ethernet Service (point to point)

◦ Secure Ethernet Multipoint Service (example: VPLS) (future)

Page 11: WAN MACsec Deployment White Paper - August 2016

page 9Cisco White Paper

Target Use Cases for MACsec

• Securing PE-CE link transport integration for MPLS IP VPN Services—Secure back-haul to an MPLS BGP VPN service (L3 service)

• Securing N-PE to U-PE/CPE—Backbone PE (L2 service offering)

• Securing links over any Ethernet service—Applications include DCI, access to Cloud utilities, storage replica-tion

• Securing Over-the-Top Ethernet Links—Enterprise/Public Sector encrypts their Ethernet links on their own CPE routers over-the-top of the SP Ethernet service (supported today on point-to-point Ethernet services only)

As discussed in “WAN MACsec and IPsec Comparison,” one technology is not necessarily better than the other; the network designer has options and can evaluate which solution best fits the network requirement given the transport and design criteria that exist. MACsec clearly offers significant design options, given the popularity of Ethernet and the ever-growing Ethernet services that service providers offer customers.

Page 12: WAN MACsec Deployment White Paper - August 2016

page 10Cisco White Paper

WAN MACsec Deployment Models

WAN MACsec Deployment ModelsStandard WAN MACsec definitions call out the following deployment models:

• Data center interconnect

• Branch connectivity

• Aggregation deployment

• Metro-E Ring/EoMPLS

• L2TPv3 (Layer 2 Tunneling Protocol)

• Overlay transport virtualization

WAN MACsec deployment modes:

• Point-to-point, CE-to-CE connectivity using Ethernet private line (EPL) service (port-mode) Typical use cases: head-office-to-large-branch-office connectivity, data center interconnect

• Point-to-point, hub and spoke connectivity using Ethernet virtual private line (EVPL) service (VLAN-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregation sites such as regional hubs, and allowing service multiplexing such that multiple P2P can co-exist on the same physical interface

• Point-to-point, hub-and-spoke connectivity with MACsec and non-MACsec spokes Typical use cases: migration from non-MACsec branches to MACsec branches, service multiplexing such as Metro-E circuit and Internet or IPsec connection on the same physical interface.

• Multipoint-to-multipoint, hub-and-spoke connectivity using Ethernet private LAN (EP-LAN) service (port-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregations sites such as regional hubs in a LAN fashion. Easy to provision and manage.

• Multipoint-to-multipoint, hub-and-spoke connectivity using Ethernet virtual private LAN (EVP-LAN) service (VLAN-mode) Typical use cases: head-office-to-multiple-branch-offices connectivity, interconnecting multiple data cen-ters, connectivity between aggregation sites such as regional hubs in a LAN fashion, and allowing service multiplexing such that multiple LAN can co-exist on the same physical interface. Easy to provision and man-age.

• MACsec with security group tag (SGT) transport Typical use case: carrying SGT over WAN MACsec

Page 13: WAN MACsec Deployment White Paper - August 2016

page 11Cisco White Paper

WAN MACsec Deployment Models

MACsec is commonly deployed in a Metro Ethernet WAN. The following four Metro Ethernet Service types are supported as part of this solution as defined in the Metro Ethernet Forum (MEF).

Table 2 MEF Ethernet Service Types

Service typePort-based services VLAN-based services

E-Line EPL EVPL

Point-to-point, UNI-to-UNI

EPL EVPL

E-LAN EP-LAN EVP-LAN

Multipoint-to-multipoint, UNIs-to-UNIs

EP-LAN EVP-LAN

Page 14: WAN MACsec Deployment White Paper - August 2016

page 12Cisco White Paper

WAN MACsec Enhancements

WAN MACsec EnhancementsThis section details the specific enhancements made to accommodate MACsec in the WAN environment.

CLEAR TAG OPTION TO ENABLE POINT-TO-MULTIPOINT DEPLOYMENTSClear Tag is a Cisco MACsec innovation implemented to enable service multiplexing at the CE by providing tun-neling information such as VLAN tag in the clear so that multiple point-to-point or multipoint services can co-exist on a single physical interface. Enabling this option allows customers to configure a VLAN (802.1Q) tag to bypass the MACsec encryption, as shown in the following figure.

Figure 5 VLAN tag configured to bypass MACsec encryption

Encrypted

AuthenticatedAuthenticated

Eth 802.1Q 802.1AE ETYPE PAYLOAD ICV CRC

60

26

F

The macsec dot1q-in-clear 1 command controls the behavior of allowing 802.1Q tag in the clear. The default is no macsec dot1q-in-clear (that is, no dot1q tag in the clear).

Tech Tip

With Cisco ASR1001-X, macsec dot1q-in-clear 1 can only be configured on physical interface, and the setting is automatically inherited by the sub-interfaces.

ACCESS CONTROL OPTION FOR SMOOTHER MIGRATIONBy default, when MACsec is enabled on an interface, it is expected that the customer’s intention is to secure the entire interface traffic; as such, MACsec does not allow any unencrypted packets to be transmitted or received from the same physical interface. But it is not uncommon to see a mix of MACsec-enabled and non-MACsec-enabled sub-interfaces co-existing, such as during migration (example: customers may wish to enable MACsec for one branch at a time). To accommodate this, an additional Cisco proprietary extension has been implemented to allow unencrypted packets to be transmitted or received from the same physical interface.

The macsec access-control [must-secure | should-secure] command controls the behavior of unencrypted packets processing as follows:

• should-secure allows unencrypted packets to be transmitted and received from the physical interface or sub interfaces.

• must-secure does not allow transmit/receive of unencrypted packets from physical interface or sub inter-faces and drops the packet except for MKA control protocol packets.

• If a mix of MACsec and non-MACsec sub interfaces (example: sub interfaces with IPsec configured) co-exists, then should-secure configuration is required.

Page 15: WAN MACsec Deployment White Paper - August 2016

page 13Cisco White Paper

WAN MACsec Enhancements

The default is macsec access-control must-secure, which is effective as soon as MACsec CLI is configured (that is, unencrypted packets are not allowed to be transmitted or received except for MKA control protocol pack-ets).

Tech Tip

Because should-secure operates on a physical interface level, it cannot be configured on a sub-interface. More importantly, configuring should-secure allows unencrypted traffic to be received even on a secured MACsec session.

CONFIGURABLE EAPOL DESTINATION ADDRESSWAN MACsec provides ability to change the MKA packet’s MAC address in order to adjust to Ethernet networks that do not support MKA transport.

Before establishing a MACsec secure session, MKA as a control protocol is used to select the cipher suite for encryption and exchanges the required keys and parameters between the peers.

MKA uses EAPoL as a transport protocol to transmit MKA messages. By default, EAPoL uses a destination mul-ticast MAC address of 01:80:C2:00:00:03 to multicast the packets to multiple destinations. Since EAPoL is a standards-based protocol and other authentication mechanisms such as IEEE 802.1X also use the same proto-col, there may be situations where the switches in the service provider cloud might consume this packet (based on the destination multicast MAC address)—assuming the packet is destined for them, trying to process it, and eventually dropping it, essentially causing MKA session establishment to fail. To avoid this situation, another Cisco proprietary extension has been implemented; it allows an administrator to change the destination MAC address of an EAPoL packet that is transmitted on an interface towards the service provider so that the service provider tun-nels the packet like any other data packet instead of consuming it.

The eapol destination-address command is used to configure the EAPoL destination MAC address. The follow-ing options are supported:

• H.H.H (any MAC address)

• Bridge-group address (0180.C200.0000)

• LLDP-multicast address (0180.C200.000E)

• Broadcast

• Default: (0180.C200.0003)

Tech Tip

You can configure the EAPoL destination address independently on either physical or sub interface level. If it is configured on the physical interface, it is automatically inherited by the sub interfaces. Explicit configuration on a sub-interface overrides the inherited value or policy for that sub-interface.

Page 16: WAN MACsec Deployment White Paper - August 2016

page 14Cisco White Paper

WAN MACsec Enhancements

CONFIGURABLE REPLAY-PROTECTION-WINDOW SIZEReplay protection is a MACsec feature to counter against replay attacks by assigning a unique sequence number to each encrypted packet and verifying the sequence received at the remote end. Although it is not common to see packets reordered in a campus environment and hence strict replay protection may be used, frames trans-mitted through a Metro Ethernet SP network are highly susceptible to reordering due to various prioritization and load-balancing mechanisms used within the SP network that increase the chances of frames arriving out of order and hence may require a high tolerance to packet reordering.

Again, a replay window is necessary to support use of MACsec over provider networks that reorder frames for various reasons. Frames within the window can be received out of order but are not replay protected. The default window size is set to 64, the same as IPsec. This can be changed to a lower or higher value through the following command:

macsec replay-protection window-size [0-4294967295]

The replay protection window may be set to zero to enforce strict reception ordering and replay protection.

Tech Tip

You can configure a replay protection window independently on either physical or sub interface. If you configure it on the physical interface, it is automatically inherited by the sub interfaces. Explicit configuration on sub-interface overrides the inherited value or policy for that sub-interface.

CONFIGURABLE KEY LIFETIME AND HITLESS KEY ROLLOVERA MACsec key chain can have multiple pre-shared keys, each configured with a key id and an optional lifetime. A key lifetime specifies at which time the key expires. In the absence of a lifetime configuration, the default lifetime is unlimited. When a lifetime is configured, MKA rolls over to the next configured pre-shared key in the key chain after the lifetime is expired. Time zone of the key can be local or UTC. The default time zone is UTC.

key chain [name] macsec

key 01

key-string [Hex string]

lifetime [time] [local]

The hex string must be 32 Hex characters or 64 characters depending on the Cryptographic algorithm selected (example: AES-128-CMAC or AES-256-CMAC, respectively)

Keys roll over to the next key within the same key chain by configuring a second key (key 02) in the key chain and configuring a lifetime for the first key. When the first key (key 01) lifetime expires, it automatically rolls over to the next key in the list. If the same key is configured on both sides of the link at the same time, then the key rollover is hitless (that is, the key rolls over without traffic interruption).

Page 17: WAN MACsec Deployment White Paper - August 2016

page 15Cisco White Paper

WAN MACsec Enhancements

CONFIGURABLE 128/256-BIT ENCRYPTION AND HITLESS ROLLOVERTwo encryption cipher suites must be selected for MACsec to operate: a cipher suite that encrypts data packets and another cipher suite that encrypts MKA control packets.

It is recommended that cipher suites be paired, allowing for consistent cryptography (pair 128 bit ciphers for data and MKA or 256 bit ciphers for data and MKA).

Cipher suite selection for MACsec data packets encryption is as follows:

• MACsec cipher suite to encrypt data packets is configured as part of MKA policy. There can be more than one cipher suite in the list.

• A key server uses the first cipher suite from the list.

• All non-key servers must use the same cipher suite as the key server. The cipher suites can be configured in any order in the non-key server. If the same cipher suite is not available in the list, the session establishment fails.

• In the absence of a MACsec cipher suite configuration, all the peers have a default list, which includes all the cipher suites supported by the local hardware. For ASR1001-X, the cipher suite list is GCM-AES-128, GCM-AES-256.

• If a MACsec cipher suite is not configured by an admin, the default cipher suite of GCM-AES-128 (Galois/Counter Mode of Advanced Encryption Standard cipher with 128-bit key) is used.

CLI SYNTAX FOR CONFIGURING ENCRYPTION ALGORITHM FOR DATA PACKETSmka policy POLICY1

macsec-cipher-suite [gcm-aes-128 | gcm-aes-256]

Cryptographic algorithm selection for MKA control protocol packets encryption is as follows:

• The cryptographic algorithm to encrypt MKA control protocol packets is configured as part of key chain. There can be only one cryptographic algorithm configured per key chain.

• A key server uses the configured MKA cryptographic algorithm from the key chain used.

• All non-key servers must use the same cryptographic algorithm as the key server.

If an admin does not configure an MKA cryptographic algorithm, a default cryptographic algorithm of AES-CMAC-128 (Cipher-based message authentication code with 128-bit Advanced Encryption Standard) is used.

CLI SYNTAX TO CONFIGURE ENCRYPTION ALGORITHM FOR MKA CONTROL PACKETS

key chain [name] macsec

key 01

key-string [Hex string]

cryptographic-algorithm [aes-256-cmac | aes-256-cmac]

Page 18: WAN MACsec Deployment White Paper - August 2016

page 16Cisco White Paper

Configuration Guidelines

Configuration GuidelinesSolution deployment prerequisites:

• Cisco ASR 1001-X with 15.S(01)S or later software. On-board 1 GE and 10 GE ports of ASR 1001-X support WAN MACsec.

• Layer 2 transparent Ethernet Services-MetroE Circuit (EVCs) Service provider network should provide MACsec Layer 2 control protocol transparency such as EAPoL. Note: The WAN MACsec solution in ASR1001-X depends on MKA (as defined in 802.1X-2010) for advertis-ing capabilities, exchange keying material, cipher suite negotiations, etc. MKA uses EAPoL protocol.

There is no special license needed for WAN MACsec, but it is available as part of regular crypto RTU license.

Figure 6 Network topology

MPLS Cloud

Carrier Ethernet ServiceEoMPLS/VPLS

PE3

PE4

PE1 PE2ASR 1001-X CE1

Traffic Generatorin CE1 VLAN

ASR 1001-X CE3

ASR 1001-X CE2

E-LINE (P2P)E-LAN (P2MP)

MACsec Encrypted Link

60

25

F

Page 19: WAN MACsec Deployment White Paper - August 2016

page 17Cisco White Paper

Configuration Guidelines

USE CASE 1: POINT TO POINT E-LINE SERVICE

Point-to-Point SA Configuration—CE to CE (Port-based E-Line Service)

EnterpriseNetwork

Remote Site

• MACsec enabled interface • Physical • Sub-interface (802.1Q)

EnterpriseNetwork

CentralCampus/DC

MKA Keying(802.1X-2010)

Carrier EthernetService

E-LINE (P2P)

MKA Session

MACsec Flow

MKA Key

MACsec Interface

CE2 CE1

60

27

F

Ethernet service:

• Point-to-point pseudo wire service (no MAC address lookup)

• Port-mode

This guide uses the following conventions for commands that you enter at the command-line interface (CLI).

Commands to enter at a CLI prompt: configure terminal

Commands that specify a value for a variable: ntp server 10.10.48.17

Commands with variables that you must de�ne: class-map [highest class name]

Commands at a CLI or script prompt: Router# enable

Long commands that line wrap are underlined. Enter them as one command:

police rate 10000 pps burst 10000 packets conform-action

Noteworthy parts of system output (or of device con�guration �les) are highlighted: interface Vlan64 ip address 10.5.204.5 255.255.255.0

How to Read Commands

Page 20: WAN MACsec Deployment White Paper - August 2016

page 18Cisco White Paper

Configuration Guidelines

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

CE2 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

ip address 10.3.1.2 255.255.255.0

mka pre-shared-key key-chain KEY1

macsec

Point to Point SA Configuration—Hub and Spoke, VLAN-based E-Line Service (P2P)—Only MACsec sub-interfaces

EnterpriseNetwork

EnterpriseNetwork

Branch Site

EnterpriseNetwork

Branch Site

• MACsec enabled interface • Physical • Sub-interface (802.1Q)

Port-mode wouldrequire 2 physical

interfaces

MKA Keying(802.1X-2010)

Carrier EthernetService

E-LINE (P2P)

CE2 CE1

CE3 VLAN20

VLAN10

CentralCampus/DC

60

28

F

Ethernet service:

• Point-to-point PW service (no MAC address lookup)

• Port-mode or 802.1Q offering

Page 21: WAN MACsec Deployment White Paper - August 2016

page 19Cisco White Paper

Configuration Guidelines

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

interface GigabitEthernet0/0/4.2

encapsulation dot1Q 20

ip address 10.3.2.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

CE2 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

Page 22: WAN MACsec Deployment White Paper - August 2016

page 20Cisco White Paper

Configuration Guidelines

CE3 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

Point-to-Point SA Configuration—Mix of MACsec and Non-MACsec Spokes, VLAN-based E-Line Service (P2P)

EnterpriseNetwork

Branch Site

EnterpriseNetwork

Branch Site

CentralCampus/DC

• MACsec enabled interface • Physical • Sub-interface (802.1Q)

MKA Keying(802.1X-2010)

Carrier EthernetService

E-LINE (P2P)

CE2 CE1

CE3

CE4

EnterpriseNetwork

60

29

F

Ethernet service

• Point-to-point PW service (no MAC address lookup)

• Port-mode or 802.1Q offering

Page 23: WAN MACsec Deployment White Paper - August 2016

page 21Cisco White Paper

Configuration Guidelines

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec access-control should-secure

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

interface GigabitEthernet0/0/4.2

encapsulation dot1Q 20

ip address 10.3.2.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

interface GigabitEthernet0/0/4.3

encapsulation dot1Q 30

ip address 10.3.3.1 255.255.255.0

CE2 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec access-control should-secure

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.2 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

Page 24: WAN MACsec Deployment White Paper - August 2016

page 22Cisco White Paper

Configuration Guidelines

CE3 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 20

ip address 10.3.2.2 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

CE4 Configurationinterface GigabitEthernet0/0/4.1

encapsulation dot1Q 30

ip address 10.3.3.2 255.255.255.0

USE CASE 2: E-LAN SERVICE (VPLS SERVICE)

Point-to-Point SA Configuration—Hub and Spoke

Branch Site

EnterpriseNetwork

Branch Site

EnterpriseNetwork

EnterpriseNetwork

CentralCampus/DC

MKA Keying(802.1X-2010)

Carrier EthernetService

E-LAN (multi-point)

CE2 CE1

CE3

60

31

F

Ethernet Service

• Multipoint service (typically VPLS)

• Port mode or 802.1Q offering

Page 25: WAN MACsec Deployment White Paper - August 2016

page 23Cisco White Paper

Configuration Guidelines

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

eapol destination-address broadcast

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

CE2/CE3 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.2 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

macsec replay-protection-window-size 1000

eapol destination-address broadcast

Page 26: WAN MACsec Deployment White Paper - August 2016

page 24Cisco White Paper

Configuration Guidelines

Point to Point SA Configuration—Hub and Spoke, Spoke to Spoke

Figure 7 Point to Point SA Configuration—Hub and Spoke, Spoke to Spoke

Branch Site

EnterpriseNetwork

Branch Site

EnterpriseNetwork

EnterpriseNetwork

CentralCampus/DC

MKA Keying(802.1X-2010)

Carrier EthernetService

E-LAN (multi-point)

CE2 CE1

CE3

• MACsec enabled interface • Physical • Sub-interface (802.1Q)

60

32

F

Multiple VLAN-based E-LAN Services (P2MP)—Multiple VLAN-based E-LAN Services (P2MP)

Metro EthernetNetwork

P2MPEVCsVLAN 10

VLAN 10

VLAN 10

VLAN 20

VLA

N 2

0 VLA

N 20

CE1

CE2

CE3

CE4 CE5 60

33

F

Ethernet Service

• Multipoint service (typically VPLS)

• Port mode or 802.1Q offering

Page 27: WAN MACsec Deployment White Paper - August 2016

page 25Cisco White Paper

Configuration Guidelines

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

interface GigabitEthernet0/0/4.2

encapsulation dot1Q 20

ip address 10.3.2.1 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

CE2/CE3 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.1

encapsulation dot1Q 10

ip address 10.3.1.2 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

Page 28: WAN MACsec Deployment White Paper - August 2016

page 26Cisco White Paper

Configuration Guidelines

CE4/CE5 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

macsec dot1q-in-clear 1

macsec replay-protection-window-size 1000

interface GigabitEthernet0/0/4.2

encapsulation dot1Q 20

ip address 10.3.2.2 255.255.255.0

mka pre-shared-key key-chain KEY1 macsec

USE CASE 3: MACSEC + SGT TRANSPORT

Branch Site

EnterpriseNetwork

EnterpriseNetwork

CentralCampus/DC

Classification

• MACsec enabled interface • Physical • Sub-interface (802.1Q)

MKA Keying(802.1X-REV)

SGT Transport

ISE Directory

Carrier EthernetService

E-LINE (P2P)

CE CE

MKA Session

MACsec Flow

MKA Key

MACsec Interface

60

34

F

CE1 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

ip address 10.3.1.1 255.255.255.0

cts manual

propagate sgt

mka pre-shared-key key-chain KEY1 macsec

Page 29: WAN MACsec Deployment White Paper - August 2016

page 27Cisco White Paper

Configuration Guidelines

CE2 Configurationkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface GigabitEthernet0/0/4

ip address 10.3.1.2 255.255.255.0

cts manual

propagate sgt

mka pre-shared-key key-chain KEY1 macsec

Page 30: WAN MACsec Deployment White Paper - August 2016

page 28Cisco White Paper

Configurable MKA, Key Chain & MACsec Parameters

Configurable MKA, Key Chain & MACsec ParametersMKA global policy configurable parameters:

• key-server priority

◦ 0 to 64

◦ Default: 0

• macsec-cipher-suite

◦ macsec-cipher-suite gcm-aes-128

◦ macsec-cipher-suite gcm-aes-256

◦ Default: macsec-cipher-suite gcm-aes-128

• confidentiality-offset

◦ 0, 30, 50

◦ Default: 0

Keychain global configurable parameters:

• key

◦ Key ID

• cryptographic-algorithm

◦ cryptographic-algorithm aes-128-cmac

◦ cryptographic-algorithm aes-256-cmac

◦ Default: cryptographic-algorithm aes-128-cmac

• keystring

◦ Hex Characters

◦ Default: NA

• lifetime

◦ hh:mm:ss

◦ Local time in local time zone

◦ Default: unlimited

Page 31: WAN MACsec Deployment White Paper - August 2016

page 29Cisco White Paper

Configurable MKA, Key Chain & MACsec Parameters

MACsec-interface-configurable commands and parameters:

• macsec replay-protection-window-size

◦ 0-x

◦ Default: 64

• macsec-access-control

◦ must-secure

◦ should-secure

◦ Default: must-secure

• macsec-dot1q-in-clear

◦ 0, 1

◦ Default: 0

• macsec

• eapol destination-address

◦ H.H.H (any mac address)

◦ bridge-group-address

◦ lldp-multicast-address

◦ broadcast

◦ Default: (01:80:C2:00:00:03)

Page 32: WAN MACsec Deployment White Paper - August 2016

page 30Cisco White Paper

Performing Maintenance Tasks

Performing Maintenance Tasks

PERFORMING MAINTENANCE TASKS (WITHOUT IMPACTING TRAFFIC)

Changing a Pre-Shared Key (CAK Rollover): Sample ConfigurationKeys can be configured to automatically roll over to the next key by configuring a lifetime on both routers.

Fromkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

Tokey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

lifetime local 10:30:00 Oct 30 2014 11:30:00 Oct 30 2014

key 02

key-string 11145678901234567890123456789012

Changing a Key Chain (Keychain Rollover): Sample Configuration

Fromkey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

interface TenGigabitEthernet0/0/0.10

mka pre-shared-key key-chain KEY1

Tokey chain KEY1 macsec

key 01

key-string 12345678901234567890123456789012

key chain KEY2 macsec

key 01

key-string abcdef0987654321abcdef0987654321

interface TenGigabitEthernet0/0/0.10

mka pre-shared-key key-chain KEY2

Page 33: WAN MACsec Deployment White Paper - August 2016

page 31Cisco White Paper

Performing Maintenance Tasks

Changing a Key Server Role: Sample ConfigurationYou can force a router to become a key server by configuring a lower priority than other peer routers that par-ticipate in the same session. It is recommended that you configure a key server priority so that the key server selection is deterministic. For example, in a hub-and-spoke scenario, the ideal place for a key server is the hub site router.

Hub Site (Key Server)mka policy POLICY1

key-server priority 0

interface TenGigabitEthernet0/0/0.10

mka pre-shared-key key-chain KEY1

mka policy POLICY1

Spoke Sites (Non-Key Servers)mka policy POLICY1

key-server priority 1

interface TenGigabitEthernet0/0/0.10

mka pre-shared-key key-chain KEY1

mka policy POLICY1

Changing MACsec Cipher-Suite to Encrypt Data Traffic with 128/256 bit Encryptionmka policy POLICY1

macsec-cipher-suite gcm-aes-128

# alternative configuration using AES 256

# macsec-cipher-suite gcm-aes-256

interface GigabitEthernet0/0/1.10

mka policy POLICY1

Changing MKA Cipher-Suite to Encrypt Control Traffic with 128/256 bit Encryptionkey chain KEY3 macsec

key 01

key-string abcdef0987654321abcdef0987654321

cryptographic-algorithm aes-128-cmac

# alternative configuration using AES 256

# cryptographic-algorithm aes-256-cmac

interface TenGigabitEthernet0/0/0.10

mka pre-shared-key key-chain KEY3

Page 34: WAN MACsec Deployment White Paper - August 2016

page 32Cisco White Paper

Performing Maintenance Tasks

Changing EAPoL Destination MAC AddressThe EAPoL destination MAC address can be changed from physical interface configuration mode or sub-interface configuration mode and is automatically inherited by the sub-interfaces if configured at the physical-interface level. If you need to override the inherited value, configure it at the sub interface mode. The default EAPoL desti-nation MAC address is 01:80:c2:00:00:03.

interface TenGigabitEthernet0/0/0

eapol destination-address [ip address | bridge-group-address | lldp-multicast-address]

Changing EAPoL EtherTypeMEF standards define a set of MAC addresses that carrier Ethernet transit switches consume for their own control plane processing requirements. Current MACsec and MKA implementations negotiate MKA keys using an EAPoL packet. Because these EAPoL MAC addresses are part of the set of MAC addresses defined by the MEF to be consumed by the provider switches, customers deploying MACsec over a public carrier Ethernet transport cannot use MACsec across these providers without changing the default MACsec behavior using EAPoL MAC addresses.

To mitigate this problem, Cisco introduced the ability for the operator deploying WAN MACsec to change the EAPoL destination address and/or EtherType to an address that is not consumed by the provider equipment.

The EAPoL destination EtherType can be changed to an alternate value from the default EtherType of 88E5, to avoid being consumed by a provider bridge.

interface TenGigabitEthernet0/0/0

eapol eth-type 876F

Changing Confidentiality Offsetmka policy POLICY1

confidentiality-offset 30

interface GigabitEthernet0/0/1.10

mka policy POLICY1

PERFORMING MAINTENANCE TASKS (TRAFFIC IMPACTING)

Changing a Replay-Protection-Window SizeReplay protection window can be changed from physical interface configuration mode or sub-interface configu-ration mode and is automatically inherited by the sub interfaces if configured at the physical interface level. If you need to override the inherited value, configure it at the sub interface mode. The default replay protection window size is 64.

interface TenGigabitEthernet0/0/0

macsec replay-protection window-size 10

interface TenGigabitEthernet0/0/0.10

macsec replay-protection window-size 5

Page 35: WAN MACsec Deployment White Paper - August 2016

page 33Cisco White Paper

Performing Maintenance Tasks

Enable/Disable VLAN (dot1q) tag in the clear optionThe macsec dot1q-in-clear command can only be configured on physical interface, and the setting is automati-cally inherited by the sub-interfaces.

interface GigabitEthernet0/0/1

macsec dot1q-in-clear 1

Enable/Disable Unencrypted Packets Processing (Access Control Setting)The macsec access-control must-secure/should-secure command can only be configured on physical inter-face, and the setting is automatically inherited by the sub-interfaces.

interface GigabitEthernet0/0/1

macsec [access-control must-secure | should-secure]

Page 36: WAN MACsec Deployment White Paper - August 2016

page 34Cisco White Paper

Scalability

ScalabilityTable 3 Number of supported MACsec peers per port in ASR 1001-X

1 Gb 10 Gb

Max # of SA 16 (receive)+16 (transmit) 64 (receive)+64 (transmit)

Max # of SC 8 (receive)+8 (transmit) 32 (receive)+32 (transmit)

Max # of point-to-point 8 32

Max # of sites in P2MP setup 8 32

Max # of group CA4 (with 2 members in each group)

16 (with 2 members in each group)

Although MACsec is supported up to line-rate on each interface, the forwarding capability may be limited by the maximum system forwarding capability.

Page 37: WAN MACsec Deployment White Paper - August 2016

page 35Cisco White Paper

Monitoring and Troubleshooting

Monitoring and TroubleshootingYou can use the following show commands to monitor MACsec session details:

• show macsec summary

• show macsec statistics interface [interface name]

• show macsec status interface [interface name]

MONITORING MACSEC SESSIONS: SAMPLE OUTPUTR2#show macsec summary

MACsec Capable Interface Extension

---------------------------------------------

TenGigabitEthernet0/0/1 One tag-in-clear

GigabitEthernet0/0/1 One tag-in-clear

MACsec Enabled Interface Receive SC VLAN

-----------------------------------------------------

GigabitEthernet0/0/1.10 : 8 10

R2#show macsec status int gi0/0/1.10

Capabilities:

Validate Frames: Strict

Ciphers Supported: GCM-AES-128 GCM-AES-256

Include SCI: Yes

Cipher: GCM-AES-128

Confidentiality Offset: 0

Transmit SC:

SCI: 0022BDEF43830014

Transmitting: TRUE

Transmit SA:

Next PN: 1712

Receive SC:

Receiving: TRUE

Receive SA:

In Use: TRUE

Next PN: 1731

Page 38: WAN MACsec Deployment White Paper - August 2016

page 36Cisco White Paper

Monitoring and Troubleshooting

You can use the following show commands to monitor MKA session details:

• show mka sessions

• show mka sessions detail

• show mka sessions interface [interface] port [port] detail

• show mka policy [MKA policy name]

MONITORING MKA SESSIONS: SAMPLE OUTPUT

Figure 8 Show MKA sessions, 1 MKA session with 5 MACsec peers

You can use the following debug commands to collect debug information:

• debug mka [events | errors | packets] Use: Troubleshooting MKA session bring-up issues

• debug mka linksec-interface Use: Troubleshooting MKA keep-alive issues

• debug platform software macsec [error | info | detail | none] Use: MACsec info/error debugging

Page 39: WAN MACsec Deployment White Paper - August 2016

page 37Cisco White Paper

Monitoring and Troubleshooting

Example syslog messages%MKA-5-SESSION_START: MKA Session started

%MKA-5-SESSION_SECURED: MKA Session was secured

%MKA-4-SESSION_UNSECURED: MKA Session was stopped

%MKA-5-SESSION_STOP: MKA Session stopped by MKA

%MKA-5-CAK_REKEY: MKA Session is beginning a CAK Rekey

%MKA-6-CAK_REKEY_SUCCESS: MKA Session CAK rekey is successful

%MKA-6-SAK_REKEY: MKA Session is beginning a SAK Rekey

%MKA-4-SAK_TRANSMIT: Installing new TxSA

%MKA-6-SAK_REKEY_SUCCESS: MKA Session successfully completed a SAK Rekey

%MKA-4-MKA_MACSEC_CIPHER_MISMATCH: Lower strength MKA-cipher than macsec-cipher

%MACSEC-6-RX_SC_EXCEED: RX SCI %x : TX SCI %x : vport %d : secy vport %d

%MACSEC-6-TX_SA_PN_EXPIRE: (TX SCI %x : AN %d) TX SA AN about to expire

Page 40: WAN MACsec Deployment White Paper - August 2016

page 38Cisco White Paper

Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings

Network Design Considerations When Leveraging MACsec over Ethernet Service OfferingsOne critical aspect of using Ethernet encryption is its ability to create flat network topologies, specifically for connecting core routers. This is not recommended. Flat Ethernet transport networks can cause many problems and shortfalls when you are designing a core backbone, and you should take caution as to how the network is designed. Ethernet has an inherent broadcast element, so you might view this as an advantage (because 10s or 100s of routers can have a single point of access into this Ethernet cloud) and leverage MACsec for encryp-tion. But a problem is created with N-1 concept (N being the number of routers are attached to the flat Ethernet domain).

Figure 9 Router peering model for E-LAN (example 1)

EthernetTransport

Physical View Logical ViewPE1 PE2

PE4 PE3

PE1

Routers areAttached to“Flat” BridgeDomain

PE2

PE4 PE3

FlatEthernetBridgeDomain

60

35

F

In Figure 10, each PE router has a single physical attachment into the bridge domain. Each router creates a rout-ing adjacency (an IGP or BGP) with every other router on the network, so applying N-1, each router establishes three routing adjacencies. Now, consider 100 routers attached to the same bridge domain, and again, applying N-1 routing adjacencies. Each PE now contains 99 IGP/BGP routing adjacencies. This flat network model poses many significant challenges, including:

• The N-1 maintenance of routing protocol adjacencies.

• The inability to efficiently prune routers from a given multicast tree.

The lack of state for QoS on SLAs for bandwidth being sent per peer is because the bandwidth is shared at the access links (example: 99 sites can send to 1, creating a major over-subscription factor for the access link).

Page 41: WAN MACsec Deployment White Paper - August 2016

page 39Cisco White Paper

Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings

Figure 10 Router peering model view for E-LAN (example 2)

PE1 PE2

PE4 PE3

EthernetVirtualCircuitService

Ethernet Sub-interfacewith 802.1Q Support

802.1Q Trunk

EthernetTransport

PE1 PE2

PE4 PE3

Appears as Routersare Peering OverSecure Ethernet Wire

Ethernet Sub-interfacewith 802.1Q Support

Physical View Logical View

60

36

F

For core link interconnect, the recommended approach (Figure 11) is to leverage a point-to-point Ethernet ser-vice that allows the deployment of a peering model and is supported with MACsec link encryption. This peering approach offers several key advantages over the flat multipoint model, including:

• Much better scaling from a routing peer perspective.

• Deterministic QoS, because an SLA can be applied per Ethernet VC, quantifying the amount of bandwidth per logical connection.

• Deterministic traffic shaping per logical connection, eliminating the overrun of a particular site (based on how the logical connections are sized).

As previously discussed, the Ethernet encryption concept offers advantages over IPsec and overlay networks, but from an IP design perspective, there are scaling vulnerabilities lurking if the designer is not cautious.

Assuming the customer/agency is leveraging these various Ethernet services along with MACsec in order to gain the benefits of a more simplistic deployment, a hybrid approach can be taken. The benefits of the hybrid ap-proach include:

• A hierarchical network design that allows massive scale for routing.

• Leveraging hierarchy to segment core from aggregation.

• Leverage both point-to-point and multipoint Ethernet services.

• Use point-to-point Ethernet services in the core, while leveraging multipoint service at the access/aggrega-tion back-haul.

Page 42: WAN MACsec Deployment White Paper - August 2016

page 40Cisco White Paper

Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings

Figure 11 Flat Ethernet E-LAN multipoint service

Regional POP

EthernetTransport

60

30

F

In Figure 12, there is a mix of small and large sites, most regionally located, sharing the same flat Ethernet ser-vice. This design poses many of the challenges mentioned above when leveraging a flat Ethernet service, and in this example, there are 24 total routers sharing the Ethernet service, so each router contains 23 routing adjacen-cies. In addition, there is no deterministic nature for how traffic is sent to a given site, and none of the routers on the shared Ethernet domain will ever have the awareness of senders and receivers of traffic, so it is easy to over-run the access circuit from CPE to Cloud.

One approach to overcoming these challenges, and building the foundation for a massively scalable network de-sign, is to take a hierarchical approach.

Page 43: WAN MACsec Deployment White Paper - August 2016

page 41Cisco White Paper

Network Design Considerations When Leveraging MACsec over Ethernet Service Offerings

Figure 12 Hybrid Ethernet E-LAN multipoint service/Ethernet point to point service

Regional POP

Ethernet “Point-to-Point” Service (E-LINE)

Ethernet “Multipoint” Service (E-LAN)

60

22

F

The hybrid model demonstrated in Figure 13 is leveraging the same routers and locations as in Figure 12, but the hybrid model applies a hierarchical design while leveraging both point-to-point and multipoint Ethernet service capabilities. The locations are split into regional PoPs, which dedicate a bridge domain and core router. Multi-point Ethernet service is leveraged for the back-haul of remote-site/branch locations, and a point-to-point E-Line service is used for core router interconnects. This approach allows deterministic QoS within the core and sim-plicity of multipoint for back-haul, while limiting the number of routing adjacencies needing to be maintained on each router. In Figure 12, in contrast, the peering model is N-1.

In the hybrid model, the maximum routing adjacency is seven (versus 23 in the flat model), but more importantly, this design offers massive scalability and can leverage common core technologies such as MPLS traffic engi-neering for bandwidth management, L2/L3 VPN services, and rapid convergence to name a few.

When you are leveraging Ethernet and MACsec, understanding the impact the Ethernet service can have on the overall design is critical to choosing the appropriate transport service.

Page 44: WAN MACsec Deployment White Paper - August 2016

page 42Cisco White Paper

WAN MACsec Best Practices

WAN MACsec Best Practices • Ensure basic Layer 2 Ethernet connectivity is established and verified before attempting to enable MACsec.

(Example: a basic ping between the CEs).

• When configuring for the first time, ensure that you have out-of-band connectivity to the remote site in order to avoid locking yourself out after enabling MACsec if the session fails to establish.

• It is recommended that you configure access-control should-secure while enabling MACsec for the first time and subsequently remove the CLI to change to default “Must secure” after the session establishment is successful, unless it is needed for migration cases as mentioned in “Point-to-Point SA Configuration—Mix of MACsec and Non-MACsec Spokes, VLAN-based E-Line Service (P2P).”

• It is recommended that you configure an interface MTU adjusting for MACsec overhead, ~32 bytes. Although MACsec encryption/decryption occurs at the PHY level and MTU is not an issue for the source or destination router, it may be an issue for the intermediate SP router. Configuring an MTU value at the interface allows for MTU negotiation that includes MACsec overhead.

Page 45: WAN MACsec Deployment White Paper - August 2016

page 43Cisco White Paper

Caveats and Restrictions

Caveats and Restrictions • macsec dot1q-in-clear and macsec access-control [must-secure | should-secure] can only be config-

ured on the main interface, and the setting is automatically inherited by the sub-interfaces. Due to hardware restriction, this behavior cannot be changed.

• mka policy, macsec replay-protection-window and eapol destination-address can be configured on the main and/or sub-interface and the value is automatically inherited by the sub-interfaces when configured on the main interface. Explicit configuration on the sub-interface overrides the inherited value or policy for that sub-interface.

• mka pre-shared-key key-chain [name of key-chain] For port based sessions, pre-shared-key should be configured on main interface only. For VLAN based sessions, pre-shared-key should be configured on sub-interface(s) only. If the pre-shared-key is configured on the main interface, it will not be inherited to the sub-interface(s).

FEATURE LIMITATIONSMACsec with EtherChannel (link bundling) is not supported.

• MACsec configuration on EtherChannel is not supported.

• MACsec-configured interfaces cannot be part of EtherChannel.

Page 46: WAN MACsec Deployment White Paper - August 2016

page 44Cisco White Paper

Glossary

GlossaryCA connectivity association

CAK connectivity association key

CE router customer edge router

CPE customer premise equipment

DCI data center interconnect

EAPoL Extensible Authentication Protocol over LAN

EP-LAN Ethernet private local-area network

EPL Ethernet private line

EVC Ethernet virtual circuit

EVP-LAN Ethernet virtual private local-area network

EVPL Ethernet virtual private line

GE gigabit Ethernet

GRE generic routing encapsulation

IP Internet protocol

IPsec Internet protocol security

L2TP Layer 2 tunneling protocol

LLDP link layer discovery protocol

MACsec media access control security

MAN metropolitan-area network

MEF Metro Ethernet Forum

MetroE Metro Ethernet

MKA MACsec key agreement

MPLS multiprotocol layer switching

MTU maximum transmission unit

NIST National Institute of Standards and Technology

P router provider router, sometimes referred to as P

P2MP point to multipoint

PE router provider edge router, sometimes referred to as PE

PoP point of presence for a service provider

QoS quality of service

SAK security association key

SA security association

Page 47: WAN MACsec Deployment White Paper - August 2016

page 45Cisco White Paper

Glossary

SC secure channel

SGT security group tag

UDP user datagram protocol

VRF virtual routing and forwarding

WAN wide-area network

Page 48: WAN MACsec Deployment White Paper - August 2016

Americas HeadquartersCisco Systems, Inc.San Jose, CA

Asia Pacific HeadquartersCisco Systems (USA) Pte. Ltd.Singapore

Europe HeadquartersCisco Systems International BV Amsterdam,The Netherlands

Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.

ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, “DESIGNS”) IN THIS MANUAL ARE PRESENTED “AS IS,” WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO.

Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.

© 2016 Cisco Systems, Inc. All rights reserved.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Cisco White Paper

Please use the feedback form to send comments and suggestions about this guide.

B-00940C-1 09/16