©OraInternals Riyaj Shamsudeen

MTU Maximum Transfer Unit

By Riyaj Shamsudeen

©OraInternals Riyaj Shamsudeen 2

MTU

  MTU defines Maximum Transmission Unit of a packet. Essentially, limits the size of a packet, default is ~1500 bytes.

  For example, to transfer a buffer of 8K size, 6 packets must be transmitted. These packets can leave and arrive any order.

2

8K buffer

LMS

3456

1

Fragment Switch

2

8K buffer

FG

3456

1

Assembly

©OraInternals Riyaj Shamsudeen 3

UDP vs TCP

TCP

User

Kernel UDP

IP

Server processes Cluster ware

NIC Hardware

Clusterware uses TCP for network heartbeat traffic between the nodes.

Cache fusion traffic can use UDP

©OraInternals Riyaj Shamsudeen 4

Jumbo frames – Why?

  Assembly and Fragmentation of network packets are CPU intensive operations.

  Jumbo frames can be helpful if there is CPU starvation already.

  Since this operation is performed in system calls, CPU is used in Kernel mode.

  With Jumbo frame usage, MTU is increased beyond 8K, typically 9000 bytes.

  Just one packet is needed to transmit 8K buffer eliminating the need for fragment and assembly operations.

©OraInternals Riyaj Shamsudeen 5

Jumbo frames and Interrupts

  Jumbo frames also reduces CPU interrupts.

  Interrupts have higher priority, so runnable processes can be kicked out of CPU to service an interrupt.

  Each incoming network packet will cause a CPU interrupt. With jumbo frames, you can reduce the number of interrupts six fold.

  If you have very high number of interrupts, then you could use jumbo frames to reduce the interrupts.

©OraInternals Riyaj Shamsudeen 6

Packet frames …2

With jumbo frame one block of 8K can be transmitted in a network packet.

Without jumbo frames, one 8K block (8192) transfer will require 6 transmissions of packets sized ~1500

Ethernet Header

IP Header

UDP Header

Data

Ethernet Trailer

14 bytes

20 bytes

8 bytes

1472 bytes or 8954 bytes

4 bytes

©OraInternals Riyaj Shamsudeen 7

Jumbo frames – Caution …3

  Make sure that jumbo frames are setup properly in the Network hardware in all nodes in the cluster, routers/VLAN between the nodes.

  If the routers or switches do not support Jumbo frames, you can run in to complex performance issues.

  In huge organizations, it is hard to convince network administrators to configure Jumbo frames.

  Jumbo frames also will improve the performance slightly, approximately 3%-5%, if there is no CPU starvation. So, if network admins do not support jumbo frames, be wary of it.

©OraInternals Riyaj Shamsudeen 8

Contact info: Email: rshamsud@gmail.com Blog : orainternals.wordpress.com URL : www.orainternals.com

Thank you for attending!