INTRODUCTION

The microprocessor architecture transition frommulticore to many-core will increase chip-to-chipinput/output (I/O) bandwidth demands at pro-cessor/memory interfaces and in multiprocessorsystems. Near-term projections, shown in Fig. 1,estimate that CPU-to-memory interconnects willrequire 100 Gbytes/s bandwidth in 2012–2013.Future many-core architectures will requirebandwidths from 200 Gbytes/s to 1.0 Tbyte/s andbegin the era of tera-scale computing.

To meet these bandwidth demands, tradition-

al chip-to-chip electrical interconnect techniqueswill require increased transceiver circuit com-plexity and costlier materials. However, due toelectrical channel loss, increasing I/O bandwidthin electrical links eventually comes at the cost ofreducing interconnect link length, reducing sig-nal integrity, or increasing power consumption.

In contrast, optical interconnects have negli-gible frequency-dependent loss and low crosstalk.Performance is independent of link length (forlengths of interest in chip-to-chip I/O), and littleor no equalization is required. This motivateschip-to-chip I/O architects to consider opticalI/O as a means of scaling data rates in a power-efficient manner.

ELECTRICAL LINK ISSUESFigure 2 shows the components of a typical high-speed electrical link, including the transmitter,receiver, timing system, and channel. A phase-locked loop (PLL) frequency synthesizer gener-ates the transmit serialization clocks, and thereceiver timing system provides the serial datasampling clocks. The design complexity of thetransmitter and receiver increases to includeadditional equalization circuitry as data ratesscale above electrical channel bandwidths.

Electrical channel frequency characteristicsare dependent on channel length. An inset inFig. 2 shows the channel response for three typi-cal electrical channels, a 17-in server backplanechannel with two connectors, a 7-in desktopchannel with no connectors, and an 8-in high-performance cable channel. The frequency-dependent loss exponentially increases withchannel length, as illustrated by the loss differ-ence between a 17-in backplane channel and a 7-in desktop channel. Attenuation and dispersionin these low-pass channels introduces intersym-bol interference (ISI) in high-speed data pat-terns. Equalization can cancel ISI and open thereceived data eye, but requires additional circuitcomplexity, which increases I/O power and area.Equalization is typically implemented with a pro-gressive combination of transmitter (Tx) feed-forward equalization (FFE), receiver (Rx)continuous-time linear equalizers (CTLEs), anddecision feedback equalization (DFE).

IEEE Communications Magazine • October 2010184 0163-6804/10/$25.00 © 2010 IEEE

S. Palermo was with IntelCorp. and is now withTexas A&M University.

ABSTRACT

Future high-performance computing systemswill require optical I/O to achieve their aggressivebandwidth requirements of multiple terabytes persecond with energy efficiency better than 1 pJ/b.Near-term optical I/O solutions will integrateoptical and electrical components in the package,but longer-term solutions will integrate photonicelements directly into the CMOS chip to furtherimprove bandwidth and energy efficiency. Thepresented near-term optical I/O uses a customizedpackage to assemble CMOS integrated transceiv-er circuits, discrete VCSEL/detector arrays, andpolymer waveguides. Circuit simulations predictthis architecture will achieve energy efficiencybetter than 1 pJ/b at the 16 nm CMOS technolo-gy node. Monolithic photonic CMOS processtechnology enables higher bandwidth andimproved energy efficiency for chip-to-chip opti-cal I/O through integration of electro-opticalpolymer based modulators, silicon nitride waveg-uides, and polycrystalline germanium (Ge) detec-tors into a CMOS logic process. Experimentalresults for the photonic CMOS ring resonator(RR) modulators and Ge detectors demonstrateperformance at up to 40 Gb/s and analysis pre-dicts that photonic CMOS will eventually enableenergy efficiency of 0.3 pJ/b with 16 nm CMOS.Optical interconnect technologies with multilanecommunication or wavelength-division multiplex-ing will further increase bandwidth to provide themultiple-terabyte-per-second optical interconnectsolution that enables scaling of high-performancecomputing into and beyond the tera-scale era.

INTEGRATED CIRCUITS FOR COMMUNICATIONS

Ian A. Young, Edris M. Mohammed, Jason T. S. Liao, and Alexandra M. Kern, Intel Corporation

Samuel Palermo, Texas A&M University

Bruce A. Block, Miriam R. Reshotko, and Peter L. D. Chang, Intel Corporation

Optical Technology for Energy EfficientI/O in High Performance Computing

YOUNG LAYOUT 9/21/10 11:47 AM Page 184

IEEE Communications Magazine • October 2010 185

A detailed circuit simulation study shows thatelectrical link bandwidth is limited by either thechannel, at the frequency beyond which loss can-not be overcome with equalization, or the com-plementary metal oxide semiconductor (CMOS)technology, at the frequency beyond which therequired equalization techniques cannot be imple-mented in an energy-efficient manner [1]. In 45nm CMOS with constant 1 Vpp transmit signaling,power efficiency of transmitter and receiver front-end circuits initially improves as the data rateincreases. This trend reverses, and power efficien-cy begins to decline with data rate as more com-plex equalization becomes necessary. Whiletransmit equalization can be implemented withlittle additional energy, a CTLE with sufficientgain-bandwidth product requires significantpower, so the energy efficiency degrades rapidlyonce a CTLE becomes necessary. Ultimately, themaximum data rates in all three channels shownin Fig. 2 are limited by the equalization circuitspeed, as the 45 nm technology cannot supportefficient DFE in the 20 Gb/s range.

Circuit simulation estimates based on a predica-tive 16 nm CMOS technology node reveal that thefaster transistors remove the CMOS technologylimitations and allow efficient implementation ofall equalization circuitry necessary to operate thetwo conventional electrical channels at their funda-mental limits [1]. Channel loss, transmit peakpower constraints, receiver sensitivity, and jittereventually limit the maximum data rate at whichthe desired 10–12 bit error rate (BER) can beachieved in backplane and desktop channels, eventhough significant equalization is used. For theshorter lengths, a high-performance low-loss flexcable channel is still technology-limited because itdoes not require DFE until the data rate exceeds40 Gb/s, at which point a DFE cannot be imple-mented efficiently in the projected 16 nm node.

OPTICAL I/O IMPLEMENTATIONUSING A HYBRID MCM PACKAGE

For the near term, the proposed 12-channel opticaltransceiver architecture allows package integrationof low-cost high-performance optical componentsin existing microprocessor package technology.This hybrid architecture integrates CMOS and dis-crete optical components in a multichip module(MCM) package. In this architecture a multichan-nel optical transceiver chip, an 850 nm 10 Gb/sGaAs vertical-cavity surface-emitting laser(VCSEL) 1 × 12 linear array (or n × 12 array), anda PIN photodiode 1 × 12 linear array (or n × 12array) are flip-chip mounted on a standard micro-processor organic land grid array (OLGA) packagesubstrate. The CMOS drivers and receivers on thetransceiver chip are electrically coupled to theVCSELs and photodiodes with very short trans-mission lines routed on the top surface of thepackage. The VCSEL and photodiode arrays areoptically coupled to on-package integrated polymerwaveguide arrays with metalized 45° mirrors. Thewaveguides couple the optical signals from theVCSELs and photodetectors to standard multiter-minal (MT) fiber optic connectors, which connectto 1 × 12 (or n × 12 array) waveguide or fiberarrays to couple the light off-chip.

THE TRANSCEIVER CHIP ARCHITECTURE

The transceiver architecture for optical I/O sharesmany common features with the typical electricaltransceiver shown in Fig. 2, including the serializ-er, deserializer, clock generation, and clock recov-ery. Transmitter clocks are generated with a PLL,and receiver clocks are either recovered from thedata with a CDR (as in the presented link) or for-warded from the transmitter. Equalization com-plexity is significantly reduced compared toelectrical links, but new circuits are required toperform the electrical-optical-electrical conver-sion. New package technologies are required tointegrate the optical and electrical components ofthe link. The following sections describe the pack-age and circuit innovations that enable a 10 Gb/shybrid optical link in 90 nm CMOS.

PACKAGE ARCHITECTUREThe package architecture allows the integration oflow-cost high-performance optical componentswith standard microprocessor flip-chip OLGApackage technology [2]. Figure 3 shows a photo-graph of the fully assembled optical transceiverpackage and a drawing illustrating the subcompo-nents. The package substrate is a stack of laminat-ed copper layers separated by a dielectric. Atrench is fabricated in the substrate to accommo-date the multimode polymer waveguides, whichhave square apertures with a total height of 100μm, core dimension of 35 μm × 35 μm, and pitchof 250 μm. The 12-channel polymer waveguidearray is 10 mm long and 3 mm wide. A standard12-channel MT optical connector on one end ofthe waveguide array connects to a fiber opticalcable to couple light in and out of the package.An array of 45° mirrors on the other end of thewaveguide bend the optical signal 90° in order tocouple into and out of the VCSEL and photodi-ode arrays, which are flip-chip bonded face downonto the package. The 45° mirror cut is formedeither by microtome or laser ablation and itsreverse side is metalized. The loss from this 45°mirror is 0.3 dB. The high-speed electrical linesused to connect the CMOS chip to the optical

Figure 1. Historical CPU trend — I/O bandwidth vs. year.

2002

1

0.1

Effe

ctiv

e ba

ndw

idth

(G

byte

s/s)

10

100

2000 1998 1996 2004

CPU-CPU, CPU-memory bandwidth roadmap-Driven by 2x performance/2 years Tera-scale

computingHundreds of Gbytes/s

FSB

2 ch RDRAM

QPI

FSB

64 bit SDRAM 64 bit EDO

128 bit DDR

128 bit DDR2

192 bit DDR2/3

Core

$

Router

Core $

Router

Memory

Router

Core

$

Router

Core $

Router PCI-E or FSB

Core

$

Router

Fixed function unit

Router

Core

$

Router Core $

Router

Multi-core CPU to bulk memory > 100’s GB/s

Trend puts memory BW target at >200 GB/s in 2015.Tera-scale computing BW driven by future multi and many-core.

2010 2012 2014 2006 Year

2008

YOUNG LAYOUT 9/21/10 11:47 AM Page 185

IEEE Communications Magazine • October 2010186

components are routed as controlled impedance(50Ω single-ended or 90Ω differential) microstriptraces on the top surface of the substrate wherethey have the best high-frequency characteristics.The optical signal is transmitted by an oxide-con-fined 850-nm 10 Gb/s 1 × 12 VCSEL array withpeak optical output power greater than 3 mW (~5dBm) and received by a 10 Gb/s, 1 × 12 GaAs PINdetector array with a diameter of 75 μm, a capaci-tance of 330 fF, a 3 dB bandwidth of 8 GHz, anda responsivity of 0.6 A/W. The total optical lossbudget for the end-to-end link includes VCSELand photodiode coupling loss through the 45° mir-rors, propagation loss through the waveguide, MTconnector loss, and Fresnel losses at the interfacesin the connectors. The total optical loss budgetcalculated for the complete link is 10 dB [2].Improvements in optical coupling for this hybridpackage architecture are in development to reducethe optical loss budget to as low as 6.8 dB.

CIRCUITS: VCSEL DRIVER ANDTRANSIMPEDANCE AMPLIFIER

Current VCSEL technology is rated for 10 Gb/s,beyond which the VCSELs are bandwidth-limit-

ed with a slow transient tail due to intrinsic andextrinsic parasitic effects such as carrier diffusionand device parasitic capacitance. Pre-emphasiscan compensate for these effects and increasethe achievable data rate. The VCSEL driverdescribed in [1] directly generates dual-edge pre-emphasis with sub-bit-period pre-emphasis wave-form timing precision. The pre-emphasizedcurrent waveform is generated by summing themain modulation current with a delayed andweighted peaking current in order to producepre-emphasis pulses at each data transition. Typ-ical average currents provided to the VCSELsrange from 6 to 10 mA, which corresponds to anaverage optical power of 1.5 to 2 mW. TheVCSEL driver is output terminated and connect-ed to the VCSEL through a 50Ω microstriptransmission line routed on the top surface ofthe package. As the VCSEL technology developsfor higher modulation speed (using quantumdots rather than quantum well technology), high-data-rate VCSELs at 20 Gb/s and higher will stillbenefit from these pre-emphasis techniques tofurther extend data rates.

The transimpedance amplifier (TIA) uses thedifferential symmetric-feedback topology [1],

Figure 2. High-speed electrical link block diagram showing serializer, TX PLL, TX finite impulse response(FIR) equalizer, RX continuous-time linear equalizer (CTLE), RX decision feedback equalizer (DFE),CDR, and deserialize.

TX Clk generation

(PLL)

Seri

aliz

er

Des

eria

lizer

8“ FLEX TX FIR equalization

RX CTLE + DFE equalization

Channel

ΣDTX[N:0] DRX[N:0]

RX Clk recovery

(CDR/Fwd Clk)

|S21|

5GHz 0GHz

-60dB

-80dB

-40dB

-20dB

0dB

10GHz 15GHz

: 8” Flex : 7” desktop : 17” refined backplane w/2 connectors

Figure 3. a) A fully assembled optical transceiver unit; b) a schematic side view of the same unit, showingthe optical coupling scheme of VCSELs/photo-detectors to waveguides through a 45° mirror.

Board

Input optical

connector

Output optical connector

VCSEL

CMOSTxRx

WaveguidesPhotodiodes

VCSELS

DecouplingcapacitorsOpticalconnector

(a) (b)

PD CMOS chip

Optical connector

Optical waveguide

50 Ohm TXLine

Package substrate and electrical IO

Current VCSEL

technology is rated

for 10 Gb/s, beyond

which the VCSELs

are bandwidth-limit-

ed with a slow

transient tail due to

intrinsic and extrinsic

parasitic effects such

as carrier diffusion

and device parasitic

capacitance.

Pre-emphasis can

compensate for

these effects and

increase the achiev-

able data rate.

YOUNG LAYOUT 9/21/10 11:47 AM Page 186

IEEE Communications Magazine • October 2010 187

which converts the single-ended input current toa differential output voltage to help mitigate sup-ply noise at subsequent gain stages and providesa data rate above 12.5 Gb/s when the total inputparasitic capacitance Cp is less than 250 fF. TheTIA receives a single-ended photocurrent of 200μA from the photodiode and generates a differ-ential 2 × 50 mVpp output that is fed to a follow-ing limiting amplifier (LA), which converts it to aCML level output. The LA consists of a cascadeof CML buffers. In the packaged transceiver thecombined capacitance of the photodiode, metalpad, bump, and ESD could be as high as 500 fF.This capacitance limits the maximum data ratethat can be measured for the packaged receiverchannel. The same TIA tested electrically withwafer probing had an open electrical eye diagramat 18 Gb/s for an input capacitance of 90 fF. Thisindicates there is a strong dependence of band-width on the input parasitic capacitance.

EXPERIMENTAL RESULTS10 Gb/s optical measurement results are shownin Fig. 4 for a fully assembled transmitter andreceiver. For the transmitter measurement, exter-nal differential electrical pseudorandom data was

sourced into the chip to drive the CMOS pre-emphasis VCSEL driver, and the VCSELs werebiased with an average current of 7 mA. Themeasured transmitter optical eye opening was 70ps. The receiver demonstrated an open electricaleye for optical 10 Gb/s input data. The electricalreceived signal eye opening was 60 ps with apeak-to-peak jitter of 30 ps. The individual trans-mitter and TIA receiver circuits are capable ofoperation at up to 18 Gb/s [2].

PHOTONIC CMOS OPTICAL I/OARCHITECTURE

In the longer term, monolithic integration of pho-tonic elements in a CMOS process will enable sig-nificant improvements in I/O performance, energyefficiency, and cost. The proposed monolithicphotonic CMOS process, illustrated in Fig. 5,integrates modulators, waveguides, and detectorson top of the metal interconnect layers in the farback-end of a standard CMOS process. Lightfrom a continuous-wave (CW) source is coupledonto the die and modulated using integratedwaveguide-based modulators driven by on-chipcircuits, such that the electrical signals do not

Figure 4. Optical eye diagrams for 10 Gb/s tested with fully packaged: a) transmitter optical output; b)receiver optical input.

10 Gb/s: Tx optical eye

(a)

70ps

10 Gb/s: Rx optical eye

(b)

60ps

1 level 1 level

0 level 0 level

Figure 5. Photonics optical interconnect architecture.

Photodetectors

Couplers Optical data in

Optical data out

CMOS receivers

CMOS drivers

Modulators

Si3N4 waveguide

Laser

CW

In a photonic CMOS

process for integrat-

ed optical links, the

additional process

steps required for

photonics must not

degrade or interfere

with the front-end

CMOS transistor per-

formance. Further-

more, the process

must allow fabrica-

tion of all required

optical components

on the same die.

YOUNG LAYOUT 9/21/10 11:47 AM Page 187

IEEE Communications Magazine • October 2010188

leave the die. The modulated light is coupled offthe die through a fiber or waveguide to a receiv-ing chip, where it is coupled through an integrat-ed waveguide into a compact photodetector. Thephotodetector output current is converted to afull-swing electrical signal by a TIA and an LA.

Monolithic integration of photonics onto themicroprocessor will reduce the power and the costof I/O. Integration reduces the capacitive load onthe driver and receiver circuits and leads to higherbandwidth and lower power. Parasitic capacitanceis reduced because integration of the circuits andoptical devices on the same die removes the bump,package, and ESD capacitance from the signalpath. The intrinsic device capacitance of integrat-ed optical components is smaller than the capaci-tance of discrete alternatives. Static powerconsumption is reduced because small integratedoptical devices do not require termination, in com-parison to larger discrete alternatives such asMach-Zender interferometers which require 50Ohm termination for high-speed operation. Costis reduced by decreasing the required number ofdiscrete optical components.

In a photonic CMOS process for integratedoptical links, the additional process steps requiredfor photonics must not degrade or interfere withthe front-end CMOS transistor performance.Furthermore, the process must allow fabricationof all required optical components on the samedie. The optical components in previouslydemonstrated integrated optical links were fabri-cated in the front-end of a semiconductor-on-insulator (SOI) CMOS process [3], whichconstrains the transistor processing. The present-ed experimental photonic process is based on asilicon nitride single-mode waveguide with silicondioxide cladding and provides waveguides, elec-tro-optic (EO) polymer ring resonator (RR)modulators [2, 4], and waveguide-embeddedmetal-semiconductor-metal (MSM) detectorsfabricated from polycrystalline germanium on the

bulk CMOS (not SOI) process back-end compat-ible silicon-dioxide dielectric [4, 5].

FABRICATIONThe photonic elements are added to the CMOSprocess in the metal interconnect fabrication back-end section of the process after all the high tem-perature front-end processing of transistors iscompleted. The waveguides are formed with a 450nm silicon nitride layer deposited by plasma-enhanced chemical vapor deposition (PECVD) onthe SiO2 interlayer dielectric (ILD) and patternedwith photolithography and plasma dry etch. Thisshared waveguide layer is used to build all thewaveguides, RRs, and coupling waveguides for theactive electro-optic devices. After patterning thewaveguides, silicon dioxide cladding is deposited,and three subsequent lithography steps define thedetector regions, the electrodes for all activedevices, and modulator regions. The photodetectorregions are filled with polycrystalline germanium ina damascene process, the detector electrodes areformed in a standard copper damascene process,and then the modulators are formed by depositingEO polymer cladding over the ring resonators inthe regions defined for the modulators. The addi-tional cost to add photonic devices to the CMOSprocess is low, since only four additional pho-tolithography steps are required.

Figure 6 shows both a top view and an SEMcross-section of the modulator, waveguide, anddetector constituting a complete optical link. A sin-gle patterned silicon nitride layer forms all of thewaveguides in the active and passive components.Similarly, one metal layer forms all the electrodesfor both the modulator and photodetector. Fur-thermore, this optical layer is compatible with stan-dard microprocessor CMOS as it is created on anamorphous ILD and can therefore be fabricated inthe back-end metal interconnect section of theCMOS process. In order to stay within the thermalbudget for standard back-end processing, all stepsin the process flow must occur below ~450°C.

EXPERIMENTAL RESULTSWaveguide — The waveguide is the foundationfor the proposed photonic CMOS technology [6].The waveguide is processed as a 450 nm PECVDsilicon nitride film deposited on a 2 μm silicondioxide undercladding layer at 400°C. The waveg-uide is patterned using conventional 248 nmlithography and plasma etching. Loss measure-ments at 1310 nm using the cut-back method showthat the silicon nitride waveguide loss is ~1 dB/cmfor waveguides with a width of 0.5 μm. This loss issufficiently low for on-die applications where thetotal waveguide length is on the order of 1 cm.

Modulator — The electro-optic cladding RRmodulator and photodetector share the highindex contrast waveguide fabrication process. Themodulator design is based on a high-performancering resonator built with a silicon nitride waveg-uide and ring. Copper damascene electrodes arefabricated around the ring, and the top claddingis removed and replaced with the EO polymer.This work uses a proprietary chromophor-dopedEO polymer [6]. The modulator design is opti-mized for a quality factor (Q) between 5000 and10,000: high enough that a small resonance shift

Figure 6. a) Schematic of top view of full on-die optical link showing buswaveguide connecting modulator to photo detector; b) cross section SEMimage (along the dotted line in 6a) showing optical components in one pieceof silicon.

Ge

Cu

3um

Silicon nitride

SiO2

SiO2

Detector Waveguide Modulator

Lighta.

b.

Electrical out Electrical in

YOUNG LAYOUT 9/21/10 11:47 AM Page 188

IEEE Communications Magazine • October 2010 189

results in a large modulation depth, but not sohigh that the modulator is unable to switch athigh data rates. The EO polymer is poled beforewafer processing is completed using an electricfield of 100 V/cm around the glass temperature of143°C. The electrodes have a 4.5 μm gap centeredaround the waveguide ring, and the ring has aradius of 28 μm. An SEM image of the modulatoris shown on the right of Fig. 6b. The resonancespectrum of a typical modulator under +20 Vand –20 V bias is shown in Fig. 7a. The resonanceshift calculated with a linear fit to the resonancefrequencies measured at +20 V and –20 V bias is5 pm/V. The measured Q was ~7000, and theresonance depth was ~11 dB. The highest mea-sured modulation depth for a 10 GHz clock inputwith a 6 V swing was 8 dB. A 20 Gb/s pseudo-random binary sequence (PRBS) eye diagram fora typical device was measured [4] (Fig. 7b).

Photodetector — Unlike a PIN detector, thelateral MSM detector requires only one lithogra-phy step to form the contacts. An evanescentlycoupled waveguide, shown on the left of Fig. 6b,efficiently couples the light into the absorbingactive material of the photodetector. The poly-crystalline germanium in the detector was deposit-ed by CVD processing at 600°C. Fabrication of aphotodiode from polycrystalline germaniumdeposited on ILD is an important step towardcompatibility with a standard back-end (BE)CMOS process. Measurements at a higher fre-quency showed that 40 Gb/s operation is withinreach. To improve the noise performance, band-gap engineering can be used to create a Schottkybarrier at the metal/germanium contact in orderto reduce the dark current further. Anotherimportant step toward BE compatibility is lower-ing the process temperature. Devices were fabri-cated using PVD Ge at 350˚C, and the bestmeasured devices had a dark current of 77 μAwith open PRBS eyes at both 20 and 40 Gb/s [4].

OPTICAL LINK MODELING ANDCOMPARISONS

The optical I/O link power efficiency is a strongfunction of the received optical power, which isdetermined by the transmit power and the linkoptical loss budget. A feasible best case value for

the hybrid optical link budget is –6.8 dB withsome packaging improvements. This is dominat-ed by coupling losses from the VCSEL and pho-todetector to the multimode fiber (MMF) andthe finite extinction ratio penalty. The hybridoptical I/O link budget is calculated using thefollowing assumptions:Average TX power 3.0 dBmVCSEL to MMF coupling –1.1 dBMMF to photodetector coupling –1.1 dBExtinction ratio (7.3 dB) penalty –1.6 dBMargin –3.0 dBLink budget –6.8 dBRequired RX sensitivity –3.8 dBm

The integrated optical link budget is nearly 9dB worse than the hybrid optical link budget dueto the coupling loss between the off-chip single-mode fiber and the on-chip single-mode wave-guide, and the extra coupling loss from theoff-chip CW laser. However, the integrated pho-todetector’s ultra-low capacitance allows theintegrated optical receiver to achieve approxi-mately 13 dB of sensitivity improvement at thesame bandwidth, which results in significant sys-tem power savings. The integrated optical I/Olink budget is calculated using the followingassumptions:Average VCSEL TX power 3.0 dBmSource laser to SMF coupling –2.0 dBSMF to modulator coupling –2.0 dBModulator loss –2.0 dBModulator to SMF coupling –2.0 dBSMF to photodetector coupling –3.0 dBExtinction ratio (8.0 dB) penalty –1.4 dBMargin –3.0 dBLink budget –15.4 dBRequired RX sensitivity –12.4 dB

Circuit simulation-based power efficiency esti-mates of both transmit and receive front-end cir-cuits for these two optical I/O architectures wasperformed for CMOS technologies starting froma 45 nm node and ending with a predicative 16nm CMOS node [1]. A current-mode VCSELdriver and a simple CMOS inverter-based volt-age-mode modulator driver are modeled for thehybrid and integrated optical systems, respective-ly. In both systems a TIA is followed by simpledifferential-pair LA stages to realize the opticalreceiver. The models are constructed with thecircuits optimized to provide the minimum band-width necessary for a particular data rate, and

Figure 7. a) Resonance spectra obtained with –20 V (♦) and 20 V (×) bias on the EO modulator; b) 20Gb/s PRBS eye diagram of EO polymer modulator.

Wavelength (nm)

(a) (b)

~8 dB modulation

No voltage applied

Voltage applied

Δλ/ΔV = 5 pm/V Laser λ

1314 1313.8

-8

-14

Nor

mal

ized

tra

nsm

issi

on [d

B]

-10 -12

-6 -4 -2 0

1314.2 1314.4 1314.6 1314.8

6Vpp 20 Gb/s PRBS

Measurements at a

higher frequency

showed that 40 Gb/s

operation is within

reach. To improve

the noise

performance, band-

gap engineering can

be used to create a

Schottky barrier at

the metal/

germanium contact

in order to reduce

the dark current

further.

YOUNG LAYOUT 9/21/10 11:47 AM Page 189

IEEE Communications Magazine • October 2010190

thus approximate a power optimal solution. Thehybrid optical link power efficiency initiallyimproves as the data rate increases due to theassumed-constant 3 dBm optical power from the850 nm VCSEL. Power efficiency degrades fromthe optimum at higher data rates due to the opti-cal RX amplifier gain-bandwidth requirements.As technology scales, this optimum occurs at ahigher data rate due to the increased transistorfT. This analysis predicts that hybrid optical datatransmission at less than 1 pJ/b will be realized inthe future. Assuming a 1310 nm CW laser sourcewith 3 dBm optical TX power, the integratedoptical link power efficiency displays similarbehavior at a much lower power level due to lowcapacitance of the modulator and photodetectorallowing for very efficient optical drivers andreceivers. Ultra-low receiver input capacitanceenables a TIA-based receiver without any LAstages to provide sufficient sensitivity at datarates exceeding 30 Gb/s. The data rate at whichextra LA stages become necessary scales with theimproved CMOS technology fT. These projec-tions indicate that photonic CMOS could enableintegrated optical interconnect to reach 0.3 pJ/b.

The power-performance analysis of the hybridoptical link was compared with electrical link sys-tems that employ all three electrical channels dis-cussed early in the electrical link analysis. Thecomparison reveals that the hybrid optical architec-ture is equal to or better in power efficiency thanboth the electrical backplane and desktop channelsat data rates near where RX equalization becomesnecessary. This data rate is dependent on the chan-nel loss characteristics and is 13 Gb/s and 19 Gb/sfor the 17 in backplane and 7 in desktop channels,respectively. While the hybrid optical link cannotoutperform the high-performance electrical cablechannel at the 45 nm node, the increased gain-bandwidth offered by the 16 nm node allows thehybrid optical link to become comparable near 40

Gb/s. Note that this assumes the availability of 40Gb/s-class VCSELs, which are currently emergingfrom research [7]. The reduced parasitics offeredby the integrated photonics with CMOS opticalarchitecture allows it to achieve superior powerefficiency over the majority of data rates comparedto the three electrical channels as well as thehybrid optical architecture. This assumes furtherimprovements in modulator EO polymer perfor-mance to enable sufficient optical modulationdepth at voltage modulation levels compatible withCMOS inverter-based drivers [4, 6].

FUTURE DIRECTIONSAs CMOS scaling continues in the future, largernumbers of CPU cores will be integrated on themicroprocessor chip, and it will become neces-sary to provide interconnect scaling to higherbandwidth between cores on chip, and betweenthese cores and the off-chip DRAM. Wave-length-division multiplexed (WDM) links trans-mit multiple wavelengths through the samewaveguide in order to increase the aggregateoptical data transmission. A photonic CMOSarchitecture for optical WDM of signals mono-lithically integrated on chip is shown in Fig. 8.The RR modulator selectively modulates a sin-gle wavelength from a multiwavelength source,and eliminates the need for separate opticaldemultiplexers and multiplexers. At the receiver,passive RR optical filters can demultiplex theoptical data by selecting a single unique wave-length for detection at each photodetector. Sincethe photonic CMOS RR modulators have such anarrow tuning range (Fig. 7), the WDM wave-lengths can be spaced at less than 1 nm (100GHz in optical frequency with a reference of 230THz). Thus, the RR technology provides themeans for bandwidth to scale by adding morewavelengths to each waveguide channel [1].

Figure 8. Photonic CMOS enabled wavelength-division multiplexing architecture.

10011010110 00110101011 00101111001 10101010011

10010010001 00011101010 10110101011 10011011001

Optical layer

Optical layer

Chip (CPU, memory, graphics, etc.)

Chip

Ge photo-detector

SiN waveguide (single-mode)

Monolithic integration R-R modulators

Optical layer

Laser

λ1 λ2 λ3 λ4(broadband)

λ1 λ2 λ3 λ4

Optical I/O

Laser

Chip

The comparison

reveals that the

hybrid optical

architecture is equal

to or better in power

efficiency than both

the electrical

backplane channel

and the desktop

channel at data rates

near where

RX equalization

becomes necessary.

YOUNG LAYOUT 9/21/10 11:47 AM Page 190

IEEE Communications Magazine • October 2010 191

SUMMARY

This work provides a comparison of electricalI/O to optical I/O for chip-to-chip interconnect.While electrical interconnect will continue to usemore sophisticated equalization techniques toovercome the loss of the interconnect channel,the high data rate and long interconnect lengthsrequired by future many-core processors willrequire the introduction of optical interconnect.Optical interconnect for CPUs will first be intro-duced with optical package-to-package I/O usinghybrid MCM single-package technology. In thelong term, monolithic integration of optical com-ponents will provide terabyte-per-second inter-connect data rates with the required energyefficiency at less than 1 pJ/bit.

REFERENCES[1] I. A. Young et al., “Optical I/O Technology for Tera-

Scale Computing,” IEEE JSSC, Jan. 2010, pp. 235–48.[2] E. Mohammed et al., “Optical Hybrid Package with an

8-Channel 18GT/s CMOS Transceiver for Chip-to-ChipOptical Interconnect,” Proc. SPIE, vol. 6899, Feb. 2008.

[3] B. Analui et al., “A Fully Integrated 20-Gb/s Optoelec-tronic Transceiver Implemented in a Standard 0.13mCMOS SOI Technology,” IEEE JSSC, Dec. 2006, pp.2945–55.

[4] I. A. Young et al., “Integration of Nano-PhotonicDevices for CMOS Chip-to-Chip Optical I/O,” CLEO/QELSConf. Digest Tech. Papers, May 16–21, 2010.

[5] M. R. Reshotko et al., “Waveguide Coupled Ge-on-Oxide Photodetectors for Integrated Optical Links,”IEEE/LEOS Int’l. Conf. Group IV Photonics, Sept. 17–19,2008, pp. 182–84.

[6] B. A. Block et al., “Electro-Optic Polymer Cladding RingResonator Modulators,” Optics Express, vol. 16, no. 22,Oct. 2008, pp. 18326–33.

[7] T. Anan et al., “High-Speed 1.1-mm-Range InGaAsVCSELs,” Optical Fiber Communication/National FiberOptic Engineers Conf., Feb. 24–28, 2008, pp. 1–3.

BIOGRAPHIESIAN. A. YOUNG [M’78, SM’96, F‘99] (ian.young@intel.com) is asenior fellow and director of Advanced Circuits and Technolo-gy Integration in the Technology and Manufacturing Groupat Intel Corporation. He does research and development ofmixed-signal circuits for microprocessor and SOC productsalong with process technology development. He joined Intelin 1983. Starting with the development of circuits for a 1 MbDRAM, he then led the design of three generations of SRAMproducts and manufacturing test vehicles, and developed theoriginal PLL-based clocking circuit in a microprocessor whileworking on the 50 MHz Intel486 processor design. He subse-quently developed the core PLL clocking circuit buildingblocks used in each generation of Intel microprocessorsthrough the 0.13 μm 3.2 GHz Pentium 4. He is currentlydirecting the research and development of analog mixed-sig-nal and RF circuits for high-speed serial I/O in advanced logicprocess, RF wireless circuits in advanced SOC process, andresearching chip-to-chip optical I/O technology. He receivedhis Bachelor’s and Master’s degrees in rlectrical rngineeringfrom the University of Melbourne, Australia, in 1972 and1975. He received his Ph.D. in electrical engineering from theUniversity of California, Berkeley in 1978. He was a memberof the Symposium on VLSI Circuits Technical Program Com-mittee from 1991 to 1996, serving as Program CommitteeChairman in 1995–1996 and Symposium Chairman in1997–1998. He was a member of the ISSCC Technical Pro-gram Committee from 1992 to 2005, serving as Digital Sub-committee Chair from 1997 to 2003, Technical ProgramCommittee Vice-Chair in 2004, and Chair in 2005.

EDRIS MOHAMMED is a research engineer at Intel Corporationresponsible for system design, integration, and characteriza-tion of hybrid optical interconnect at Intel’s componentsresearch division. He joined Intel Corporation in 2001. Beforejoining Intel, he was a member of technical staff at LucentTechnologies where he worked as a test and reliability engi-neer for 980 nm pump laser diodes. His research interestsinclude high-speed optoelectronic devices, package integra-

tion, and passive optical and electrical devices. He received aB.Sc. in physics from Addis Ababa University in 1987, anM.Sc. in physics from Florida A&M University in 1995, and anM.Sc. in electrical engineering (optoelectronics) and a Ph.D.in applied physics from Georgia Institute of Technology in2000. Since 2005 he has been serving as an associate editorof optical interconnects for SPIE Optical Engineering Journal.

JASON T. S. LIAO [M’09] received his M.S. degree in commu-nication engineering from National Chiao Tung University,Taiwan, in 1996 and his Ph.D. degree in electrical engineer-ing from the University of California, San Diego in 2004. In2005 he joined Intel Corporation, Hillsboro, Oregon, as asenior electrical engineer. His research focuses on high-speed optical I/O and receiver clocking techniques for high-speed electrical I/O links.

ALEXANDRA M. KERN [S’01, M‘07] received A.B. and B.E.degrees from Dartmouth College in 2002, and S.M. andPh.D. degrees in electrical engineering and computer sci-ence from the Massachusetts Institute of Technology in2004 and 2007. In June 2007 she joined the Logic Technol-ogy Development group at Intel Corporation, where she isinvolved in the development of high-speed electrical seriallinks in advanced CMOS technology. Her research interestsinclude electrical and optical interconnect for on-chip andchip-to-chip applications, and analog/mixed-signal circuitdesign techniques for deep-submicron CMOS.

SAMUEL PALERMO [S’98, M’07] received B.S. and M.S. degreesin electrical engineering from Texas A&M University, Col-lege Station in 1997 and 1999, respectively, and his Ph.D.degree in electrical engineering from Stanford University,California, in 2007. From 1999 to 2000, he was with TexasInstruments, Dallas, where he worked on the design ofmixed-signal integrated circuits for high-speed serial datacommunication. From 2006 to 2008 he was with Intel Cor-poration, where he worked on high-speed optical and elec-trical I/O architectures. In 2009 he joined the Electrical andComputer Engineering Department of Texas A&M Universi-ty, where he is currently an assistant professor. His researchinterests include high-speed electrical and optical links,clock recovery systems, and techniques for device variabilitycompensation. He is a member of Eta Kappa Nu.

BRUCE A. BLOCK joined Intel in 1998 after receiving his Ph.D.degree in materials science and engineering from Northwest-ern University in 1997 and a B.S. degree in materials scienceand engineering from Cornell University in 1989. From 1989to 1992 he worked for IBM Corporation, East Fishkill, NewYork, in the Advanced Packaging Laboratory. He has beenworking since 2000 on the process integration and character-ization of CMOS-compatible optical devices in the Compo-nents Research Organization. He has developed many passiveand active devices based on silicon nitride waveguides,including waveguide coupled germanium photodetectors,electro-optic polymer modulators, ring resonator filters, andsurface plasmon polariton-based polarizers and detectors.

MIRIAM R. RESHOTKO [M’07] joined Intel’s ComponentsResearch department in 2001 after completing a Ph.D.degree in physics at the Hebrew University of Jerusalem,Israel. She received her M.Sc. degree in physics from thesame university, and her B.A. in physics from Cornell Uni-versity. Her current research centers on integrated CMOScompatible optoelectronic devices for optical interconnects.Specific areas of focus are development of monolithicallyintegrated high-speed waveguide coupled photodetectorsfor various interconnect applications, and process integra-tion for discrete optical devices, including waveguides,modulators, photodetectors, and optical links.

PETER L. D. CHANG [M’92] received his B.S. degree in physicsfrom National Taiwan University in 1977 and his Ph.D. intheoretical solid state physics from the State University ofNew York at Stony Brook in 1985. From 1985 to 1991 heconducted research in the Physics and ECE Departments ofthe University of California, Santa Barbara. From 1991 to1994 he was with Lockheed Sanders working on MMIC. Hejoined Intel in 1994, developing 0.25 μm flash memorytechnology. Sine 1996 he has been involved in R&D andhigh volume manufacturing of logic and SRAM process. Heis currently with the Components Research of Technology,Manufacturing and Enterprise Services Group. His researchinterests are in high-density memory and optical intercon-nects for high memory bandwidth required for future gen-erations of CPUs.

Optical interconnect

for CPUs will first be

introduced with opti-

cal package-to-pack-

age I/O using hybrid

MCM single-package

technology. In the

long term, monolith-

ic integration of opti-

cal components will

provide TB/s inter-

connect data rates

with the required

energy efficiency at

less than 1 pJ/bit.

YOUNG LAYOUT 9/21/10 11:47 AM Page 191