Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots...
Transcript of Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots...
![Page 1: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/1.jpg)
Architecture Tutorial
Alan GoodrumChairman, PCI-X Workgroup
Staff Fellow, Compaq Computer Corporation
May 23, 2000Applied Computing Conference 2000
![Page 2: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/2.jpg)
2
Purpose of this tutorial� This tutorial is
� Introduction to PCI-X– Key features– Key benefits
� Aimed primarily at digital designers� Limited by time
� This tutorial is NOT� Detailed study of every protocol feature� Detailed study of electrical requirements� Detailed study of bridge requirements� Detailed study of new configuration registers
![Page 3: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/3.jpg)
3
Agenda
� Key Features� Card and System Interoperability� Protocol� Software Aspects� Electricals� Performance� Summary
![Page 4: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/4.jpg)
4
Key PCI-X Features
![Page 5: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/5.jpg)
5
I/O Bandwidth vs. Time
1986 1988 1990 1992 1994 1996 1998 2000 2002
Ethernet
1 Gbit/s
InternetBackbone
OC 192
T3
SCSI
10 Gbit/s
EISA
1
10
100
1,000
10,000
Ban
dwid
th (M
B/s
)
ISA
Today 2000
6MHz 16-bit
10MHz 32-bit
33MHz 32/64-bit
66MHz 32/64-bit
66/133MHz 32/64-bit
![Page 6: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/6.jpg)
6
Key PCI-X Features� PCI-X Systems
� 32- or 64-bit, � 3.3 Volt I/O� Trade off speed for slots
1 slot @ 133 MHz; 2 slots @ 100 MHz; 4 slots @ 66 MHz� PCI-X Devices
� 32- or 64-bit� 66, 133MHz� 3.3Volt I/O or Universal
� Bus runs in PCI-X or conventional mode (similar to 33/66 MHz modes in PCI 2.2)
� PCIXCAP mode pin similar to M66EN� Integrates well with emerging switched fabric protocols like
InfiniBand
![Page 7: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/7.jpg)
7
� Attribute phase for each transaction� Byte count� Initiator ID� Handling instructions� Tag
� More intelligent use of wait states� Only target initial wait states supported
� Standard block size movements (I/O Cache line)� Fixed transaction disconnection points on 128 byte
boundaries� Split Transactions replace Delayed Transactions� Makes multi-threaded operation practical� Electrical design for PCI-X easier than conv 66 MHz
Key PCI-X Features - cont’d
![Page 8: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/8.jpg)
8
� Relaxed Transaction Ordering� Optional function for removing unnecessary blocking
cases� Support for non-cache-coherent transaction� New configuration registers accessed via Capabilities
data structure� No S/W initialization required
� Default values always functional
� Improved error handling � Allows cards an increased range of options for handling
data parity errors
Key PCI-X Features - cont’d
![Page 9: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/9.jpg)
9
Building on the Foundationof PCI 2.1 and 2.2� Compatibility mode for 33MHz PCI 2.2 (3.3v)
� PCI-X systems can accept standard PCI cards� PCI-X cards can work in current PCI systems
� Requires no Device Driver or OS modification� New features designed for easy migration� Similarity to 2.1/2.2 protects development infrastructure� Required support for Message-Signaled Interrupts
and PCI Power Management (D0 & D3) � Designed to support PCI Hot-Plug
(new Hot-Plug System Driver)
![Page 10: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/10.jpg)
10
PCI-X System Flexibility --Speed vs. Slot Tradeoff
BusWidth
BusFrequency
BusBandwidth PCI Slots PCI-X Slots
32-bit 33 MHz 133 MB/s N/A
64-bit 66 MHz 533 MB/s
64-bit 100 MHz 800 MB/s N/A
64-bit 133 MHz 1066 MB/s N/A
ConventionalPCI
PCI-X
PCI-X doubles the numberof slots per bus segment
at 66 MHz
PCI-X at 100 MHz provides enterprise class
I/O bandwidth
PCI-X at 133 MHz is the first interconnect to
exceed 1Gbyte/s
The most commonimplementation of PCI today
33MHz per bus segment
![Page 11: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/11.jpg)
11
PCI-X System Flexibility --Hierarchical Structure
64-bit 66-MHz533-MB/s
64-bit 133-MHz1066-MB/s
64-bit 100-MHz800-MB/s
133-MHz
Chipset
PCI-Xto
PCI-XBridge
PCI-Xto
PCI-XBridge
PCI-Xto
PCI-XBridge
PCI-X bus
PCIPCI--X bridges allow PCIX bridges allow PCI--X to link up to 256 separate bus segments. X to link up to 256 separate bus segments.
Smarter protocol makes high-performance PCI-X bridges practical
Bus segment #1
Bus segment #2
Bus segment #3
Bus segment #4
![Page 12: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/12.jpg)
12
Card and System Interoperability
![Page 13: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/13.jpg)
13
The Million-Dollar Question
� What is PCI’s I/O voltage migration plan?
� The correct answer is….
a. All 3.3V PCI cards must be 5V tolerant.b. Universal slots accept both 5V and 3.3 V keyed cards.c. PCI slots have been mostly 3.3V I/O for years.d. Universal cards plug into both 5V and 3.3V keyed slots.
![Page 14: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/14.jpg)
14
Hardware Compatibility --PCI 5v to 3.3v I/O Migration Story
Universal PCI adapter cards are keyed for both 5v and 3.3v slots
5v Keyed5v Keyed
3.3v Keyed3.3v Keyed
6464--bit Extensionbit Extension
5v Keyed Slot & 5v Keyed Slot & Adapter CardAdapter Card
6464--bit Extensionbit Extension
3.3v Keyed Slot & 3.3v Keyed Slot & Adapter CardAdapter Card
![Page 15: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/15.jpg)
15
Adapter Card Selection
� Cost sensitive 32-bit Cards� PCI-X cards will work in current PCI
systems just like 66MHz conventional cards
� Conventional Speeds & Bandwidth� 33MHz 133MB/sec� 66MHz 266MB/sec (optional)
� PCI-X Speeds & Bandwidth� 66MHz 266MB/sec� 133MHz 533MB/sec (optional)
� 3.3v or Universal
� High performance 64-bit cards� PCI-X cards will work in current PCI
systems just like 66MHz conventional cards
� Conventional Speeds & Bandwidth� 33MHz 266MB/sec� 66MHz 533MB/sec (optional)
� PCI-X Speeds & Bandwidth� 66MHz 533MB/sec� 133MHz 1066MB/sec (optional)
� 3.3v or Universal
6464--bit Extensionbit Extension
![Page 16: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/16.jpg)
16
PCI-X System Configuration
� PCI-X slots accept both PCI & PCI-X adapters
� For best performance group cards by speed and type (PCI-X or PCI) per bus segment:
� Conventional 33MHz cards � Conventional 66MHz card� PCI-X 66 MHz cards� PCI-X 133 MHz card
MemoryMemoryControllerController
PCIPCI--X X HostHost
BridgesBridges
CPUCPU CPUCPU CPUCPU CPUCPUMemoryMemory
![Page 17: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/17.jpg)
17
Interoperability Matrix
Note:Note:1. PCI1. PCI--X system and devices must support conventional 33MHz timing, andX system and devices must support conventional 33MHz timing, and may optionally may optionally support conventional 66MHz timing.support conventional 66MHz timing.
conventional PCI cards PCI-X cards33 MHz(5V I/O)
33 MHz(3.3V I/O or
Universal
66 MHz(3.3V I/O or
Universal
66 MHz(3.3V I/O or
Univeral
133 MHz(3.3V I/O or
Universal33 MHz 33
(5V I/O)33
(5V I/O)33
(5V I/O)33
(5V I/O)33
(5V I/O)conventionalsystem 66 MHz 33 66 a) 331
b) 66a) 331
b) 6666 MHz 33 33 66 66
100 MHz 33 a) 331
b) 6666 100PCI-X
system133 MHz 33 a) 331
b) 6666 133
Legend
xx Conventional PCI system or expansion card operating in conventional mode.xx = nominal clock frequency in MHz.
xx PCI-X system and expansion card operating in PCI-X mode.xx = nominal clock frequency in MHz.
xx Most popular cases.
![Page 18: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/18.jpg)
18
System Initialization & Interoperability� Device and Expansion Card Requirement
� PCI-X Expansion card identifies PCI-X capability via PCIXCAP pin– (open = PCI-X 133, pulldown = PCI-X 66, GND = conv. PCI)
� PCI-X devices enter PCI-X mode when RST# deasserts with TRDY#, STOP#, or DEVSEL# asserted on an idle bus
� System Requirements (M66EN & PCIXCAP)� The system is required to determine the proper operating mode for
the bus and to apply the appropriate PCI-X initialization pattern to the bus before the rising edge of RST#
� Mode and Frequency Initialization Sequence� Bus mode (Conventional PCI or PCI-X) � If PCI-X “133”, “100”, “66” modes� If conventional PCI “66”, “33” modes
![Page 19: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/19.jpg)
19
Reset & Initialization Sequence
PCI_CLK
RST#trlcx trst_clk (ref)
tprsu tprh
trst (ref)
STOP#
TRDY#
IRDY#
FRAME#
trhff(ref)
DEVSEL#
transparentlatch
Q
D
PCI-Xinitialization
patterndecode
PCI-X_mode_en
TRDY#
STOP#
DEVSEL#
FRAME#
IRDY#
EnRST#
� Conventional vs. PCI-X mode selected at rising edge or RST#
� PCI-X bus freq range encoded at rising edge of RST#
![Page 20: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/20.jpg)
20
PCI-X Protocol
![Page 21: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/21.jpg)
21
PCI 2.2 vs. PCI-X Comparison�Write transaction, same device select timing,
same wait states, 6 data phases �Conventional PCI bus requires 9 clocks,
PCI-X bus requires 10 clocks
PCI 2.2 PCI 2.2 PCIPCI--XX
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11 12
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Bus Transaction
DEVSEL#
TRDY#
IRDY#
FRAME#
BUS CMDC/BE# BE#'s-0ATTR BE#'s-1 BE#'s-2 BE#'s-3 BE#'s-4 BE#'s-5
ADDRESSAD DATA-0 DATA-1 DATA-2 DATA-3 DATA-4ATTR DATA-5
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11 12
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Bus Transaction
ADDRESSAD DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5
BUS CMDC/BE# BE#'s-0 BE#'s-1 BE#'s-4BE#'s-3BE#'s-2 BE#'s-5
FRAME#
IRDY#
TRDY#
DEVSEL#
![Page 22: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/22.jpg)
22
Transaction Phases
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11
DataPhase
DataPhase
DataPhase
DataPhase
DataPhase
DataPhase
Address Phase
Turn Around
Attribute Phase
Target Response Phase
12
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Bus Transaction
Initiator Termination
DEVSEL#
TRDY#
IRDY#
FRAME#
BUS CMDC/BE# BE#'s-0ATTR BE#'s-1 BE#'s-2 BE#'s-3 BE#'s-4 BE#'s-5
ADDRESSAD DATA-0 DATA-1 DATA-2 DATA-3 DATA-4ATTR DATA-5
![Page 23: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/23.jpg)
23
PCI 2.1/2.2PCI 2.1/2.2
How can the bus be faster but the timing easier?Behind the Curtain
� Register-to-register design allows maximum flight time
LogicLogic
RegReg
I/O BuffersI/O BuffersBoundary ScanBoundary Scan
PCIPCIClockClock
2.1/2.2 2.1/2.2
PCIPCI--XX
PCIPCI--XX RegRegPCIPCIClockClock
![Page 24: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/24.jpg)
24
Behind the Curtain
SenderAssertsSignal
Propagation delay across bus
ReceiverDecodes Logic
ReceiverResponds
Receiverregisters
signal
32
PCI -X Clock
1
PCI Clock, 33 MHz
SenderAssertsSignal
Propagation delay across bus
ReceiverResponds
Receiver DecodesLogic
1 2
SenderAssertsSignal
Propagation delayacross bus
Receiver Decodes Logic
PCI Clock, 66 MHz
321
ReceiverResponds
� PCI @ 33MHz� 30 ns period� 7 ns setup time
� PCI-X registered protocol allocates a full clock period for logic decision
� @ 66MHz - 15ns� @ 133MHz - 7.5ns
� PCI @ 66MHz� 15 ns period� 3ns setup time
![Page 25: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/25.jpg)
25
PCI-X TermsAllowableDisconnectBoundary (ADB)
The initiator and target are permitted to disconnect byte-counttransactions only on naturally aligned 128-byte boundaries.
Requester Initiator that first introduces a transaction into the PCI-X domain.
Completer The device addressed by a transaction (other than a SplitCompletion).
Sequence One or more transactions associated with carrying out a singlelogical transfer by a requester.
Attributes Byte count, requester or completer ID, bus number, sequencenumber, and other transaction handling instructions.
Split Transaction A single logical transfer containing a initial transaction (the SplitRequest) that the target (the completer) terminates with SplitResponse, followed by one or more transactions (the SplitCompletions) initiated by the completer to send the read data (if aread) or a completion message back to the requester.
![Page 26: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/26.jpg)
26
PCI-X Peer Transaction Flow
SplitCompletion
InitiatorI/O
CFGInt. Ack.
MemRead
SplitResponse
I/O, CFG,
Int. Ack, MemRead
ImmediateResponse
PostedMemoryWrites,
I/O,CFG,
Int. Ack,MemRead
RetryResponse
(TransactionRescheduled)
ErrorTermination(Sequence
Ends)
Completer(e.g. Host Bridge)
Initiator Interface
Target Interface Initiator Interface
Target Interface
SequenceInitiator
I/O,CFG,
Int. Ack.Memory,
Special Cyc,
SequenceRequester
I/OCFG
Int. Ack.MemRead
Requester(e.g. Adapter Card)
ErrorTermination(Sequence
Ends)StartHere
StartHereStartHere
PCI-X Mode
![Page 27: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/27.jpg)
27
PCI-X Features
Decode Speed PCI-X Conventional PCI1 clock after address Not Supported Fast2 clocks after address Decode A Medium3 clocks after address Decode B Slow4 clocks after address Decode C SUB6 clocks after address SUB N/A
� AD bus specifies starting byte address (including AD[2:0])� Byte Enable bus is reserved (driven high) for all transactions
except Memory Write� Wait states not allowed, except target initial wait states. Always
pairs for memory write and Split Completion� PCI-X “DWORD transaction” like convention PCI “single data
phase” transaction (Config, I/O, Special Cycle).� DEVSEL# decode speed:
![Page 28: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/28.jpg)
28
Burst Transactions DWORD TransactionsCommands:• Memory Read Block• Memory Write Block• Memory Write• Alias to Memory Read Block• Alias to Memory Write Block• Split Completion
Commands:• Interrupt Acknowledge• Special Cycle• I/O Read• I/O Write• Configuration Read• Configuration Write• Memory Read DWORD
64- or 32-bit data transfers. 32-bit data transfers onlyStarting address specified on AD bus down to a byteaddress (includes all AD bus).
Starting address specified on AD bus down to a byteaddress (includes all AD bus), except forconfiguration transactions, which are DWORDaligned (AD[1:0] indicate configuration transactiontype).
Supports one or more data phases and always inaddress order.
Supports only single data phase.
During the data phases the C/BE# bus is reservedand driven high by the initiator for all transactionsexcept Memory Write.
The C/BE# bus contains valid byte enables forMemory Write transactions. Any byte enablepattern is permitted (between the starting and endingaddress, inclusive), including no byte enablesasserted.
During the attribute phase the Requester Attributescontains valid byte enables. Any byte enable patternis permitted, including no byte enables asserted.
During the data phase the C/BE# bus is reserved anddriven high by the initiator.
PCI-X Burst and DWORD Transactions
![Page 29: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/29.jpg)
29
C /B E [3 :0 ]#o r
C /B E [7 :4 ]#
C o n v e n tio n a l P C IC o m m a n d(re fe re n c e )
P C I-X C o m m a n d L e n g th
0 0 0 0 b In te r ru p tA c k n o w le d g e
In te r ru p t A c k n o w le d g e D W O R D
0 0 0 1 b S p e c ia l C y c le s S p e c ia l C y c le s D W O R D0 0 1 0 b I/O R e a d I/O R e a d D W O R D0 0 1 1 b I/O W r ite I/O W r ite D W O R D0 1 0 0 b R e s e rv e d R e s e rv e d n a0 1 0 1 b R e s e rv e d R e s e rv e d n a0 1 1 0 b M e m o ry R e a d M e m o ry R e a d D W O R D D W O R D0 1 1 1 b M e m o ry W r ite M e m o ry W r ite B u rs t1 0 0 0 b R e s e rv e d A lia s to M e m o ry R e a d B lo c k B u rs t1 0 0 1 b R e s e rv e d A lia s to M e m o ry W r ite B lo c k B u rs t1 0 1 0 b C o n f ig u ra tio n R e a d C o n f ig u ra tio n R e a d D W O R D1 0 1 1 b C o n f ig u ra tio n W r ite C o n f ig u ra tio n W r ite D W O R D1 1 0 0 b M e m o ry R e a d
M u lt ip leS p lit C o m p le tio n B u rs t
1 1 0 1 b D u a l A d d re s s C y c le D u a l A d d re s s C y c le n a1 1 1 0 b M e m o ry R e a d L in e M e m o ry R e a d B lo c k B u rs t1 1 1 1 b M e m o ry W r ite a n d
In v a lid a teM e m o ry W r ite B lo c k B u rs t
PCI-X Command Encoding
![Page 30: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/30.jpg)
30
DWORD TransactionsDWORD Write Transaction DWORD Write Transaction
with No Wait States. with No Wait States.
Notice the initiator continues driving the Notice the initiator continues driving the bus, and IRDY# remains asserted in clock bus, and IRDY# remains asserted in clock 7, even though this is one clock past the7, even though this is one clock past the
single clock in which data was transferred single clock in which data was transferred (clock 6). (clock 6). In PCIIn PCI--X the initiator requires two X the initiator requires two
clocks to respond to the assertion of clocks to respond to the assertion of TRDY#.TRDY#.
PCI_CLK
1 2 3 4 5 6 7 8 9 10
ADDRESSAD[31::00] ATTR DATA-0
FRAME#
IRDY#
TRDY#
STOP#
DEVSEL#
BUS CMDC/BE[3::0]# ATTR BE#'s = Fh
DWORD Read Transaction DWORD Read Transaction with two Target initial Wait with two Target initial Wait
StatesStates
Notice BE# bus reserved Notice BE# bus reserved and driven high, because and driven high, because byte enables are in the byte enables are in the
requester attributesrequester attributes
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11 12
ADDRESSAD[31::00] ATTR DATA-0
BUS CMDC/BE[3::0]# ATTR
FRAME#
IRDY#
TRDY#
DEVSEL#
BE#'s = Fh
![Page 31: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/31.jpg)
31
PCI-X Protocol--Attributes and Split Transactions
![Page 32: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/32.jpg)
32
Transaction Attributes
RO -- Relax orderingNS -- No Snoop
UpperByte Count
C/BE[3-0]# AD[31:0]
LowerByte Count
RequesterBus
Number
RequesterDevice Number
RequesterFunctionNumber
TagNS
ROR
000708101115162324313235 282930
Requester Attributes for Burst Transactions (most memory read/write)
Requester Attributes for DWORD Transactions (Memory Read DWORD, I/O read/write, Special Cycle, Int Ack)
Byte Enables
C/BE[3-0]# AD[31:0]
ReservedRequester
BusNumber
RequesterDevice Number
RequesterFunctionNumber
TagNS
ROR
000708101115162324313235 282930
![Page 33: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/33.jpg)
33
Transaction Attributes
SCM -- Split Completion MessageSCE -- Split Completion ErrorBCM -- Byte Count Modified
UpperByte Count
LowerByte Count
CompleterBus
Number
CompleterDeviceNumber
CompleterFunctionNumber
R
00070810111516232431
SCM
2829BCM
SCE
30
AD[31:0]C/BE[3:0]#
3 0
RequesterBus
Number
RequesterDevice
Number
RequesterFunctionNumber
Lower Address [6:0]BUS CMD
00070810111516232431
C/BE[3-0]#
TagR RO R
062930 28
AD[31:0]
03
R
Split Completion Address
Completer Attributes
RO -- Relax ordering,
![Page 34: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/34.jpg)
34
Split Transactions
Target
Initiater
Completer B
Target
Initiater
Requester A
PCI-X Bus
Address,Memory Read
RequesterAttributes
REQGNT
DataSplit Reponse
SplitCompletion
CompleterAttributes DATA
REQGNT(Requester's
Attribute)
ImmediateResponse
SplitTransactionCompleter
SplitTransactionRequester
![Page 35: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/35.jpg)
35
PCI-X Protocol--Configuration Transactions
![Page 36: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/36.jpg)
36
Config Address & Attributes
Secondary Bus NumberRequester
BusNumber
RequesterDevice
Number
RequesterFunctionNumber
Tag
00070810111516232431
RRR
282930
Byte Enables
3235
AD[31::00]C/BE#[3::0]
Type 1
Type 1
Type 0
BUS CMD
C/BE[3-0]#
See PCI 2.2 Specification FuncNumber Register Number
0001020708101131
0 0
AD[31:0]
3 0
Reserved Bus Number DevNumber
FuncNumber
RegisterNumberBUS CMD
000102070810111516232431
C/BE[3-0]#
0 1
AD[31:0]
3 0PCI-X Config Type 1 to Type 0 Configuration Address
PCI 2.2 Type 1 to Type 0 Configuration Address (ref)
BUS CMD
C/BE[3-0]#
3 0
Reserved Bus Number Dev Number FuncNumber Register Number
000102070810111516232431
0 1
AD[31:0]
ReservedDev
NumberFunc
NumberRegisterNumberBUS CMD
00010207081011151631
C/BE[3-0]#
0 0
AD[31:0]
3 0
Type 0
Configuration Attributes
![Page 37: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/37.jpg)
37
Configuration Transactions
4 clocks of valid address before asserting FRAME# 4 clocks of valid address before asserting FRAME#
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11 12 13 14
DEVSEL#
TRDY#
IRDY#
FRAME#
C/BE[3:0]# BUS CMD ATTR BE# 's = Fh
AD[31:0] ADDRESS DATA-0ATTR
IDSEL
![Page 38: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/38.jpg)
38
PCI-X Protocol--PCI-X Bridges
![Page 39: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/39.jpg)
39
PCI-X Bridge Design“This is not your father’s bridge”
�Split Transactions make consistently high-performance bridges possible
� No speculative prefetch penalty� Forwards read completion data similar to posted memory writes
� Converts between PCI-X and conventional PCI, if necessary� Frequency and mode of secondary bus independent of
primary bus� Transaction ordering rules similar to conventional PCI� Performance-tuning registers to enable adjustment of request
rate to match completion rate
![Page 40: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/40.jpg)
40
PCI-X Bridge Transaction Flow
Target Interface
Bridge X
Primary Bus
Secondary Bus
SplitResponse
I/O CFG
Int. Ack. MemRead
ErrorTermination(Sequence
Ends)
ImmediateResponse
PostedMemory
Write
RetryResponse
(TransactionRescheduled)
ForwardedSequence
I/OCFG
Int. Ack.MemRead
ForwardedSequence
PostedMemory
Write
ImmediateResponseCompletion
DataAccepted
RetryResponse
(TransactionRescheduled)
ErrorTermination(Sequence
Ends)
ForwardedCompletion Completion Data Read SCM Write Ack Error Status
SplitCompletion
InitiatorI/O
CFGInt. Ack.
MemRead
SplitCompletion
InitiatorI/O
CFGInt. Ack.
MemRead
SplitResponse
I/O CFG
Int. Ack. MemRead
ImmediateResponse
RetryResponse
(TransactionRescheduled)
ErrorTermination(Sequence
Ends)
Completer(e.g. Host Bridge)
SequenceInitiator
I/OCFG
Int. Ack.Memory
Special Cyc
SequenceRequestor
I/OCFG
Int. Ack.MemRead
Requester(e.g. Adapter Card)
ErrorTermination(Sequence
Ends)
SplitCompletionExceptionMessageInitiator
Initiator Interface
Target Interface Initiator Interface
Target Interface
Initiator Interface
Target InterfaceInitiator Interface
StartHere
StartHereStartHere
![Page 41: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/41.jpg)
41
PCI-X Bridge Design--Summary
�Great system performance�More complex that conventional PCI bridges
�See the PCI-X spec for full details
![Page 42: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/42.jpg)
42
PCI-X Protocol--Parity Generation and Checking
![Page 43: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/43.jpg)
43
Burst Write Parity Operation
Burst Write or Split Completion Transaction Parity Operation
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11 12 13
ADDRESSAD[31:0] DATA-0 DATA-1ATTR ADDRESS ATTRDATA-2 DATA-3
C/BE[3:0]# BUS CMD ATTR BUS CMD ATTRBE#'s-0 BE#'s-2BE#'s-1 BE#'s-3
PAR
PERR#
FRAME#
IRDY#
TRDY#
DEVSEL#
14
DATA-0
BE#'s-0
![Page 44: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/44.jpg)
44
Burst Read Parity Operation
PCI_CLK
1 2 3 4 5 6 7 8 9 10 1211 13
ADDRESSAD DATA-0 DATA-1ATTR ADDRESS ATTRDATA-2 DATA-3
C/BE# BUS CMD ATTR BUS CMD ATTR
PAR
FRAME#
TRDY#
DEVSEL#
PERR#
IRDY#
14
DATA-0
Burst Read Transaction (Immediate Completion) Parity Operation
![Page 45: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/45.jpg)
45
DWORD Read Parity Operation
DWORD Read Parity Operation with Decode A and No Initial Wait States
PCI_CLK
1 2 3 4 5 6 7 8 9 10 1211
ADDRESSAD[31::00] DATA-0ATTR
C/BE[3::0]# BUS CMD ATTR
PAR
FRAME#
TRDY#
DEVSEL#
PERR#
IRDY#
ADDRESS ATTR
BUS CMD ATTR
![Page 46: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/46.jpg)
46
DWORD Read Parity Operation
DWORD Read Parity Operation with Decode B and No Initial Wait States
PCI_CLK
1 2 3 4 5 6 7 8 9 10 1211 13
ADDRESSAD[31::00] DATA-0ATTR
C/BE[3::0]# BUS CMD ATTR
PAR
FRAME#
TRDY#
DEVSEL#
PERR#
IRDY#
ADDRESS ATTR
BUS CMD ATTR
![Page 47: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/47.jpg)
47
PCI-X Protocol--Exception Handling
� All PCI-X devices must provide one of the following levels of support for data parity error recovery:
� Notify device driver of problem. Driver attempts recover or resets device or system.
� Assert SERR#.� Split Transaction Exception rules
� Parity errors on Split Completion� Master-Abort and Target-Abort messages
� Status registers in configuration space can only be cleared by the system-specific software after logging the exceptions
![Page 48: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/48.jpg)
48
PCI-X Protocol--Arbitration Rules
� Relevant clock for GNT# one clock earlier than conventional PCI (registered bus)
� In general, GNT# asserted two clocks prior to start of transaction� Initiator permitted to start transaction one clock after GNT# deasserted� If the arbiter deasserts GNT# to one device, it cannot assert GNT# to
another device until the next clock.
� Fair opportunities for all devices to execute configuration transactions
� In PCI Hot-Plug systems arbiter must coordinate with the Hot-Plug Controller
� The default Latency Timer value for initiators in PCI-X mode is 31
![Page 49: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/49.jpg)
49
PCI Hot Plug Support� Hardware Impact
� Hot-Plug Controller provides to the Hot-Plug System Driver the means to check the PCIXCAP pin to identify PCI-X adapters
� The Hot-Plug Controller drives the PCI-X initialization pattern on the bus with the proper timing prior to the rising edge of RST# for that slot
� The Hot-Plug Controller coordinates with the arbiter for bus ownership during hot insertion
� PCI-X devices ignore TRDY#, STOP# and DEVSEL# on an idle bus
� Software Impact� The Hot-Plug System Driver must read the inserted card’s M66EN
and PCIXCAP pin to ensure that inserted adapter supports bus frequency and operating mode of the bus
![Page 50: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/50.jpg)
50
Software Aspects of PCI-X
![Page 51: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/51.jpg)
51
Software Compatibility� No OS or driver change required
� New config registers default to functional values� Optional performance tuning registers � Other config registers unchanged� No device programming model changes required
� Optional improved error handling� Enables smart device and new driver to recover
from PERR# event� Updated Hot-Plug System Driver
![Page 52: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/52.jpg)
52
Configuration Space� PCI-X devices use the standard PCI config header� New PCI-X registers use the Capability List� The PCI-X list item includes
� An 8-bit PCI-X Capability ID (standard reg)� An 8-bit pointer to next list item (standard reg)� A 32-bit PCI-X Command--controls various modes
and features of the PCI-X device� A 24-bit PCI-X Status--identifies the capabilities and
current operating mode of the device� PCI-X bridges have different registers
![Page 53: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/53.jpg)
53
Software Summary--“It Just Works”
Up to 8 times I/O throughput improvement without changing your OS
OpenVMS
3.x/9x/NT/2000
ServerServer
![Page 54: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/54.jpg)
54
PCI-X Electrical Design
![Page 55: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/55.jpg)
55
A Word to the Wise� 33 MHz PCI spec was forgiving� High-freq designs are more complex
� PCI 66, PCI-X 66, PCI-X 133� Use high-freq design techniques and principles
– E.g., Electrical Design Considerations for PCI-X and 66-MHz PCI Cards,http://www.compaq.com/support/techpubs/whitepapers/WhitePapers_Industry_Technology.html
Doc # tc000301tb� Simulate system topology� Must follow PCI-X spec precisely
� If you don’t understand every word of Chap 9, find someone who does.
![Page 56: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/56.jpg)
56
Signal Quality Guidelines for PCI-X� Report of the PCI-X Electrical Subgroup to be available
mid year 2000 � Includes discussion of:
� I/O Buffer Design� Overshoot� Ringback� Settling time � Inter Symbol Interference� Secondary Effects
– Ground Bounce– Cross Talk– DC Offset
� Summary results of almost 2 million SPICE simulations
![Page 57: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/57.jpg)
57
Parameter 133 MHzPCI-X
100 MHzPCI-X
66 MHzPCI-X
66 MHzConventional PCI
(ref)
33 MHzConventional PCI
(ref)
Units
Tval (max) 3.8 3.8 3.8 6 11 nsTprop (max) 2.0 4.5 9.0 5 10 nsTskew (max) 0.5 0.5 0.5 1 2 nsTsu (max) 1.2 1.2 1.7 3 7 nsTcyc 7.5 10 15 15 30 ns
Timing Budget
Parameter PCI-X 66 MHzConventional
PCI (ref)
33 MHzConventional
PCI (ref)
Units
Tval (min) 0.7 2 2 nsTprop (min) 0.3 0 0 nsTskew (max) 0.5 1 2 nsTh (min) 0.5 0 0 ns
Setup Time Budget
Hold Time Budget
![Page 58: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/58.jpg)
58
PCI-X V/I Curves vs. PCI 2.2 V/I Curves
PCI-X Pull-Up Output Buffer V/I Curves PCI-X Pull-Down Output Buffer V/I Curves
00.10.20.30.40.50.60.70.80.9
1
0 20 40 60 80Iout (mA)
Vout
(vol
t)
- - - -Vcc Vcc Vcc Vcc
Vcc x
PCI-XPCI
00.10.20.30.40.50.60.70.80.9
1
0 20 40 60 80
Iout (mA)Vo
ut (v
olt)
- Vcc - Vcc - Vcc - Vcc
Vcc x
PCI-X
PCI
![Page 59: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/59.jpg)
59
PCI-X Mechanical Requirements
![Page 60: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/60.jpg)
60
PCI-X Mechanical Requirements� Same card and slot mechanical requirements as
conventional PCI (3.3V I/O)� New labeling requirement (ECR in progress)
� Systems must identify slot capability (method not specified)� Cards must be marked 66 133
Standard LengthAdd-in CardLow ProfilePCI
Add-in Card
Short Length(Variable Height)
Add-in Card
66
66
66
![Page 61: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/61.jpg)
61
PCI-X Performance
![Page 62: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/62.jpg)
62
PCI-X Demonstration Using Compaq ProLiant 8500 Server
� Cable-less, tool-less design� Serviceable in minutes� Reduce training & downtime� Improved product availability
Key Customer Benefit - Easy configuration, installation, upgrade, and service.
![Page 63: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/63.jpg)
63
Compaq & IntelIndustry Standard 8-Way Architecture
100 MHz AGTL + bus 1 100 MHz AGTL + bus 2
ProFusion
2 Way Interleaved SDRAM Memory
2 Way Interleaved SDRAM Memory
Even
Mem
ory
Port
Even
Mem
ory
Port
Odd
Mem
ory
Port
Odd
Mem
ory
Port
Bus 1 CacheBus 1 CacheCoherencyCoherencyAcceleratorAccelerator
Bus 2 CacheBus 2 CacheCoherencyCoherencyAcceleratorAccelerator
100 MHz GTL I/O bus
Pentium IIIXeon
Pentium IIIXeon
Pentium IIIXeon
Pentium IIIXeon
33 MHz 33 MHz 66 MHz
With only I/O With only I/O subsystem subsystem
upgradeupgrade
PCIPCI--X X vs.vs.
conventional PCIconventional PCIprotocol protocol
comparisoncomparison
Pentium IIIXeon
Pentium IIIXeon
Pentium IIIXeon
Pentium IIIXeon
![Page 64: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/64.jpg)
64
PCI-X Advanced Protocol Gives You More Usable Bandwidth at Any Speed
PCI 64bit/33MHz
140MByte/s 2000
PCI-X 64bit/33MHz
230MByte/s
With “real hardware” With “real hardware” PCIPCI--X is over 60 % faster X is over 60 % faster
than conventional PCIthan conventional PCI
6 conventional PCI 6 conventional PCI adapters pulling data adapters pulling data
2 PCI2 PCI--X adapters X adapters pulling data pulling data
vs.vs.
(33 MHz for demo purposes only)(33 MHz for demo purposes only)(your mileage may vary)(your mileage may vary)
![Page 65: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/65.jpg)
65
PCI-X vs. PCI: 4K-Byte Read Performance
SCSI
SCSI
Enet
Enet
FC
FC
PCI vs. PCIPCI vs. PCI--XX
PCI-XBridge
66MHz PCI66MHz PCI--X up to X up to 33% faster33% faster than 66MHz PCI 2.2than 66MHz PCI 2.2
1 2 3 4 5 6 7
PCI (33 MHz, MRM)
PCI (66 MHz, MRM)PCI-X (66MHz)
PCI-X (100 MHz)
100150200250300350400450500550600650700
MB
/sec
# of Requests
PCI (33 MHz, MRM) PCI (66 MHz, MRM) PCI-X (66MHz) PCI-X (100 MHz)
Note: Assumes Ideal memory controller with 32Note: Assumes Ideal memory controller with 32--byte CPU cache line and ideal 64byte CPU cache line and ideal 64--bit PCI adaptersbit PCI adapters
![Page 66: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/66.jpg)
66
Prognostications for PCI-X� Rapid deployment in Server market beginning 2H2000
� Server vendors on Workgroup: Compaq, Dell, HP, IBM, Intel� Peripheral vendors on Workgroup
– Disk: Adaptec, Mylex (IBM), LSI– NIC: 3Com, Intel
� Si and IP vendors on Workgroup: InSilicon (Phoenix), Intel, LSI,ServerWorks
� Migration into workstation, desktop, and embedded markets� Replaces conv PCI as devices move to 3.3V-only in next 2 years
� Continue to dominate local I/O applications even after InfiniBand begins solving new problems in distributed computing applications in 2001 and 2002
� Extend the life of PCI 5-10 years
![Page 67: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/67.jpg)
67
Summary� PCI-X compatibility
� Fully interoperable with conventional PCI� Easy design migration from conventional PCI
– Your conv PCI experience prepares you for PCI-X � Easier electrical design than 66 MHz conventional PCI� No OS or driver changes required (only Hot-Plug System Driver)
� PCI-X performance� Over 1 Gbyte/s� Byte counts, Split Transactions make whole system work smarter� PCI-X adapter cards are good citizens: No more bus hogs� Efficient P2P bridges for hierarchy of buses� Increases system flexibility: 4-slots per bus at 66MHz � Integrates well with distributed I/O standards like InfiniBand
![Page 68: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/68.jpg)
68
Take away
PCI-X is THE logical next step for your I/O designs
Hardware and Software CompatibilityBetter PerformanceNeed I say more?
![Page 69: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/69.jpg)
69
For More Information�PCI SIG www.pcisig.com
� PCI-X 1.0 specificationhttp://www.pcisig.com/members/index.html
� PCI-X compliance checklisthttp://www.pcisig.com/tech/docs.html
� PCI-X spec erratahttp://www.pcisig.com/tech/ecn_ecr.html
� Technical [email protected]
� General information [email protected]
![Page 70: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/70.jpg)
70
For More Information�Compaq
� PCI-X enablement site (sample core, docs, Golden Master Program, links to other vendors):
www.compaq.com/PCI-X� Information
[email protected]� Electrical Design Considerations for
PCI-X and 66-MHz PCI Cards,http://www.compaq.com/support/techpubs/whitepapers/WhitePapers_Industry_Technology.html,
Doc # tc000301tb
![Page 71: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/71.jpg)
Architecture Tutorial
Alan GoodrumChairman, PCI-X Workgroup
Staff Fellow, Compaq Computer Corporation
May 23, 2000Applied Computing Conference 2000
![Page 72: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/72.jpg)
72
Backup
![Page 73: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/73.jpg)
73
PCI-X ComparisonFEATURE PCI AGP 1.0 AGP 2.0 PCI-XPCI slot compatibility Yes No No Yes100 MHz Bus speed No No No Yes133 MHz Bus speed No 66 MHz DDR 66 MHz DDR Yes266 MHz Bus speed No No 66 MHz QDR * NoData Bus Width 32/64 32 32 32/64Address Bus Width 32/64 32/36/64 32/47/64 64Max Bus Bandwidth(MB/sec) 533 533 1064 1064Multiple slots Yes No No YesHierarchical bus topology Yes No No YesSplit Transactions Open Yes Yes YesTransaction Byte count No 32 (256 **) 64 (256**) 4KNon-coherent Transactions No Yes Yes YesNo/Relax Ordering Rules No Yes Yes YesDevice & Bus # ID *** No No, N/A No, N/A Yes* Quad data rate (4X)** Special long read command*** Allows system to be tuned more precisely for optimal performance
![Page 74: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/74.jpg)
74
Background--The need for a faster PCI bus
![Page 75: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/75.jpg)
75
Changing Systems ArchitecturesEmergenceEmergence
of distributedof distributedsystems systems
architectures, architectures, SAN and STANSAN and STAN
Memory
Memory Memory Memory
S A N
Current PCI Based Systems Need a RefreshCurrent PCI Based Systems Need a Refresh
SeparatingSeparatingCPUCPU--plexesplexes
and I/Oand I/Oby fast thin by fast thin
bus segmentsbus segments
PCI-64
PCI-32
PCI-32
Compaq’sAlpha
Faster CPUFaster CPUrequires morerequires moreI/O bandwidthI/O bandwidth
![Page 76: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/76.jpg)
76
I/O Bandwidth vs. Time
1986 1988 1990 1992 1994 1996 1998 2000 2002
Ethernet
1 Gbit/s
InternetBackbone
OC 192
T3
SCSI
10 Gbit/s
EISA
1
10
100
1,000
10,000
Ban
dwid
th (M
B/s
)
ISA
6MHz 16-bit
10MHz 32-bit
33MHz 32/64-bit
66MHz 32/64-bit
![Page 77: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/77.jpg)
77
PCI-X Objectives� Address customer requirements for greater I/O performance� Evolutionary I/O upgrade
� Investment protection� Leverage PCI prevalence--
“The most successful interconnect”� Maintain compatibility with installed base
� More Slots @ 66MHz� Ease of design� Strong industry support
![Page 78: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/78.jpg)
78
PCI-X Burst Write with Target Wait States
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11
Initiator SignalsTermination
2 Clocks beforeEnd of Transaction
12 13 14
PCI Bus
Initiator's View of the PCI Bus
Target's View of the PCI Bus
BusTransaction
Target InitialWait State Pair
15
DecodeSpeed A
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
FRAME#
ADDRESSAD ATTR DATA-0 DATA-1 DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5
BUS CMDC/BE ATTR BE#'s-0 BE#'s-1 BE#'s-0 BE#'s-1 BE#'s-2 BE#'s-3 BE#'s-4 BE#'s-5
IRDY#
TRDY#
DEVSEL#
FRAME#
IRDY#
AD DATA-1 DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5ADDRESS DATA-0ATTR
C/BE# BE#'s-1 BE#'s-0 BE#'s-1 BE#'s-2 BE#'s-3 BE#'s-4 BE#'s-5BUS CMD BE#'s-0ATTR
s1_TRDY#
s1_DEVSEL#
s1_FRAME#
ADDRESS ATTRs1_AD DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5
s1_IRDY#
TRDY#
DEVSEL#
BUS CMD ATTRs1_C/BE# BE#'s-0 BE#'s-1 BE#'s-2 BE#'s-3 BE#'s-4 BE#'s-5
![Page 79: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/79.jpg)
79
PCI-X Burst Read with Target Wait States
PCI_CLK
1 2 3 4 5 6 7 8 9 10 11
Initiator SignalsTermination
2 Clocks beforeEnd of Transaction
12 13 14
PCI Bus
Initiator's View of the PCI Bus
BusTransaction
Target InitialWait State
DecodeSpeed A
15
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Data Transfer
Target's View of the PCI Bus
DEVSEL#
TRDY#
DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5AD
s1_IRDY#
BUS CMD ATTRs1_C/BE#
ADDRESS ATTRs1_AD
s1_FRAME#
s1_DEVSEL#
s1_TRDY#
C/BE# BUS CMD ATTR BE#'s = FF
DATA-0 DATA-1 DATA-2 DATA-3 DATA-4 DATA-5s1_AD
IRDY#
FRAME#
DEVSEL#
TRDY#
IRDY#
ADDRESSAD DATA-0 DATA-1 DATA-2ATTR DATA-3 DATA-4 DATA-5
FRAME#
BUS CMDC/BE ATTR BE#'s = FF
AD ADDRESS ATTR
![Page 80: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/80.jpg)
80
Split Transactions� Bus efficiency of Read almost as good as Write � Split Transaction components
Step 1. Requester requests bus and arbiter grants busStep 2. Requester initiates transactionStep 3. Targeted (completer) communicates intent with new
target termination, Split ResponseStep 4. Completer executes transaction internallyStep 5. Completer requests bus and arbiter grants busStep 6. Completer initiates Split Completion
� Split Completion routed back to requester across bridges using requester’s bus number and device number
![Page 81: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/81.jpg)
81
Logic Block Diagram for Bypassing Source Sampling
PCILOGIC
PCI BusZ
Logic Gates I/O Buffer
outputenable
IOB1
data_in
M3
PCI-X / PCIMode
PCI-X Feedback Enable
PCI_in
* -- All flip-flops are assumed rising edge triggered
F7d
q ck*
F6d
q ck*
PCI Feedback Enable
F5d
qck*
F1d
qck*
F2d
qck*
M1 SourcingPCI
SamplededPCI F7
d
qck*
F5d
q ck*
SourcingPCI-X
M2
data_out
data_output_enable
![Page 82: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/82.jpg)
82
Device Internal Timing Example
PCILOGIC
PCI BusF2
d
q W2AW2B
F1d
q W1AW1B
data_in
data_out
data_output_enable
F3d
q
P1
W3A
clockdriver PLL
P2
A*
ZB*
X
M1
M2
M3
bs_in_1
bs_in_2
bs_in_3
C*
PCI Clock
Logic Gates
I/O Buffer
Package
oe
D
Y
ASIC InternalClock
Distribution toA, B, & C
CB2
IOB1
* -- All Flip Flops are assumed to be rising edge triggered
![Page 83: Architecture Tutorialfienup/cs142f05/lectures/PCI-X_tutorial_final.pdf · PCI Slots PCI-X Slots 32-bit 33 MHz 133 MB/s N/A 64-bit 66 MHz 533 MB/s 64-bit 100 MHz 800 MB/s N/A 64-bit](https://reader034.fdocuments.us/reader034/viewer/2022043014/5fb165ded092251c62603fe9/html5/thumbnails/83.jpg)
83
PCI-X vs PCI: 8x512-Byte Read Simulation
SCSI
SCSI
Enet
Enet
FC
FC
PCI vs. PCIPCI vs. PCI--XX
PCI-XBridge
66MHz PCI66MHz PCI--X up to X up to 19% faster19% faster than 66MHz PCI 2.2than 66MHz PCI 2.2
1 23
45
67
PCI (33 MHz, MRM)
PCI (66 MHz, MRM)
PCI-X (66MHz)PCI-X (100 MHz)
100
150
200
250
300
350
400
450
500
550
600
650
700
MB
/sec
# of Requests
PCI (33 MHz, MRM) PCI (66 MHz, MRM) PCI-X (66MHz) PCI-X (100 MHz)
Note: Assumes ideal memory controller with 32Note: Assumes ideal memory controller with 32--byte CPU cache line and ideal 64byte CPU cache line and ideal 64--bit PCI adaptersbit PCI adapters