Dr A Sahu Dept of Comp Sc & Engg. IIT Guwahati. PCI Devices NIC Cards NIC card architecture Access...
-
Upload
job-wilkerson -
Category
Documents
-
view
220 -
download
0
Transcript of Dr A Sahu Dept of Comp Sc & Engg. IIT Guwahati. PCI Devices NIC Cards NIC card architecture Access...
Network Interface Card & its Linux programming
Dr A SahuDept of Comp Sc & Engg.
IIT Guwahati
Outline• PCI Devices• NIC Cards• NIC card architecture• Access to NIC register –PCI access
Major Player of NIC Cards
• RealTek (Example RTK 8169S)• Broadcom (Example BCM5751)• Intel (Example 82573L NIC)
• Download manual from vender site–Or search “Realtek 8169 pdf” in google
Detecting PCI Devices : Early PCs
• Peripheral devices in the early PCs used fixed i/o-ports and fixed memory-addresses, e.g.:– Video memory address-range: 0xA0000-0xBFFFF – Programmable timer i/o-ports: 0x40-0x43– Keyboard and mouse i/o-ports: 0x60-0x64– Real-Time Clock’s i/o-ports: 0x70-0x71– Hard Disk controller’s i/o-ports: 0x01F0-01F7– Graphics controller’s i/o-ports: 0x03C0-0x3CF– Serial-port controller’s i/o-ports: 0x03F8-0x03FF– Parallel-port controller’s i/o-ports: 0x0378-0x037A
The PC’s evolution
• It became clear in the 1990s that there would be contention among equipment vendors for ‘fixed’ resource-addresses, which of course were in limited supply
• Among the goals that motivated the PCI Specification was the creation of a more flexible scheme for allocating addresses that future peripheral devices could use
PCI Configuration Space
PCI Configuration Space Body(48 doublewords – variable format)
64doublewords
PCI Configuration Space Header(16 doublewords – fixed format)
A non-volatile parameter-storage areafor each PCI device-function
PCI Configuration Header
StatusRegister
CommandRegister
DeviceID
VendorID
BISTCacheLineSize
Class CodeClass/SubClass/ProgIF
RevisionID
Base Address 0
SubsystemDevice ID
SubsystemVendor ID CardBus CIS Pointer
reserved capabilitiespointer Expansion ROM Base Address
MinimumGrant
InterruptPin reserved
LatencyTimer
HeaderType
Base Address 1
Base Address 2Base Address 3
Base Address 4Base Address 5
InterruptLine
MaximumLatency
31 0 31 0
16 doublewords
Dwords
1 - 0
3 - 2
5 - 4
7 - 6
9 - 8
11 - 10
13 - 12
15 - 14
Three IA-32 address-spaces
memoryspace(4GB)
i/o space(64KB)
PCIconfiguration
space(16MB)
accessed using a large variety of processor instructions (mov, add, or, shr, push, etc.) and virtual-to-physical address-translation
accessed only by using the processor’s special ‘in’ and ‘out’ instructions (without any translation of port-addresses)
i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space
reserved
Interface to PCI Configuration Space
CONFADD( 0x0CF8)
CONFDAT( 0x0CFC)
31 23 16 15 11 10 8 7 2 0EN
bus(8-bits)
device(5-bits)
doubleword (6-bits)
function(3-bits) 00
PCI Configuration Space Address Port (32-bits)
PCI Configuration Space Data Port (32-bits)31 0
Enable Configuration Space Mapping (1=yes, 0=no)
Reading PCI Configuration Data
• Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port
• Step two: Read the designated data from the Configuration-Space Data-Port
• Already discussed PCI-probes pciprobes.c– Lect 29..Showing vram, pciprobe.cpp
Example: network interface
• We can identify the network interface controller in PC’s by class-code 0x02
• The subclass-code 0x00 is for ‘ethernet’• We can identify the NIC from its VENDOR and DEVICE
identification-numbers:• VENDOR_ID = 0x14E4• DEVICE_ID = 0x1677
• You can use the ‘grep’ command to search for these numbers in this header-file:
</usr/src/linux/include/linux/pci_ids.h>
Vendor’s identity example
• The VENDOR-ID 0x14E4 belongs to the Broadcom Corporation
• Information about this firm may be learned from the corporation’s website
• The DEVICE-ID 0x1677 is used to signify Broadcom’s BCM5751 ethernet product
nic
Typical NIC
TX FIFO
RX FIFO
transceiver LANcableB
US
main memory
packet
buffer
CPU
Packet filtering capability
• Network Interface’s hardware needs to implement ‘filtering’ of network packets
• Otherwise the PC’s memory-usage and processor-time will be wasted handling packets not meant for this PC to receive
network packet’s layout
Destination-address (6-bytes) Source-address (6-bytes)
Each data-packet begins with the 6-byte device-address of the network interface which is intended to receive it
Your NIC’s unique address
• You can see the Hardware Address of the ethernet controller on your PC by typing:
$ /sbin/ifconfig
• Look for it in the first line of screen-output that is labeled ‘eth0’, for example:
eth0 Link encap: Ethernet HWaddr 00:11:43:C9:50:3A
Demo “nic_pci_info.c”
• Lets write a kernel module that lets users see certain register-values which pertain to the network interface in your system :– (1) the PCI Configuration Space registers– (2) the Media Access Controller’s address
• It also shows your machine’s node-name (in case you want to save the information)
How we got the MAC-address
• We do not have NIC’s programming datasheet -- but we do have Linux source code for the ‘nic_pci_info.c’ device-driver, which includes a header-file ‘tg3.h’ found here:
</usr/src/linux/drivers/net/>
• If you scroll through the #define directives you will see the offset where the hardware address is stored in the memory-mapped register-space of the ‘nic_pci_info.c’ interface
How we get PCI_NIC registers
StatusRegister
CommandRegister
DeviceID0x1677
VendorID0x14E4
BISTCacheLineSize
Class CodeClass/SubClass/ProgIF
RevisionID
Base Address 0
SubsystemDevice ID
SubsystemVendor ID CardBus CIS Pointer
reserved capabilitiespointer Expansion ROM Base Address
MinimumGrant
InterruptPin reserved
LatencyTimer
HeaderType
Base Address 1
Base Address 2Base Address 3
Base Address 4Base Address 5
InterruptLine
MaximumLatency
31 0 31 0
16 doublewords
Dwords
1 - 0
3 - 2
5 - 4
7 - 6
9 - 8
11 - 10
13 - 12
15 - 14
Linux helper-functions#include <linux/pci.h>struct pci_dev *devp;unsigned int iomem_base, iomem_size;void *io;
devp = pci_get_device( 0x14E4, 0x1677, NULL );if ( !devp ) return –ENODEV;
iomem_base = pci_resource_start( devp, 0 );iomem_size = pci_resource_len( devp, 0 );
io = ioremap( iomem_base, iomem_size );if ( !io ) return -EBUSY;
Big-Endian to Little-Endian
mac1
mac0
mac5
mac4
mac3
mac2
0x0410 0x0411 0x0412 0x0413 0x0414 0x0415 0x0416 0x0417Broadcom network interface storage-addresses
Intel IA-32 character-array storage
mac0
mac1
mac2
mac3
mac4
mac5
Little-Endian to Little-Endian
mac0
mac1
mac2
mac3
mac4
mac5
0x5400 0x5401 0x5402 0x5403 0x5404 0x5405 0x5406 0x5407 Intel network interface storage-addresses
Intel IA-32 character-array storage
mac0
mac1
mac2
mac3
mac4
mac5
For Intel NICs
• For Intel NICs :#define VENDOR_ID 0x8086 // Intel Corp#define DEVICE_ID 0x109A // 82573L NIC
• Intel’s filter-register at offset 0x5400 uses the ‘little endian’ storage-convention
Ethernet LAN
host-1 host-2 host-3
host-4
HUB“Collision Domain”
CSMA/CD = “Carrier Sense Multiple Access/Collision Detection”
Acronyms
• PCI = Peripheral Component Interconnect• MAC = Media Access Controller• Phy = Physical-layer functions• AMT = Active Management Technology• LOM = LAN On Motherboard
Intel 82573L NIC: Hardware Features
• 32K configurable RX and TX packet FIFO• IEEE 802.3x Flow Control support• Host-Memory Receive Buffers 16K/256K• IEEE 802.3ab Auto-Negotiation• TCP/UDP checksum off-loading• Jumbo-frame support (up to 16KB)• Interrupt-moderation controls
External Architecture
PCI/PCI-e Bus
10/100/1000 PHY
MAC/Controller
MDI interface
SM Businterface
EEPROM
Flashinterface
LEDindicators
S/W Defined pins
GMII/MII interface MDIO
interface
Access to PRO/1000 registers
• Device registers are hardware mapped to a range of addresses in physical memory
• You obtain the location (and the length) of this memory-range from a Base Add register in the nic device’s PCI Configuration Space
• Then you request the Linux kernel to setup an I/O ‘remapping’ of this memory-range to ‘virtual’ addresses within kernel-space
i/o-memory remapping
dynamicram
nic registers
vram
IO-APIC Local-APIC
userspace
APIC registers
kernel code/data
nic registersvram
‘virtual’ address-spacephysical address-space
1-GB
3-GB
portability syntax
• Linux provides device-driver writers with some macros for accessing i/o-memory:
#include <asm/io.h>unsigned int datum; iowrite32( datum, address ); datum = ioread32( address );
module_init()#include <linux/pci.h>#include <asm/io.h>#define E1000_STATUS 0x0008unsigned int iomem_base, iomem_size;void *io;
// remap the device’s i/o-memory into kernel spacedevp = pci_get_device( VENDOR_ID, DEVICE_ID, NULL );if ( !devp ) return –ENODEV;iomem_base = pci_resource_start( devp, 0 );iomem_size = pci_resource_len( devp, 0 );io = ioremap_nocache( iomem_base, iomem_size );if ( !io ) return –ENOSPC;
// read and display the nic’s STATUS registerdevice_status = ioread32( io + E1000_STATUS );printk( “ Device Status Register = 0x%08X \n”, status );
0
Device Status (0x0008)
? 0 0 0 0 0 0 0 0 0 0 0GIO
MasterEN
0 0 0
0 0 0 0 PHYreset ASDV
ILOS
SLU
0 TXOFF 0 0 F
D
15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
FunctionID
LU
31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16
SPEED
FD = Full-DuplexLU = Link UpTXOFF = Transmission PausedSPEED (00=10Mbps,01=100Mbps, 10=1000Mbps, 11=reserved)ASDV = Auto-negotiation Speed Detection Value 82573L
some undocumented functionality?
Thanks