Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network...
-
date post
22-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network...
![Page 1: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/1.jpg)
Accessing the NIC
A look at the mechanisms that software can use to interact with our 82573L network interface
![Page 2: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/2.jpg)
nic
Typical NIC hardware
TX FIFO
RX FIFO
transceiver LANcableB
US
main memory
packet
buffer
CPU
![Page 3: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/3.jpg)
No way to communicate
Linux operating system kernel(no knowledge of the NIC)
Network interface controller(no knowledge of the OS)
HARDWARE
SOFTWARE
These components lack a way to interact
![Page 4: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/4.jpg)
Role for a ‘device driver’
Linux operating system kernel(no knowledge of the NIC)
Network interface controller(no knowledge of the OS)
HARDWARE
SOFTWARE
device driver module (knows about both the OS and the NIC)
![Page 5: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/5.jpg)
Three x86 address-spaces
memoryspace(4GB)
i/o space(64KB)
PCIconfiguration
space(16MB)
accessed using a large variety of processor instructions (mov, add, or, shr, push, etc.) and virtual-to-physical address-translation
accessed only by using the processor’s special ‘in’ and ‘out’ instructions (without any translation of port-addresses)
i/o-ports 0x0CF8-0x0CFF dedicated to accessing PCI Configuration Space
![Page 6: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/6.jpg)
reserved
Interface to PCI Configuration Space
CONFADD( 0x0CF8)
CONFDAT( 0x0CFC)
31 23 16 15 11 10 8 7 2 0
EN
bus(8-bits)
device(5-bits)
doubleword (6-bits)
function(3-bits) 00
PCI Configuration Space Address Port (32-bits)
PCI Configuration Space Data Port (32-bits)
31 0
Enable Configuration Space Mapping (1=yes, 0=no)
![Page 7: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/7.jpg)
82573L
• Two mechanisms for accessing the NIC:– I/O space (allows booting over the network)– Memory-mapped I/O (available after booting)
• Both mechanisms require probing for PCI Configuration Space information (for I/O port-number or memory-mapped address)
• Probing requires knowing the device’s IDs (the VENDOR_ID and the DEVICE_ID)
![Page 8: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/8.jpg)
Our NIC’s ID numbers
• The VENDOR_ID for Intel Corporation:– 0x8086 (famous chip-number for IBM-PCs)
• The DEVICE_ID for Intel’s 82573L NIC:– 0x109A (found in Intel’s documentation p.141)
![Page 9: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/9.jpg)
NIC’s access mechanisms
• The two ways for accessing our network controller’s ‘control’ and ‘status’ registers are explained in Chapter 13 of the Intel Open Source Programmer’s Reference
• This Chapter also includes a detailed list of the names and functional descriptions for the various network interface registers
• All are accessed as 32-bit ‘doubewords’
![Page 10: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/10.jpg)
PCI Configuration Space
PCI Configuration Space Body(48 doublewords – variable format)
64doublewords
PCI Configuration Space Header(16 doublewords – fixed format)
A non-volatile parameter-storage area for each PCI device-function
![Page 11: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/11.jpg)
PCI Configuration Header
StatusRegister
CommandRegister
DeviceID
VendorID
BISTCacheLineSize
Class CodeClass/SubClass/ProgIF
RevisionID
Base Address 0
SubsystemDevice ID
SubsystemVendor ID
CardBus CIS Pointer
reservedcapabilities
pointer Expansion ROM Base Address
MinimumGrant
InterruptPin
reserved
LatencyTimer
HeaderType
Base Address 1
Base Address 2Base Address 3
Base Address 4Base Address 5
InterruptLine
MaximumLatency
31 0 31 0
16 doublewords
Dwords
1 - 0
3 - 2
5 - 4
7 - 6
9 - 8
11 - 10
13 - 12
15 - 14
![Page 12: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/12.jpg)
reserved
Interface to PCI Configuration Space
CONFADD( 0x0CF8)
CONFDAT( 0x0CFC)
31 23 16 15 11 10 8 7 2 0
EN
bus(8-bits)
device(5-bits)
doubleword (6-bits)
function(3-bits) 00
PCI Configuration Space Address Port (32-bits)
PCI Configuration Space Data Port (32-bits)
31 0
Enable Configuration Space Mapping (1=yes, 0=no)
![Page 13: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/13.jpg)
Reading PCI Configuration Data
• Step one: Output the desired longword’s address (bus, device, function, and dword) with bit 31 set to 1 (to enable access) to the Configuration-Space Address-Port
• Step two: Read the designated data from the Configuration-Space Data-Port:# read the PCI Header-Type field (byte 2 of dword 3) for bus=0, device=0, function=0
movl $0x8000000C, %eax # setup address in EAXmovw $0x0CF8, %dx # setup port-number in DX outl %eax, %dx # output address to port
mov $0x0CFC, %dx # setup port-number in DXinl %dx, %eax # input configuration longwordshr $16, %eax # shift word 2 into AL registermovb %al, header_type # store Header Type in variable
![Page 14: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/14.jpg)
Demo Program
• We created a short Linux utility that searches for and reports all of your system’s PCI devices
• It’s named “pciprobe.cpp” on our CS686 website• It uses some C++ macros that expand to Intel
input/output instructions -- which normally are ‘privileged’ instructions that a Linux application-program is not allowed to execute (segfault!)
• Our system administrator (Alex Fedosov) has created a utility (named “iopl3”) that will allow your command-shell to acquire I/O privileges
![Page 15: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/15.jpg)
Example: network interface
• We identify the network interface controller in our classroom PC’s by class-code 0x02
• The subclass-code 0x00 is for ‘ethernet’• We can identify the NIC from its VENDOR and
DEVICE identification-numbers:• VENDOR_ID = 0x8086 (for Intel Corporation)• DEVICE_ID = 0x109A (for 82573L controller)
• You can use the ‘grep’ command to search for these numbers in this header-file:
</usr/src/linux/include/linux/pci_ids.h>
![Page 16: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/16.jpg)
The NIC’s PCI ‘resources’
StatusRegister
CommandRegister
DeviceID0x109A
VendorID0x8086
BISTCacheLineSize
Class CodeClass/SubClass/ProgIF
RevisionID
Base Address 0
SubsystemDevice ID
SubsystemVendor ID
CardBus CIS Pointer
reservedcapabilities
pointer Expansion ROM Base Address
MinimumGrant
InterruptPin
reserved
LatencyTimer
HeaderType
Base Address 1
Base Address 2Base Address 3
Base Address 4Base Address 5
InterruptLine
MaximumLatency
31 0 31 0
16 doublewords
Dwords
1 - 0
3 - 2
5 - 4
7 - 6
9 - 8
11 - 10
13 - 12
15 - 14
![Page 17: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/17.jpg)
Linux PCI helper-functions
#include <linux/pci.h>
struct pci_dev *devp; unsigned int mmio_base; unsigned int mmio_size; void *io;
devp = pci_get_device( VENDOR_ID, DEVICE_ID, NULL );if ( devp == NULL ) return –ENODEV;
mmio_base = pci_resource_start( devp, 0 );mmio_size = pci_resource_len( devp, 0 );io = ioremap_nocache( mmio_base, iomm_size );if ( io == NULL ) return –ENOSPC;
![Page 18: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/18.jpg)
Mechanisms compared
kernel memory-space
NIC i/o-memory
CPU’s ‘virtual’ address-space
io
user memory-space
Each NIC register has its own address in memory (allows one-step access)
addr data
Access to all of the NIC’s registers is muliplexed through a pair of I/O-ports(requires multiple instructions)
CPU’s ‘I/O’ address-space
![Page 19: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/19.jpg)
C language hides complexity
• For the multiplexed i/o-space access…
// Your module uses just one C statement to ‘input’ a NIC register-value:int device_status = inl( ioport + 0x0008 );
// but the C compiler translates this statement into SIX cpu-instructions:
mov ioport, %dxmov $8, %eaxout %eax, %dx
add $4, %dxin %dx, %eaxmov %eax, device_status
![Page 20: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/20.jpg)
Seeing through the C
• For the i/o-memory ‘mapped’ access…
// Your module uses just one C statement to ‘fetch’ a NIC register-value:int device_status = ioread32( io + 0x0008 );
// but the C compiler translates this statement into three cpu-instructions:
mov io, %esimov 8(%esi), %eaxmov %eax, device_status
![Page 21: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/21.jpg)
Course’s continuing theme is…
“Using the computer to study the computer”
![Page 22: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/22.jpg)
TimeStamp Counter
• The x86 processor has a 64-bit register named the TimeStamp Counter (TSC)
• It continuously increments each cpu cycle (so with a 2-GHz processor, it increments two-billion times every second)
• It can be read at any time using a special processor instruction (named RDTSC); its value appears in the EDX:EAX registers
![Page 23: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/23.jpg)
Using CLI and STI
• For doing accurate timing measurements, your module can temporarily disable your CPU’s response to ‘interrupt’ requests
Basic steps for performing an uninterrupted ‘elapsed time’ measurement: step 1: Turn off interrupts (using ‘cli’) step 2: Read the TimeStamp Counter step 3: Perform a NIC register access step 4: Read the TimeStamp Counter step 5: Turn on interrupts (using ‘sti’) step 6: Subtract start-time from finish-time
![Page 24: Accessing the NIC A look at the mechanisms that software can use to interact with our 82573L network interface.](https://reader035.fdocuments.us/reader035/viewer/2022062715/56649d7a5503460f94a5df0a/html5/thumbnails/24.jpg)
In-class exercise
• Look at our ‘timing.c’ demo module – for an ‘inline’ assembly language example using the ‘rdtsc’, ‘cli’ and ‘sti’ instructions
• Add some extra code of your own, and do an additional timing measurement, so you can compare the execution-times for the NIC’s two register-access mechanisms
• How much faster is memory-mapped I/O?