Börje Josefsson<bj@sunet.se>

GRID Connections inGRID Connections in

GigaSunet:GigaSunet: Nationwide 10 Gbit/s network

• Core network (22 cities):• 5.270 km of lambdas.• 27 inter-city segments with

10 Gbit/sec.• 49 routers (Cisco 124xx).

• Access network (2,5 Gbit/s):• 25 dark fiber rings.• 3 segments of 2,5 Gbit/sec.

lambda.• 1 STM-1 connection.• 70 routers (Cisco 10720).

Research networks in Europe

Local peering

For futureideas…

IPv6-network(via UTI)

National researchnetwork (via UTI)

GRIDGRID

Production network(connected to University LAN).

Suggested access scenario

4 * Gigabit Ethernet 8 * Fast Ethernet

Uplink 2,5 Gbit/sec (redundant)

GRID-connection at the edge

• Fiber directly to SUNETs access router.– There are two access routers at each university.– 1 Gbit/sec. [Gigabit Ethernet].– IP version 4 supported today at this speed.– Avoid the local LAN-equipment if possible!– As ”straight” to Your hosts as possible !

• Avoid (cheap) switches etc.!

But it is still slow?!?

Path RTT Max speed

Luleå - Umeå 3 ms 87,3 Mbit/sec

Luleå - Lund 32 ms 8,2 Mbit/sec

Luleå - Geneva 70 ms 3,7 Mbit/sec

Luleå - Boston 130 ms 2,0 Mbit/sec

Luleå - San Diego 205 ms 1,3 Mbit/sec

For a single session, GigaSunet can be felt1/ as slow! Speed of light and buffer sizes limits the speed.

1/ Unless You have tweaked Your computer and LAN.

Theoretical maximum speed with 32 Kbytes buffer.

Bwbest = buffer / RTT

”Fast-and-far”

• You have to think. Twice…

• Tweak Your OS parameters.

• Sweden is approx. 18 ms ”long”.– 1 Gbit/sec * 36 ms RTT 4,5 megabyte buffer in

the end hosts (usually it is 32 kilobyte…).

• This has nothing to do with the network itself, this is just how computer communication works.– It doesn’t help to have a “private” network!

Test results

dino# ttcp -s -t -f m -l 61440 -n 20345 wilma.sunet.se

ttcp-t: buflen=61440, nbuf=20345, align=16384/0, port=5001 tcp -> wilma.sunet.se

ttcp-t: socket

ttcp-t: connect

ttcp-t: 1249996800 bytes in 10.35 real seconds = 966181102 bits/sec +++

ttcp-t: 20345 I/O calls, msec/call = 0.52, calls/sec = 1966.33

ttcp-t: 0.0user 1.6sys 0:10real 16% 0i+0d 0maxrss 0+341060pf 41346+4csw

Luleå-Stockholm (approx. 1.000 km).Hosts connected to Gigabit Ethernet.

966,2 Mbit/sec.966,2 Mbit/sec.This is ”real” traffic, not counting the framing and packet overhead.

Can You spot the difference?

Answer:a) 0,001% more cable in ”B” (compared to total distance) .b) More (expensive) equipment in ”B”.c) 75-80% less performance in ”B”.

Accessrouter

Coreswitch

Data centerswitch

Host

Accessrouter

Coreswitch

Data centerswitch

Host

Alt. A Alt. B

Conclusion

• SUNET and GigaSunet are ready to handle this. Now.

• The ”problems” are local at each university.

• We can help You with tuning guidelines and act as testing partners.– SUNET test hosts are connected directly to the

core network, without LAN equipment in between. That helps us isolate “local” problems.

What You should do next..

• Contact Your local network group.– Get a fiber connection from Your site to one of

the SUNET access routers.– Have them1 configure the router for Your

connection.

• Connect the GRID equipment, as ”clean” as possible.

• Tune Your system!

1/ Done locally at each university, except Uppsala.

Questions?

Börje Josefsson<bj@sunet.se>