27 May, 2010 1

AN Surface Detector Communication TeamAN Surface Detector Communication Team

RT2010 - Lisboa

Hard Real-Time Wireless Communication Hard Real-Time Wireless Communication in thein the

Northern Pierre Auger ObservatoryNorthern Pierre Auger Observatory

Roger Kieckhafer

(rmkieckh@mtu.edu)

for the Pierre Auger Collaboration

27 May, 2010 RT2010 - Lisboa 2

Auger North and SouthAuger North and South

From the beginning: two sites were planned

Northern and Southern Hemispheres

Auger North:

Planning Stages

Lamar, Colorado USA

Auger South:

Fully Operational

Malargue, Argentina

27 May, 2010 RT2010 - Lisboa 3Surface Detector (SD) Array Surface Detector (SD) Array

TimestampsTimestamps+ other data+ other data

Shower of secondary particles

H +

Flourescence Detector (FD)

Telescopes

ObservatoryFunctional

Overview

27 May, 2010 RT2010 - Lisboa 4

SD Wireless – Real-Time SequenceSD Wireless – Real-Time Sequence

SD Station(in the field)

CDAS(Observatory

Campus)Record andfilter

timestamps T2 Trigger Message(Timestamps of promising events) Correlate

TimestampsAcross Stations

T3 Trigger Message

(Req. data on correlated timestamps)Access Dataon requestedtimestamps T3 Response Message Stream

(Return data on req. timestamps) Conductfurther analysis

27 May, 2010 RT2010 - Lisboa 5

Auger North vs South. Auger North vs South.

Auger North puts greater demands on the wireless net

Auger North will be much larger

Auger North will have higher data rates

Auger North will have roughly the same real-time deadlines

27 May, 2010 RT2010 - Lisboa 6

Auger North SiteAuger North Site

AugerNorth

Footprint

AugerSouth

Footprint

27 May, 2010 RT2010 - Lisboa 7

Auger South SD CommunicationAuger South SD Communication

Wide area wireless net

Microwave towers form a

communication backbone

4 Collector Towers

on periphery of array

1 Destination Tower

at Malargue campus

1-hop station-to-tower link for each station

Remarkably flat topography in the interior

Sizable hills on the periphery of the array

Easy to give each station a clear line-of-sight to a tower

27 May, 2010 RT2010 - Lisboa 8

Auger North Communication ProblemAuger North Communication Problem

Array is 7 x Bigger than South

Need > 30 towers @ 45 mto stay within radio range

Terrain is Rougher than South

No convenient peripheral hills

Many small internal hills & ridges

Can not get reliable line-of-sight over that terrain

Bottom line:

1-hop Station-to-Tower comm.does not work

>10% of stations incommunicado Springfield

Lamar

27 May, 2010 RT2010 - Lisboa 9

Only Viable Option: Peer-to-Peer WSNOnly Viable Option: Peer-to-Peer WSN

Auger North uses a new paradigm called “WAHREN”

WA Wireless Architecture

Infrastructure Nodes (INs) = the dominant feature

Non-Infrastructure Nodes (NINs) = optional “guest” nodes

HR Hard Real-time

Time-Bounded subject to hard real-time deadlines

Deterministic not subject to stochastic-timing

EN Embedded Networks

Fault-Tolerant Fully Redundant Comp. & Comm.

Fully Distributed No leaders, coordinators, masters

27 May, 2010 RT2010 - Lisboa 10

WAHREN Interconnection TopologyWAHREN Interconnection Topology

Uses 2nd Order Power chains (“Braided Chain”)

Graph Topology:

Start with a basic chain – nearest neighbor comm only

Extend range to reach second-nearest neighbors

Useful Physical Realization = 2D Triangular Mesh

1 2 3 4 5 6 7 8 9 10

1

2

3

4

5

6

7

8

9

10

27 May, 2010 RT2010 - Lisboa 11

Our Target Infrastructures are not always straight lines

Gentle curves

are no problem

But what aboutsharp corners?

Existence of Mobiusfold is transparentto the topology

7

8

9

10

7’

8’

9’

10’

Turning CornersTurning Corners

1

2

3

4

5

6

Mobius FoldMust be Prevented

by Protocol

27 May, 2010 RT2010 - Lisboa 12

Hybrid TDMA/CSMA MAC ProtocolHybrid TDMA/CSMA MAC Protocol

Hybrid Window Combines TDMA & CSMA slots

Enough TDMA slots for all INs within interference range

Enough CSMA slots for expected/desired number of NINs

Auger North-Specific Comm Window

8 TDMA slots – for neighboring Infrastructure Nodes

Based on predicted interference range, and

Alternating use of 2 RF channels

1 CSMA slot – expecting few non-infrastructure nodes

NIN Trans

CSMA

Window

IN Transmissions

TDMA

27 May, 2010 RT2010 - Lisboa 13

Systolic Broadcast Scheduling ProtocolSystolic Broadcast Scheduling Protocol

Unidirectional Single-Source Broadcast:

Window 0: Node 0 originates a message

Window w: Node w forwards node 0’s message

Redundancy: Node Red = Path Red = Time Red = 2

Unidirectional Multi-Source Broadcast

Window 0: All nodes originate a message

Window w: Node k forwards node (k-w)’s message

0Win-2

Win-1

Win-0 Win-4

Win-5Win-31

2 64

3 5

27 May, 2010 RT2010 - Lisboa 14

Adapting WAHREN to Auger TopologyAdapting WAHREN to Auger Topology Array is partitioned into “service areas”

Each served by one Concentrator Station

Each Service Area is partitioned into “sectors”

Sectors can be triangular, rectangular, or quite amorphous

Service Area Sector

27 May, 2010 RT2010 - Lisboa 15

Comm Sequence in One SectorComm Sequence in One Sector

Auger North SD is a 2 mi square array tilted 45 degrees

Any adjacent pair of E-W rows or

N-S columns naturally forms a

2nd order power chain

Can easily be organized into

Backbones & Side Chains

Linked at Mobius Folds

Side C

hain

s

Backbone

Concentrator Station

Fiber to C

ampu

s

PCHost

27 May, 2010 RT2010 - Lisboa 16

System ThroughputSystem Throughput Can Meet all required deadlines:

With up-to 128 stations / sector

4,400 stations 35 sectors

8 sectors/concent. 5 concents.

IF they are all centrally located.

Current Proposal is to put

concentrators at the 5 FDs

Power, fiber & bldgs already exist

But, some FDs are on Periphery

They serve fewer sectors

May need 6-7 Concentrators

5 at FDs + 1-2 standalone

That is still much cheaper than 30 tall towers

27 May, 2010 RT2010 - Lisboa 17

Being built Near Lamar, CO

Goal: to test the riskier new technologies

Station Configuration

10 fully functional stations

10 comms-only stations

1 concentrator

Array Layout

1 backbone (A-J)

1 side chain (K-O)

6 off-grid “infill” stations (L-T)

Research & Development Array (RDA)Research & Development Array (RDA)

LamarLamar

27 May, 2010 RT2010 - Lisboa 18

SummarySummary

WAHREN paradigm is well suited to Auger North:

Power Chain Architecture path & node redundancy

Hybrid TDMA/CSMA MAC real-time stability

Systolic Broadcast Sched time redundancy & efficiency

Deadlines and throughput can be satisfied:

With a handful of concentrators

Much more cheaply than tall towers

Some items we had to skip today:

Formal validation & verification of the protocol – done

Detailed Markov Reliability modeling of a sector – done

Multimodal Performability modeling of a sector – in progress

Station radio hardware/software development – in progress

27 May, 2010 19

AN Surface Detector Communication TeamAN Surface Detector Communication Team

RT2010 - Lisboa

Questions?Questions?