From 'Netern' to Intern: Student Developed Applications Through Internet2

Andrew Lake, Albion College

ARL10@albion.edu

Internet2 Fall Member Meeting 2005

Philadelphia, PA

2

Outline

• Neternship• Internet2 Detective• Tests• Interesting issues encountered

• BlackSquirrel• What it is• How it works

• BRUW• Overview of technology• Future development

3

Neternship

• An internship over the internet• Distributed learning environment• Gives students chance to learn

about and use Internet2 services• Great chance for smaller colleges to

get involved

4

Internet2 Detective

• Program that detects Abilene connection and other capabilities

• Originally stand-alone application

• Java Applet developed through Neternship

5

Java Applets

• Java program embedded in a web page• Runs on user's computer• Must give “permission” for certain tasks• Signing identifies creator and informs

user of what to expect

6

Abilene Test

• Abilene test server sits on backbone

• Detective says “hello” and waits for server to say “hello” back

• Simple and quick

7

Bandwidth Test

• Installs iperf in temporary directory

• Runs iperf in UDP mode

• Parses results from test

• Compares to selected bandwidth

8

Multicast Test

• Uses Java's built-in classes to join multicast group

• If can join, multicast enabled; if not, multicast not enabled

9

IPv6 Test

• Runs iperf from temporary directory (see Bandwidth Test)

• Contacts iperf server using IPv6 address and -V option

• If iperf fails, attempts to run ping6

10

Problem: Firewalls

• Detective uses many non-standard ports• Have to worry about source and

destination ports in each direction• Solution: Directly specify all ports and

document numbers

11

Problem: Network Interfaces

• If multiple network connections, which one should be tested?

• Default connection hard to predict• Solution: Allow user to choose from

list of available interfaces

12

Problem: IPv6 Fragmentation

• Presented by Bill Cerveny at Summer 2005 Joint Techs

• IPv6 Test unable to reach iperf server

• Packets being fragmented causing an extension header to be used and packet to be misidentified by filter

• Solution: Set datagram size to 1450 using iperf's -l option

13

Detective Widget

• Small program that runs in Mac OS X's dashboard

• Not a full detective, just runs Abilene Test

• Fun toy for Mac users, but Konfabulator version coming soon

14

BlackSquirrel

15

BlackSquirrel Background

• BlackSquirrel started as a DV Guide project

• Many digital video stream distributed across Abilene

• Content accessible but how do you find it?

• Eventually, the same questions arose but for archived digital video

• ...and again for video conferencing

• ...and again for non-video data (telescope data, microscope data, images, audio, etc)

• ...and again for bringing all these types of information together

• Realize need for a standard way to describe and join multiple types of distributed content

16

BlackSquirrel Guidelines

1. Primarily targeted toward developers on multiple platforms

2. Creates “back-end” for developer allowing more focus on UI

3. Uses open technologies

4. Minimal install and effort to run

5. Flexible support for many types of descriptions

6. Scalable

7. Simple technical model

17

Technical Model

• Describing the data• Use XML to descibe data

• Make available in feed form (RSS)

• Joining the data• Use hierarchal structure of XML to develop table model

for relational database

• Sharing the data• “Confederacy of parsers” - many parsers retrieve certain

feeds then store in an aggregate feed to be shared

• Using the data• Once in database, developer creates interface to use the

data

18

BlackSquirrel Model

• In simpler terms, a black squirrel retrieves acorns from a squirrel feed then takes the acorns back to the drey where a baby squirrel consumes them. A black squirrel can also take the acorns to another feed to share with other black squirrels.

• Translating back to the more technical version:Acorns = XML tags that describe dataFeed = The XML file that contains all the tagsAnother Feed = Aggregate XML feedDrey = the database that stores the data retrievedBlack Squirrel = the program that retrieves and stores the dataBaby Squirrel = The UI that presents the data

19

Acorns & Feeds

• An acorn refers to individual pieces of description data

• A feed is a text file full of acorns, i.e. description data

• An aggregate feed contains all the acorns from one feed and combines them with acorns from other feeds

20

BlackSquirrel & The Drey

• A BlackSquirrel takes acorns from a feed and puts them in a drey

• The drey is a database where the information is kept for later use

• The drey is a database called Derby developed by the Apache Foundation

21

Baby Squirrels

• A baby squirrel is the consumer of acorns brought back to the drey. This means it is the program that actually uses the data.

• A baby squirrel may be a web page or another program

• Any language that can support Apache Derby will work. This list includes PHP, Perl, ASP, JSP, and Java

• Allows developer to focus more on the UI and less on developing a system to manage data

22

Video Diagram

23

Telescope Diagram

24

BlackSquirrel Diagram

25

Podcasting

• “Podcasting” is the automatic download of an audio file attached to an RSS item

• BlackSquirrel podcasting works with any type of file.

• BlackSquirrel supports this by allowing a user to specify a special BLOB field in the database associated with a URL property or element

• When specified, automatically downloads resource and stores in database

• Useful because content can sit directly next to description

• Database allows for quick access

• Easy to extract

26

BlackSquirrel Future

• Expand Interface• Add user preferences• Allow selection of database location• Export BLOB data• Auto-Start Derby

• Expand support for downloading protocols (FTP, HTTPS, etc)

• Add BitTorrent functionality

• Create a SOAP server to update database in real time

• Add security and authentication features

27

BRUW – Got Bandwidth?

28

BRUW Overview

• BRUW stands for Bandwidth Reservation for User Work

• Started by Soo-yeon Hwang in 2004• Users reserve bandwidth through a web

interface (front-end)• MPLS Tunnels periodically built to

“reserve” bandwidth requested for that time (back-end)

• Front-end and back-end share database

29

BRUW

30

Authentication

• Shibboleth• Create new BRUW account• Administrator authorizes the account• User sent activation key to activate

account

31

Making a Reservation

32

Back-End

• Runs in cron• Builds tunnels just before every hour• Removes tunnels just after every hour

• Discovers edge routers• Makes reservation between edge routers

in each direction

33

Edge Discovery

• In order to make reservation, BRUW must locate the edge nodes

• Edge nodes are the first and last router that a transmission will cross on Abilene

• Ingress Node is router used to enter Abilene

• Egress Node is the router used to exit Abilene

34

Reservation Building

• Once edge routers discovered, bandwidth can be reserved between nodes

• Communicate with routers using JUNOScript and XML to change configurations and build/remove necessary MPLS tunnels on Juniper Routers

• Tunnels built in both directions

35

Back-End Diagram

36

Test Results

• BRUW has successfully built tunnels between Chicago and Los Angeles

• Work still needs to be done to determine the exact configurations necessary to run BRUW

37

Future of BRUW

• Merge codebase with OSCARS• Interdomain reservations• Further Shibboleth integration• Get closer to solving last mile problem• Look into cooperation with other similar

projects

38

What I learned...

• Worked with IPv6 and Multicast• Use of network measurement tools• SAX• Apache Derby• Java Server Pages• Routing Protocols• MPLS• Lots more!!!

39

http://vbvod.internet2.edu/internship