1

Benchmarking Methodology WG (bmwg)70th IETF – Vancouver, Canada

• Thursday, December 6, 2007, 9:00-11:30 (Oak)

• Chairs: – Al Morton (acmorton@att.com)

• If you are not subscribed to the BMWG mailing list and would like to be, put a BIG asterisk (*) by your legibly-printed e-mail address when you sign the blue attendance sheet, or go to https://www1.ietf.org/mailman/listinfo/bmwg 

2

BMWG Agenda (Any Bashing needed?)

• Note-Taker(s), Jabber, IPR (Yellow Sheet), Blue Sheets• WG Status (Chair, 10 min)

• IGP Dataplane Convergence Design Team. (Poretsky, 20+ min)

• Accelerated Stress Testing. (Poretsky, 10 min)

• Protection Benchmarking (Poretsky, 20 min)

• Milestones Status (Chair, 5 min)

• Proposed Work (Chair, 60 min, nominally 15 min each)– LDP Convergence (Karthik)– Multicast VPN Scalability (Dry) – MPLS Benchmarking Methodology (Akhter)– SIP Performance Benchmarking (Gurbani,Davids,Poretsky)

– (not appearing this time):MPLS-TE (Vapiwala)

Bi-Directional Forwarding Detection (Karthik)

Wireless LAN Switches and LAN Meshes (Perser, Alexander)

• AOB (???, 5 min)

3

BMWG Activity• AD/IESG Review

• <draft-ietf-bmwg-igp-dataplane-conv-term-14.txt> Rev I-Ds• <draft-ietf-bmwg-igp-dataplane-conv-meth-14.txt> • <draft-ietf-bmwg-igp-dataplane-conv-app-014.txt>• <draft-ietf-bmwg-ipv6-meth-04.txt> AD queue

• WG Last Call• <draft-ietf-bmwg-acc-bench-term-12.txt> Revised on LC comments • <draft-ietf-bmwg-acc-bench-meth-08.txt> Revised on LC

comments

• I-Ds• draft-ietf-bmwg-ipsec-meth-02.txt • draft-ietf-bmwg-ipsec-term-09.txt • draft-ietf-bmwg-protection-term-03.txt Need Readers• draft-ietf-bmwg-protection-meth-02.txt Need Readers• draft-ietf-bmwg-dsmmeth-02.txt Expired

4

BMWG Activity• Work Proposals.

– draft-alexander-bmwg-wlan-switch-term-00.txt <=same– draft-alexander-bmwg-wlan-switch-meth-00.txt <=same– draft-vapiwalla-bmwg-TE-convergence-motivation-01.txt <= Revised– draft-poretsky-sip-bench-term-03.txt <=Revised– draft-poretsky-bmwg-sip-bench-meth-02.txt <=Revised– draft-sdry-bmwg-mvpnscale-03.txt <=Revised– draft-akhter-bmwg-mpls-meth-03.txt <=Revised– draft-eriksson-ldp-convergence-04.txt <=Expired– draft-karthik-bmwg-ldp-convergence-meth-01.txt <=Revised– draft-salahuddin-bmwg-bfd-motivation-00.txt <= Expired

• Expected I-Ds– <draft-ietf-bmwg-bgpcmeth-00.txt>

5

BMWG Activity• Expired BMWG I-Ds

– <draft-ietf-bmwg-acc-bench-meth-ebgp-00.txt>,

– <draft-ietf-bmwg-acc-bench-meth-opsec-00.txt>,

– <draft-ietf-bmwg-benchres-method-00.txt> Pending term

• RFC Editor – none

• Charter Update– Meet the REVISED Milestones, start new work.

• Supplementary BMWG Page– See http://home.comcast.net/~acmacm/BMWG/

• New RFC:– none

• RFC 2889 Appendix Errata?

6

Standard “Paragraph” (intro/security) Benchmarking activities as described in this memo are limited to technology characterization using controlled stimuli in a laboratory environment, with dedicated address space and the constraints specified in the sections above.

The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network.

Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the DUT/SUT.

Special capabilities SHOULD NOT exist in the DUT/SUT specifically for benchmarking purposes. Any implications for network security arising from the DUT/SUT SHOULD be identical in the lab and in production networks.

7

Current Milestones

• Dec 2007 IPsec Device Benchmarking Terminology to AD Review• Dec 2007 IPsec Device Benchmarking Methodology to AD Review• Dec 2007 Router Accelerated Test Terminology I-D to AD Review• Dec 2007 Router Accelerated Test Method. and Considerations I-Ds

to AD Review

• DONE IPv6 Benchmarking Methodology to AD Review• Feb 2008 Terminology For Protection Benchmarking to AD Review• Feb 2007 Methodology For Protection Benchmarking to AD Review• Jul 2009 Basic BGP Convergence Benchmarking Methodology to AD Review.

• Dec 2007 Net Traffic Control Benchmarking Methodology to AD Review.

• Dec 2007 Router Accelerated Test Method for EBGP to AD Review• Dec 2007 Router Accelerated Test Method for Operational Security to

AD Review

8

Work Proposal Summary MatrixWork Area >

Criteria \/SIP

Mcast VPN

Scalability

LDP Converg

MPLS BFDWLAN switch

RSVP-TE

Proposal Y Y Y Y Y Y YIn Scope of Charter? (acm)

Y Y Y Y Y No Overlap w/802.11T Y

Draft(s) 1+1 1+ 1+1 1+Draft gives

motivat.1+1

Draft gives

motivat.

Sig. Support at meetings

Yes, but also some

objections

Sig. Support on List

Interest when first proposed

Recent comments

Lots of Recent

comments

Recent comments

Testing to support Dev?