A Difference Resolution Approach to Compressing Access...

14
1 A Difference Resolution Approach to Compressing Access Control Lists James Daly Alex X. Liu Eric Torng Abstract—Access Control Lists (ACLs) are the core of many networking and security devices. As new threats and vulnerabil- ities emerge, ACLs on routers and firewalls are getting larger. Therefore, compressing ACLs is an important problem. In this paper, we propose a new approach, called Diplomat, to ACL compression. The key idea is to transform higher dimensional target patterns into lower dimensional patterns by dividing the original pattern into a series of hyperplanes and then resolving differences between two adjacent hyperplanes by adding rules that specify the differences. This approach is fundamentally different from prior ACL compression algorithms and is shown to be very effective. We implemented Diplomat and conducted side- by-side comparison with the prior Firewall Compressor, TCAM Razor and ACL Compressor algorithms on real life classifiers. Our experimental results show that Diplomat outperforms all of them on most of our real-life classifiers, often by a considerable margin, particularly as classifier size and complexity increases. In particular, on our largest ACLs, Diplomat has an average improvement ratio of 34.9% over Firewall Compressor on range- ACLs, of 14.1% over TCAM Razor on prefix-ACLs, and 8.9% over ACL Compressor on mixed-ACLs. Index Terms—Hardware-based Packet Classification, Ternary Content Addressable Memory (TCAM), Range Encoding. I. I NTRODUCTION A. Background and Motivation Access Control Lists (ACLs) are the core of many net- working and security devices, such as routers and firewalls, which perform services such as packet filtering, virtual private networks (VPNs), network address translation (NAT), qual- ity of service (QoS), load balancing, traffic accounting and monitoring, differentiated services (Diffserv), etc. A packet can be viewed as a d-tuple over d fields with finite, discrete domains. For IP packets, d is typically 5 and the relevant fields are source IP address, destination IP address, source port number, destination port number, and protocol type, where the domains of these fields are [0, 2 32 1], [0, 2 32 1], [0, 2 16 1], [0, 2 16 1], and [0, 2 8 1], respectively. An ACL is specified as a sequence (i.e., ordered list) of predefined rules. Each rule is specified in the form predicate 〉→〈decision. The predicate over packet fields F 1 ,F 2 , ··· ,F d is typically specified as (F 1 S 1 ) (F 2 S 2 ) ∧··· , (F d S d ). If each S i is specified as a range, we call the ACL a range- ACL. If each S i is specified as a prefix, we call the ACL a James Daly, Alex X. Liu, and Eric Torng are with the Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, 48824. E-mail: {dalyjame, alexliu, torng}@cse.msu.edu. Corresponding author: Alex X. Liu The preliminary version of this paper titled “A Difference Resolution Approach to Compressing Access Control Lists” was published in the pro- ceedings of the 32rd Annual IEEE International Conference on Computer Communications (INFOCOM), pages 2040-2048, Turin, Italy, April 2013. prefix-ACL. If some S i are specified as prefixes and others as ranges, then we call the ACL a mixed-ACL. The decisionof a rule can be accept, or discard, or a combination of these decisions with other options such as a logging option. An ACL is essentially a function from the space of legal packets to a set of possible decisions. When a packet arrives at a router, the router extracts the relevant field values to form a search key and searches the ACL to find the first rule that the search key matches. The decision of this rule determines the appropriate action to take upon the packet. The rules in an ACL often conflict, which means that a packet may match multiple rules and these rules may have different decisions. The way that ACLs resolve conflicts is to follow the first match principle: for any packet, the decision for the packet is the decision of the first packet that the packet matches. In this paper, we focus on the ACL Compression Prob- lem: given a range-ACL, prefix-ACL or mixed-ACL L, find an equivalent range-ACL, prefix-ACL or mixed-ACL L so that the number of rules in L , denoted |L |, is as small as possible. We consider three different variations of the ACL Compression Problem because of varying application constraints. For applications where all fields must be prefixes, we use the prefix-ACL variant. For applications where all fields can be ranges, we use the range-ACL variant which allows for more compression. For applications where only some fields are constrained to be prefixes, we use the mixed- ACL variant. This problem has many motivations. First, as new threats and vulnerabilities emerge, ACLs on routers and firewalls are getting larger. For example, because the ACLs used for Quality of Services on routers are often automatically generated, they tend to be huge. Second, in the Open Flow architecture, which is gaining more popularity and real deployment, networking devices typically need to handle a large number of ACL- like rules; thus, compressing such ACL-like rules is very helpful for managing and optimizing such devices. A number of network system management work, such as Yu et al.’s DIFANE work [19] and Sung et al.’s work [16], have used our prior ACL compression tool called Firewall Compressor [11], [12] to reduce system complexity. Third, some networking and security devices have strict limits on the number of rules that can be stored. For example, NetScreen-100 only allows ACLs with 733 rules. Fourth, fewer ACL rules often leads to higher system performance. As ACLs are used to examine every incoming and outgoing packet, such performance is critical for network throughput.

Transcript of A Difference Resolution Approach to Compressing Access...

  • 1

    A Difference Resolution Approach to

    Compressing Access Control ListsJames Daly Alex X. Liu Eric Torng

    Abstract—Access Control Lists (ACLs) are the core of manynetworking and security devices. As new threats and vulnerabil-ities emerge, ACLs on routers and firewalls are getting larger.Therefore, compressing ACLs is an important problem. In thispaper, we propose a new approach, called Diplomat, to ACLcompression. The key idea is to transform higher dimensionaltarget patterns into lower dimensional patterns by dividing theoriginal pattern into a series of hyperplanes and then resolvingdifferences between two adjacent hyperplanes by adding rulesthat specify the differences. This approach is fundamentallydifferent from prior ACL compression algorithms and is shown tobe very effective. We implemented Diplomat and conducted side-by-side comparison with the prior Firewall Compressor, TCAMRazor and ACL Compressor algorithms on real life classifiers.Our experimental results show that Diplomat outperforms all ofthem on most of our real-life classifiers, often by a considerablemargin, particularly as classifier size and complexity increases.In particular, on our largest ACLs, Diplomat has an averageimprovement ratio of 34.9% over Firewall Compressor on range-ACLs, of 14.1% over TCAM Razor on prefix-ACLs, and 8.9%over ACL Compressor on mixed-ACLs.

    Index Terms—Hardware-based Packet Classification, TernaryContent Addressable Memory (TCAM), Range Encoding.

    I. INTRODUCTION

    A. Background and Motivation

    Access Control Lists (ACLs) are the core of many net-

    working and security devices, such as routers and firewalls,

    which perform services such as packet filtering, virtual private

    networks (VPNs), network address translation (NAT), qual-

    ity of service (QoS), load balancing, traffic accounting and

    monitoring, differentiated services (Diffserv), etc. A packet

    can be viewed as a d-tuple over d fields with finite, discretedomains. For IP packets, d is typically 5 and the relevantfields are source IP address, destination IP address, source port

    number, destination port number, and protocol type, where

    the domains of these fields are [0, 232 − 1], [0, 232 − 1],[0, 216−1], [0, 216−1], and [0, 28−1], respectively. An ACL isspecified as a sequence (i.e., ordered list) of predefined rules.

    Each rule is specified in the form 〈predicate〉 → 〈decision〉.The 〈predicate〉 over packet fields F1, F2, · · · , Fd is typicallyspecified as (F1 ∈ S1) ∧ (F2 ∈ S2) ∧ · · · ,∧(Fd ∈ Sd). Ifeach Si is specified as a range, we call the ACL a range-ACL. If each Si is specified as a prefix, we call the ACL a

    James Daly, Alex X. Liu, and Eric Torng are with the Department ofComputer Science and Engineering, Michigan State University, East Lansing,MI, 48824. E-mail: {dalyjame, alexliu, torng}@cse.msu.edu.

    Corresponding author: Alex X. LiuThe preliminary version of this paper titled “A Difference Resolution

    Approach to Compressing Access Control Lists” was published in the pro-ceedings of the 32rd Annual IEEE International Conference on ComputerCommunications (INFOCOM), pages 2040-2048, Turin, Italy, April 2013.

    prefix-ACL. If some Si are specified as prefixes and others asranges, then we call the ACL a mixed-ACL. The 〈decision〉of a rule can be accept, or discard, or a combination of these

    decisions with other options such as a logging option. An ACL

    is essentially a function from the space of legal packets to a

    set of possible decisions. When a packet arrives at a router, the

    router extracts the relevant field values to form a search key

    and searches the ACL to find the first rule that the search key

    matches. The decision of this rule determines the appropriate

    action to take upon the packet. The rules in an ACL often

    conflict, which means that a packet may match multiple rules

    and these rules may have different decisions. The way that

    ACLs resolve conflicts is to follow the first match principle:

    for any packet, the decision for the packet is the decision of

    the first packet that the packet matches.

    In this paper, we focus on the ACL Compression Prob-

    lem: given a range-ACL, prefix-ACL or mixed-ACL L, findan equivalent range-ACL, prefix-ACL or mixed-ACL L′ sothat the number of rules in L′, denoted |L′|, is as smallas possible. We consider three different variations of the

    ACL Compression Problem because of varying application

    constraints. For applications where all fields must be prefixes,

    we use the prefix-ACL variant. For applications where all

    fields can be ranges, we use the range-ACL variant which

    allows for more compression. For applications where only

    some fields are constrained to be prefixes, we use the mixed-

    ACL variant.

    This problem has many motivations. First, as new threats

    and vulnerabilities emerge, ACLs on routers and firewalls are

    getting larger. For example, because the ACLs used for Quality

    of Services on routers are often automatically generated, they

    tend to be huge. Second, in the Open Flow architecture, which

    is gaining more popularity and real deployment, networking

    devices typically need to handle a large number of ACL-

    like rules; thus, compressing such ACL-like rules is very

    helpful for managing and optimizing such devices. A number

    of network system management work, such as Yu et al.’s

    DIFANE work [19] and Sung et al.’s work [16], have used our

    prior ACL compression tool called Firewall Compressor [11],

    [12] to reduce system complexity. Third, some networking and

    security devices have strict limits on the number of rules that

    can be stored. For example, NetScreen-100 only allows ACLs

    with 733 rules. Fourth, fewer ACL rules often leads to higher

    system performance. As ACLs are used to examine every

    incoming and outgoing packet, such performance is critical

    for network throughput.

  • 2

    B. Summary and Limitations of Prior Art

    Optimal polynomial time algorithms have been developed

    for 1-dimensional range-ACL compression (Top Coder Chal-

    lenge in 2003, [2], [11]) and prefix-ACL compression [4],

    [17]. Applegate et al. proved that the 2-dimensional range-

    ACL compression problem is NP-hard [2]. The complexity

    of 2-dimensional prefix-ACL compression is still unknown,

    but prefix-ACL compression with an arbitrary number of

    dimensions is NP-hard [5]. The only approximation algo-

    rithm with a non-trivial approximation bound is one given

    by Applegate et al. which has an approximation ratio of

    O(min(n1/3, OPT 1/2)) for 2-dimensional range-ACL com-pression and O(w2 min(n1/3, OPT 1/2)) for 2-dimensionalprefix-ACL compression, where n is the number of rules inthe input rule list, w is the word size in bits, and OPTis the number of rules in the optimal rule list [2]. Their

    bounds are tight. Their method requires dividing the input

    into several subproblems to deal with the special forbidden

    rectangle pattern described in Section V. The number of

    subproblems required may be O(Opt), each of which mayrequire O(Opt1/2) rules.

    Very few prior algorithms work for more than two dimen-

    sions. Liu et al.’s Firewall Compressor algorithm [11] handles

    d-dimensional range-ACLs by converting the rule list into acanonical firewall decision diagram and applying an optimal

    weighted algorithm to each of the one-dimensional patterns

    formed between two layers of the diagram. In their algorithm,

    the “decision” used is actually the rule list that is produced by

    lower levels, which is replicated for each of the corresponding

    rules in the higher dimension. However, Firewall Compressor

    only considers whether or not the two lower rule lists are the

    same. If two lists are very similar but not the same, both lists

    are replicated in their entirety. Their later work, TCAM Razor,

    works by the same principle but uses an optimal prefix-ACL

    algorithm rather than a range-ACL algorithm [15]. Finally,

    they combined both works into ACL Compressor, which can

    use either prefix or range compressors on different fields [13].

    C. Proposed Approach

    We present a new approach, called Diplomat, to ACL

    compression. The key idea is to transform higher dimensional

    target patterns into lower dimensional patterns by dividing the

    original pattern into a series of hyperplanes. It then selects two

    adjacent planes and resolves their differences by adding rules

    to specify where the two planes differ. After resolution, the

    two planes are compatible and can be merged into a single

    plane. For example, in Figure 2, a single white rule in the

    middle row would allow it to be merged with the top row.

    Diplomat repeats this process until all of the planes have been

    merged together into a single plane. Diplomat then repeats the

    process on the reduced pattern. After recursive applications,

    the pattern is reduced to a 1-dimensional pattern for which

    optimal algorithms already exist. In terms of construction,

    the rules generated by each resolution step and the final

    compression step are joined together into one rule list with

    the rules from earlier steps appearing before the rules from

    later steps. Thus, any section specified by an earlier resolution

    cannot be undone by a later resolution. This is what allows

    Diplomat to consider the result of each resolution to be one

    homogeneous region. We illustrate this process for a two-

    dimensional range-ACL in Figure 1. We call this algorithm

    Diplomat because it compresses a rule list by repeatedly

    “resolving differences” between adjacent planes.

    Fig. 1. Overview of Diplomat in two dimensions

    Figure 2 shows that for an example input ACL, Firewall

    Compressor produces more rules than Diplomat.

    Firewall Compressor Diplomat

    0

    1

    2

    0 1 2 3 4 5

    x [1, 1] !y [4, 4] !"#$%&

    x [1, 1] !"! [0, 5] !"#$%&

    x [0, 2] !"! [1, 1] !"#$%&

    x [0, 2] !"! [4, 4] !"#$%&

    x [0, 2] !"! [0, 5] !"#$%&

    x [1, 1] !"! [0, 3] !"#$%&

    x [0, 2] !"! [1, 1] !"#$%&

    x [0, 2] !"! [4, 4] !"#$%&

    x [0, 2] !"! [0, 5] !"#$%&

    Fig. 2. A two-dimensional pattern divided into three rowsNote that Diplomat is designed to be run offline so that

    network managers do not need to interact with the compressed

    ACL. Rather, the manager can interact with the rule list in

    a comfortable and understandable form. This can then be

    input into Diplomat, which will create the compressed rule

    list actually used by the network device.

    D. Key Contributions

    The key contribution of this paper is our new ACL compres-

    sion approach that is totally different from all prior ACL com-

    pression algorithms. This novel approach is also very effective.

    We implemented Diplomat and conducted side-by-side com-

    parison with the prior Firewall Compressor, TCAM Razor and

    ACL Compressor algorithms on real life classifiers. The exper-

    imental results show that Diplomat outperforms Firewall Com-

    pressor, TCAM Razor and ACL Compressor most of the time,

    often by a considerable margin. In particular, on our largest

    ACLs, Diplomat has an average improvement ratio of 34.9%

    over Firewall Compressor on range-ACLs, of 14.1% over

    Razor on prefix-ACLs, and 8.9% over ACL Compressor on

    mixed-ACLs. Furthermore, on two-dimensional classifiers we

    demonstrate approximation ratios of O(min(n1/3, OPT 1/2)for range-ACLs and (min(w4/3n1/3, w2OPT 1/2)) for prefix-ACLs, improving on the prior best result of [2]. We do note

    that Diplomat achieves these performance gains at the cost of

    additional running time compared to both Firewall Compressor

    and TCAM Razor.

    The rest of this paper is organized as follows. First, we

    review related work in Section II. Second, we present key

    definitions and some fundamental results in Section III. We

  • 3

    then describe how Diplomat is applied to range and prefix-

    ACLs in Section IV. We cover the theoretical approximation

    bounds in Section V and the practical experimental results in

    Section VI. We conclude the paper by summarizing our results

    and discussing some open problems in Section VII.

    II. RELATED WORKThe 2-dimensional range-ACL compression problem was

    proven to be NP-hard in [2]. This implies that mixed-ACL

    compression with at least two range fields is also NP-Hard.

    Whether the prefix-ACL problem with a fixed number of

    dimensions is NP-hard is currently open. The 1-dimensional

    case for both range and prefix-ACL compression is in P.

    Applegate et al. [2] presented an algorithm for compressing

    2-dimensional range-ACLs. For the special case of strip rules

    where each rectangle spans the entire canvas in one of the two

    dimensions and only 2 colors, their algorithm is optimal. They

    then describe a way to divide the ACL into regions which can

    each be solved with their strip rule solver. When applying their

    solution to the general 2-dimensional range-ACL problem,

    they gave a O(min(n1/3, OPT 1/2))-approximation algorithm.They also adapt their methods to prefix-ACL compression

    which adds a factor of w2 to their approximation bounds wherew is the number of bits in the prefix word size. This is theonly approximation result we are aware of for either range

    or prefix-ACL compression for 2 or more dimensions. We

    adapt their methods in Section V to demonstrate that some

    versions of Diplomat can achieve the same bounds. Finally,

    they observe that the 1-dimensional range-ACL version was

    given as StripePainter in the 2003 Google TopCoder challenge

    and that a O(Kn3) running time can be achieved, where Kis the number of possible decisions [1].

    For the 1-dimensional prefix-ACL compression problem,

    Draves et al. gave an optimal algorithm based on tries [4],

    and Suri et al. gave an optimal algorithm based on dynamic

    programming [17]. Suri et al. also gave an optimal dynamic

    programming solution for the special case where two prefix

    rectangles in the ACL can intersect only if one is fully con-

    tained within the other. In this paper, we use Suri’s algorithm

    to optimally compress the 1-dimensional prefix-ACLs which

    the prefix version of Diplomat generates as subproblems.

    Liu et al. presented their own optimal 1-dimensional range-

    ACL compression solution based on dynamic programming

    methods with an improved running time of O(k2n) wheren is the number of rules and k is the maximum numberof rules with the same decision [11]. They then applied this

    algorithm to each of the layers of a firewall decision diagram

    [7] to create a solution for any number of dimensions. In their

    later paper [10], they apply a similar idea using the optimal

    1-dimensional prefix-ACL solver from [17] to expand their

    solution to operate on prefix-ACLs. In [13], Liu et al. define

    ACL Compressor which considers ACLs where some fields are

    ranges and some fields are prefixes. ACL Compressor operates

    on the same principles, using an optimal 1-dimensional range-

    ACL solver on range fields and an optimal 1-dimensional

    prefix-ACL solver on prefix fields. In this paper, we use

    Firewall Compressor to optimally compress the 1-dimensional

    range-ACLs which the range and mixed versions of Diplomat

    generate as subproblems.

    Finally, the Rectilinear Polygon Cover (RPC) Problem,

    which has been well explored in prior work [3], [6], [14], can

    be thought of as a special case of the range-ACL compression

    problem where the color of the last rule must be white and the

    color of all prior rules must be black. However, RPC is very

    different than range-ACL because the ability to use rectangles

    with different colors leads to much shorter rule lists. Thus,

    we cannot leverage any existing RPC algorithms to help with

    range-ACL compression.

    III. DEFINITIONS AND NOTATIONS

    Let K be the set of decisions. It can be any finite set, suchas {yes, no}, {accept, deny}, or {0, 1, 2, 3, 4}. The actualitems are irrelevant so long as they are distinct. A color c ∈ Kis any one of those decisions.

    We define a canvas C as the Cartesian product of d finitediscrete dimensions, [0, n1 − 1] × · · · × [0, nd − 1]. For theprefix version, ni = 2

    wi where wi is the number of bits used torepresent dimension i. A box B ⊆ C is any region representedby a Cartesian product of intervals. For prefix classifiers, the

    intervals are represented by the prefix {0, 1}k{∗}wi−k forsome k with 0 ≤ k ≤ wi. A pattern or coloring is a functionP : C→ K relating points in C to colors in K. An incompletepattern is a pattern that does not define a color for some points

    in the canvas. Given two incomplete patterns P1 and P2 thatdo not both assign colors to the same point in the canvas,

    P1 ∪ P2 is the pattern formed by joining the two patterns P1and P2.

    A rule r : B → K assigns exactly one color c ∈ K to eachpoint in box B. A rule list L = {r1, . . . , rn} is an orderedlist of rules and |L| = n is the number of rules in L. A rulelist L assigns colors to points as follows: for any point p ∈ C,L(p) is the value of ri(p) for the minimum i such that ri(p)is defined. A rule list is complete if L(p) is defined for allpoints p in the canvas and is incomplete otherwise. We use ℓto denote an incomplete rule list. We use the terms rule list

    and classifier interchangeably to refer to both a rule list Land the pattern defined by L. For a complete rule list L, wecall the color of rn the background color of L, and we canassume without loss of generality that domain(rn) = C sincern applies to all of C not covered by prior rules. For tworule lists L1 and L2, L1 · L2 denotes the rule list formed byappending L2 to the end of L1. We will also write ℓ ·P whereP is a pattern; in this case, the pattern implied by incompleterule list ℓ overrides P for points where both are defined.

    Two rule lists are equivalent, L1 ≡ L2, if L1(p) =L2(p) ∀p ∈ C. We denote the set of all rule lists equivalent toP as 〈P 〉. L is an optimal rule list for a pattern P if L ∈ 〈P 〉and |L| ≤ |L′| ∀L′ ∈ 〈P 〉. Our objective is to find an optimalrule list for an input pattern P . This was shown in [2] to beNP-hard for patterns of two or more dimensions in the range

    case (and may be for the prefix case as well), so we try to

    find one as close to optimal as possible.

    Given a pattern P , box B is P -monochromatic if P (p1) =P (p2) ∀p1, p2 ∈ B. We can break C into monochromaticboxes using Applegate et al.’s effective grid concept. Quoting

    [2], “For any rectilinear pattern P , call a horizontal or verticalline segment that separates and bounds two differently-colored

  • 4

    regions a boundary line. Extend each boundary line in both

    directions so that it crosses the full canvas. We call the

    resulting grid the “effective grid” for the pattern, and note

    that all its grid cells will be monochromatic.” Without loss

    of generality, we assume the input rule list L is convertedto the effective grid space, so C is the effective grid and ni(1 ≤ i ≤ d) is one more than the number of boundary planes indimension i. We also order the dimensions so n1 ≤ · · · ≤ nd.Applegate et al. also showed that ni ≤ 2n [2]. An exampleof an effective grid can be seen in Figure 3.

    Fig. 3. An effective grid denoted by the dashed lines

    IV. METHODOLOGY

    We now formally describe the Diplomat classifier com-

    pression algorithm. We first describe how we reduce our d-dimensional pattern into a collection of (d − 1)-dimensionalpatterns. Given a d-dimensional pattern P with a d-dimensional canvas C = [0, n1−1]×· · ·×[0, nd−1], define the(d−1)-dimensional canvas C′ = [0, n2−1]×· · ·× [0, nd−1].We divide C into n1 copies of canvas C

    ′ with C′i being the

    (d−1)-dimensional plane with f1 = i for 0 ≤ i ≤ n1−1. Wethen create n1 patterns Pi for 0 ≤ i ≤ n1− 1 where Pi is theintersection of plane C′i with the original pattern P . We definepattern Pi on the (d − 1)-dimensional canvas C′ as follows:for any point p ∈ C′, Pi(p) = P (i × p). In Figure 4 (a), wesee a 3-dimensional pattern reduced into four 2-dimensional

    patterns P0 through P3.

    P3

    P2

    P1

    d2

    d3

    d1

    P0

    (a) Before

    P3

    P[1,2]

    d2

    d3

    d1

    P0

    (b) After

    Fig. 4. Merging planes P1 and P2.

    We create the resulting rule list L for C from our sequenceof n1 patterns Pi (0 ≤ i ≤ n1 − 1) by “merging” togetheradjacent patterns until we are left with a single pattern and then

    recursively solving that single pattern. More formally, let P[i,j]denote some merging of patterns Pi to Pj inclusive. In Figure4 (b), we see P[1,2] representing the merger of patterns P1 andP2 from Figure 4 (a). For any point p ∈ C′, P[i,j](p) = Pk(p)for some i ≤ k ≤ j. For any p ∈ C′, we define pattern PC[i,j] onC by setting PC[i,j](k × p) = P[i,j](p) if i ≤ k ≤ j; otherwise,PC[i,j](k × p) is undefined.

    Initially, P[i,i] = Pi ∀i ∈ [0, n1 − 1]. Initialize L to be theempty rule list. While we have at least two patterns remaining

    Algorithm 1 Diplomat Solver

    Require: Pattern PEnsure: Rule list L

    1: L← ∅ ⊲ Final Result2: ~R← {Pi : Pi is the subpattern with P (f1) = i∀i ∈ [0, n1−1]}

    ⊲ Sub-planes

    3: ~I ← {[i, i] : ∀i ∈ [0, n1 − 1]} ⊲ Intervals

    4: while

    ~R∣

    ∣> 1 do

    5: (t, b)← Schedule(~R)6: (ℓ, Rm)← Resolve(Rt, Rb, It, Ib)7: Replace Rt and Rb with Rm8: Replace It and Ib with Join(It, Ib)9: Append ℓ to L

    10: end while11: Llower ← Solve(R)12: Append Llower to L13: return L

    in our sequence of patterns, we select two adjacent patterns

    P[i,j] and P[j+1,k] to be merged. We merge these two patternsinto a single pattern by providing extra information in a partial

    rule list ℓ defined on C which is appended to L. Intuitively, ℓcolors points where P[i,j] and P[j+1,k] differ; this allows thetwo patterns to be merged. Let ℓq be the partial rule list usedin the qth merger of patterns where 1 ≤ q ≤ n1−1. When wehave a single pattern P[0,n1−1] remaining, the rule list L is theconcatenation of partial rule lists ℓq for 1 ≤ q ≤ n1 − 1. Wethen recursively solve the (d−1)-dimensional pattern P[0,n1−1]which is then appended to L for our final solution. If d = 2which means d−1 = 1, we have reached our base case and usethe optimal one-dimensional solvers in [11] or [17] to generate

    a solution to P[0,n1−1].To merge two adjacent patterns P[i,j] and P[j+1,k] , we need

    a resolver which we formally define as follows.

    Definition 1. A resolver of P[i,j] and P[j+1,k] returns a partialrule list ℓ defined on C and pattern P[i,k] defined on C

    ′ subject

    to the following constraints: ∀p ∈ C′, P[i,k](p) equals eitherP[i,j](p) or P[j+1,k](p) and ℓ · PC[i,k] ≡ PC[i,j] ∪ PC[j+1,k].

    We observe that if P[i,j] ≡ P[j+1,k] then P[i,k] will matchboth lists and ℓ is the empty list. While a resolver is formallythe function that computes the partial rule list ℓ and the mergedpattern P[i,k], we also often refer to partial rule list ℓ as aresolver and say that ℓ resolves P[i,j] and P[j+1,k].

    Next, we review the Firewall Compressor and Suri algo-

    rithms which are used as sub-programs by our resolvers.

    We then formally present several different resolvers to merge

    adjacent patterns and schedulers to determine the merging

    order for patterns.

    A. Firewall Compressor

    We review here the 1-dimensional version of Firewall Com-

    pressor from [11]. Given some input classifier L, compute theeffective grid of L and label the cells from 0 to n−1. The firstkey insight is that the last rule rm covers the leftmost cell 0 insome optimal solution. Liu et al. use dynamic programming

    to determine which other cells are also covered by rule rm. Ifno other cells are covered, then the optimal solution consists

    of rule rm which only covers cell 0 plus an optimal solution

  • 5

    that covers region [1, n − 1]. Otherwise, let x be the nearestcell that is also covered by rm. The optimal solution is thenone that covers region [1, x − 1] and one that covers region[x, n− 1] where the last rule covers cell x (and cell 0).

    B. Suri’s Prefix ACL Solver

    We review the 1-dimensional version of the prefix ACL

    solver from [17]. Their method works by dividing the range

    into regions designated by prefixes. Each region has a desig-

    nated background color. They then proceed to merge regions

    together, sharing background colors between neighboring re-

    gions at reduced cost when possible.

    More formally, they begin by dividing P into the largestprefix segments that are monochromatic. For each prefix-color

    pair, compute Cost(p, c), where c is the background colorapplied to the given prefix. For the base case of one of the

    monochromatic prefixes, then Cost(p, c) = 0 if c is the color ofthe segment and 1 otherwise. For other prefixes, Cost(p, c) =minc0,c1(Cost(p0, c0) + Cost(p1, c1) + [c 6= c0] + [c 6= c1].The final cost, Cost(p) = minc Cost(p, c) + 1 (to fill in thebackground color). We utilize Suri’s algorithm for constructing

    our prefix resolvers.

    C. Resolvers

    We first define two different types of resolvers that constrain

    the type of scheduler Diplomat can use.

    Definition 2. An in-plane resolver is a resolver where all of

    the rules in ℓ correspond to one of the two patterns beingmerged. Stated another way, all the rules in ℓ come from thecanvas C′ corresponding to only one of the two patterns being

    merged. A general resolver has no constraints placed upon the

    rule source.

    With an in-plane resolver, the resulting merged pattern will

    be identical to the pattern that is not the source of the rules in

    ℓ. With a general resolver, the merged pattern may be a newpattern that is different from either of the two input patterns.

    The general resolver has the advantage that ℓ can be smallersince we have more flexibility in resolving differences between

    the two patterns. The in-plane resolver has the advantage that

    because the resulting pattern will be one of the input patterns,

    the scheduler can use dynamic programming. We now describe

    our efforts to develop fast and effective resolvers. We begin by

    giving optimal general and in-plane resolvers for rows (one-

    dimensional planes) and then give heuristic higher dimensional

    resolvers.

    1) Optimal one-dimensional resolvers: In this section, C′

    is a row. We use Rt and Rb to correspond to the two patternsP[i,j] and P[j+1,k] that are being merged. We use Rt[i, j]and Rb[i, j] to refer to the sub-pattern in row Rt and Rb,respectively, between columns i and j. All the resolvers wepresent use the optimal one-dimensional versions of Firewall

    Compressor (FC) in [11] or Suri’s algorithm in [17] as a

    subroutine. Proofs of optimality for the resolvers can be found

    in Section V.

    a) Optimal General Range-Resolver: Here, we present

    an optimal way to merge Rt and Rb between columns 0 andk. Let ℓk denote the partial rule list returned by this optimalsolution, let Ck = |ℓk|, and let Pk denote the resulting patternthat is returned. If Rt[k] = Rb[k], then ℓk = ℓk−1, Ck =Ck−1, and Pk = Pk−1 ∪ Rt[k] as there are no differences incolumn k. Otherwise, there is a difference between Rt andRb in position k. This means ℓk must include at least onerule that covers either Rt[k] or Rb[k]. We prove in SectionV-A that we can restrict our attention to resolvers that cover

    only Rt or Rb from column k back to some column s where0 ≤ s ≤ k. This is then combined with an optimal solutionup to column s− 1. We consider all possible choices of thissplit point s in the following description and set C−1 = 0,ℓ−1 = ∅, and P−1 = ∅ for the case where s = 0. Then Ck =min0≤s≤k(Cs−1 +min(|FC(Rt[s, k])| , |FC(Rb[s, k])|)) andSk is the corresponding value of s. Now, using Sk, we buildthe solution ℓk, Pk. We add the rules from the appropriatesolution FC(Rt[Sk, k]) or FC(Rb[Sk, k]) to ℓSk−1 to formℓk, and we add the other pattern for columns Sk to k to PSk−1to form Pk.

    b) Optimal In-Plane Range-Resolver: This version as-

    sumes that all of the rules for ℓk have to come from Rt orRb. Without loss of generality, assume the rules come fromRt. Then Pk is defined by Rb. We use a dynamic programmingtechnique similar to the general resolver to create our rule list

    ℓk. Since we know no rules for ℓk can come from Rb, we onlyneed to compute the partial solutions in Rt.

    Algorithm 2 Optimal In-Row Range Resolver

    Require: Rows Rt and Rb and interval ItEnsure: A rule list, ℓ marking how Rt is different from Rb.

    1: ℓ = ∅2: cost(−1) = 03: for k = 0→ nd − 1 do4: if Rt(k) 6= Rb(k) then5: bestS ← ∅6: bestScore←∞7: for s = 0→ k do8: score← cost(s− 1) + FC(Rt([s, k]))9: if score < bestScore then

    10: bestS ← s11: bestScore← score12: end if13: end for14: cost(k)← bestScore15: backtrace(k)← bestS16: else17: cost(k)← cost(k − 1)18: end if19: end for20: k ← nd − 121: while k ≥ 0 do22: if Rt(k) 6= Rb(k) then23: ℓ← It × FC(Rt([backtrace(k), k]))24: k ← backtrace(k)− 125: else26: k ← k − 127: end if28: end while29: return ℓ

  • 6

    c) Optimal General Prefix-Resolver: Here, we present

    an optimal way to merge Rt and Rb over the range ofsome prefix pr using only prefix rules; the end result isresolver l(pr) and pattern P (pr). We begin by observing thatSuri(Rt) and Suri(Rb) are both resolvers. Let p denotethe smallest prefix (and thus specifies the largest number

    of points) used in an optimal resolver. The restrictions

    on prefix rules means that either p = pr, p ⊆ pr0, orp ⊆ pr1. If p = pr, then the optimal resolver is eitherSuri(Rt) or Suri(Rb). Otherwise, it is some combinationof the optimal resolvers for pr0 and pr1. That is, ℓ(pr) =OptResolve(Rt, Rb, pr) = min(OptResolve(Rt, Rb, pr0) ·OptResolve(Rt, Rb, pr1), Suri(Rt, pr), Suri(Rb, pr)) Ourbase case for this recurrence relation occurs when Rt[pr]or Rb[pr] is monochromatic meaning |OptResolve| isone. We create P (pr) in the same way as for the generalrange-resolver.

    d) Optimal In-Plane Prefix-Resolver: This version as-

    sumes that all of the rules in ℓ(pr) have to come from Rt orRb and that the ranges are restricted to being prefixes. Withoutloss of generality, assume that the rules come from Rt. ThenP (pr) is defined by Rb. The algorithm is otherwise identical tothe general prefix-resolver except that we cannot use Suri(Rb)as part of the solution.

    Algorithm 3 Optimal In-Row Prefix Resolver

    Require: Rows Rt and Rb of width 2w and interval It

    Ensure: A rule list ℓ marking how Rt is different from Rb.C : p→ Z ⊲ Return values of all calls to PCost()C ← PCost(Rt, Rb, ∗

    w)ℓ← PBuild(Rt, It, C, ∗

    w)return ℓ

    Algorithm 4 Prefix Resolve Cost

    Require: Rows Rt and Rb and prefix pEnsure: Cost cp

    if Rt[p] is monochromatic thenif Rt[p] = Rb[p] then

    return 0else

    return 1end if

    elsecm ← PCost(Rt, Rb, p0) + PCost(Rt, Rb, p1)cs ← SuriCost(Rt, p)if cs ≤ cm then

    return cselse

    return cmend if

    end if

    2) Higher Order Resolvers: If d ≥ 3, we need to merge twopatterns P1 and P2 that are (d− 1)-dimensional hyperplanes.This is more complicated than merging two rows. In particular,

    we have an optimal algorithm for compressing 1-dimensional

    patterns, but compressing 2-dimensional patterns is NP-hard

    as shown in [2]. This difference in complexity carries over to

    merging patterns.

    We propose two different in-plane methods for merging

    (d−1)-dimensional hyperplanes when d ≥ 3. We use in-plane

    Algorithm 5 Prefix Resolve Builder

    Require: Row RtRequire: Interval ItRequire: Cost function C : p→ ZRequire: Prefix pEnsure: Rule list ℓ

    if SuriCost(Rt, p) = C[p] thenreturn Suri(Rt, p)

    else if C[p] > 0 thenreturn PBuild(Rt, It, C, p0) · PBuild(Rt, It, C, p1)

    end if

    methods so that we can use a dynamic programming scheduler.

    We can easily extend both methods to general resolvers if

    desired. Without loss of generality, we assume that we wish

    to merge P1 and P2 leaving pattern P1. Thus, all rules in theresolver will be within P2.

    The first method is to enumerate all the differences. That

    is, we use the FDD comparison algorithm in [9] to find the

    differences between the two patterns. We then define ℓdif bygenerating rules in P2 to cover all the places where P2 differsfrom P1. We may cover additional locations if this reducesthe total number of rules required to cover all the differences.

    This resolver implicitly uses the correct type of rule (range,

    prefix or mixed) given the characteristics of the fields.

    In the second method, we find box, the minimum boundingbox surrounding all differences. If any of the fields are prefix

    fields, we expand the bounding box in that dimension so

    that the bounding box can be defined by prefixes. We then

    solve P2(box) using the appropriate version of Diplomat(range, prefix or mixed) and call the resulting solution ℓmbb.Alternately, we can solve P2 and then clip the results againstthe bounding box, removing any rules that fall entirely outside

    the box. This solution is faster because the subsolution can

    be computed once and shared between merges; however, this

    may produce less compression because we do not optimize

    the compression for the bounding box. We use this variant for

    our experimental results in Section VI

    To achieve greater compression, we run both methods and

    use the smaller of the two resolvers. That is, our final resolver

    ℓ = min(ℓdif , ℓmbb).

    D. Schedulers

    To complete the Diplomat algorithm, we need a scheduler

    to determine an order for merging patterns. We present two

    such schedulers. The first uses a greedy scheme and works

    with any resolver while the second uses dynamic programming

    to get better overall results but requires in-plane resolvers.

    We can prove theoretical bounds for the greedy scheduler, but

    the dynamic programming scheduler outperforms the greedy

    scheduler in our experiments.

    1) Greedy Range Scheduler: For the greedy scheduler, we

    compute the cost of the original n1− 1 possible merges, pickthe minimum cost merge, and repeat until only one pattern

    remains. Note that after performing a given merge, we must

    recompute the cost of the one or two merges that involve the

    new merged pattern. Overall, we perform at most 2n1−2 callsto Resolve. For a d-dimensional classifier, we can consider

  • 7

    all possible permutations of the d fields when performing thegreedy scheduling.

    Thinking more generally, there is no reason to perform

    merges in only one dimension at a time. Instead, we can

    greedily choose the best merge in any of the d dimensionsat any point in time. In particular, for d = 2, we find thecheapest pair of rows or columns to merge. In section V,

    we show how to use this rotational greedy schedule to create

    an O(min(n1/3, OPT 1/2))-approximation for 2-dimensionalpatterns.

    2) Dynamic Programming Range Scheduler: We also

    present a dynamic programming method that requires more

    initial work and, as a result, outperforms the greedy sched-

    uler in our experiments. Using an in-plane range resolver

    such as the one in Section IV-C1b, let MergeCost[i, j] =|Resolve(Pi, Pj)| ∀i 6= j ∈ [0, n1 − 1] where the returnedpattern is Pi. This requires O(n

    21) calls to Resolve. However,

    because the merged pattern is always one of the original

    patterns, we do not need to do any more merges.

    We then define the following subproblem. Given interval

    [l, u] for 0 ≤ l ≤ u ≤ n1 − 1, we find the minimumcost for merging patterns Pl, . . . Pu together while havingPr be the remaining pattern where l ≤ r ≤ u. We usestandard dynamic programming techniques to compute the

    minimum cost solution to these O(n31) subproblems and thecorresponding optimal schedule of merges.

    3) Greedy Prefix Scheduler: For a greedy prefix scheduler,

    we take the greedy range scheduler described above and make

    two changes to it. First, when considering the cost of rules in

    P[i,j], the rules are weighted by the number of prefixes it takesto describe the range [i, j]. This acts as a conversion factorbetween the effective grid space and the prefix space. Second,

    we use a weighted prefix resolver instead of a range resolver.

    4) Dynamic Programming Prefix Scheduler: We also

    present a dynamic programming method for scheduling that

    works with prefixes. We begin by precomputing the merge

    costs as for the range scheduler above. We then define the

    recursive subproblem. Given a prefix p and row r ∈ p, wefind the minimum cost for merging the two halves, p0 and p1while having Pr be the remaining pattern. We use standarddynamic programming techniques to compute the minimum

    cost solution to each of the O(w2n21) subproblems and thecorresponding schedule of merges.

    5) Mixed Schedulers: We can generalize our greedy and

    dynamic programming range and prefix schedulers into a

    greedy and dynamic programming mixed scheduler as follows.

    The key is the type of field f1. If f1 is a range field, then it actslike a range scheduler; if f1 is a prefix field, then it acts likea prefix scheduler. To merge the resulting hyperplanes defined

    by the remaining fields, it calls an appropriate resolver. If all

    of the remaining fields are range fields, then it calls a range

    resolver. If all of them are prefix fields, then it calls a prefix

    resolver. Otherwise, it calls a mixed resolver.

    V. ALGORITHMIC ANALYSIS

    We now prove some results about our algorithms. We first

    prove that our one-dimensional resolvers are optimal. We then

    prove some approximation results for some two-dimensional

    Algorithm 6 Scheduler(l, u, r,MergeCost)

    Require: Integers l, u, and r such that 0 ≤ l ≤ r ≤ u < n1Require: n1 × n1 matrix MergeCostEnsure: Stack of triples 〈Rt, Rb, I〉, ScheduleEnsure: The last row, Lr

    1: if l = u then2: Cost[l, u, r]← 03: end if4: minCost←∞5: minS ← ∅6: minM ← ∅7: for all s ∈ [l, r] do8: for all m in[r + 1, u] do9: c← Cost[l, s, r]+Cost[s+1, u,m]+MergeCost[m,r]

    10: if c < minCost then11: minCost← c12: minS ← s13: minM ← m14: end if15: end for16: end for17: for all s ∈ [r + 1, u] do18: for all m in[l, r] do19: c← Cost[l, s,m]+Cost[s+1, u, r]+MergeCost[m,r]20: if c < minCost then21: minCost← c22: minS ← s23: minM ← m24: end if25: end for26: end for27: Cost[l, u, r]← minCost28: Split[l, u, r]← minS29: Mate[l, u, r]← minM30: rmin = argr minCost[1, n1, r] + |FC(Lr)|.31: return Schedule(1, nd, r, Split,Mate), Lr

    Algorithm 7 Schedule(l, u, r, Split,Mate)

    Require: Integers l, u, and r such that 0 ≤ l ≤ r ≤ u < n1Require: n1 × n1 × n1 matrices Split and MateEnsure: a triple 〈Rt, Rb, I〉 is added to the stack Schedule.

    1: s← Split[l, u, r]2: m←Mate[l, u, r]3: if l < r then4: if m ≤ s then5: push 〈m, s, [l, s]〉 onto Schedule6: else

    7: push 〈m, s, [s+ 1, u]〉 onto Schedule8: end if

    9: end if

    Algorithm 8 Prefix-Scheduler

    Require: n1 × n1 matrix MergeCostEnsure: Stack of triples 〈Rt, Rb, I〉, ScheduleEnsure: The last row, Lr

    1: Costs, Mates2: for all p ∈ ∗w do3: PrefixScheduleCost(∗w , p)4: end for5: r ← argmin

    r

    Costs[∗w, r]

    6: Schedule← PrefixSchedule(Mates, ∗w, r)7: Lr ← Rr

  • 8

    Algorithm 9 Prefix-Scheduler-Cost

    Require: Prefixes p and r : r ⊆ sRequire: n1 × n1 matrix MergeCostEnsure: Costs[p, r], Mates[p, r]

    1: if p = r then2: Costs[p, r] ← 03: Mates[p, r] ← ∅4: else if r ⊆ p0 then5: PrefixSceduleCost(p1, r)6: cbest ←∞7: mbest ← ∅8: for all m ∈ p1 do9: PrefixSceduleCost(p1, r)

    10: end for11: m← argmin

    m∈p1

    MergeCost[r,m] +Costs[p1,m]

    12: Costs[p, r] ← MergeCost[r,m] + Costs[p0, r] +Costs[p1,m]

    13: Mates[p, r] ← m14: else15: Right side is analogous16: end if

    Algorithm 10 Prefix-Schedule

    Require: MatesRequire: Prefixes p and r : p ⊆ rEnsure: 〈Rt, Rb, p〉 is added to the stack Schedule.

    1: if r ⊂ p then2: m←Mates[p, r]3: Schedule← 〈Rr, Rm, p〉4: if r ⊆ p0 then5: PrefixSchedule(Mates, p0, r)6: PrefixSchedule(Mates, p1,m)7: else8: PrefixSchedule(Mates, p1, r)9: PrefixSchedule(Mates, p0,m)

    10: end if11: end if

    implementations of Diplomat. Finally, we prove that Diplomat

    with a dynamic programming scheduler should outperform the

    current state of the art algorithms, Firewall Compressor for

    range-ACLs and TCAM Razor for prefix-ACLs.

    A. Optimal One-Dimensional Resolvers

    In this section, we prove that our one-dimensional resolvers

    are optimal.

    We first prove that there are optimal one-dimensional re-

    solvers in which no rule spans both rows.

    Theorem 1. Let ℓ be a rule list that resolves Rt and Rb, Thenthere exists a rule list ℓ′ that also resolves Rt and Rb suchthat |ℓ′| ≤ |ℓ| and that no rule r ∈ ℓ′ spans both rows. Inparticular, there is an optimal resolver with this property.

    Proof: We prove this result by contradiction. Let ℓ be anoptimal resolver with the minimum possible number of rules

    k that span both rows. If k = 0 then ℓ = ℓ′ and we are done.Otherwise, let ri be the last rule in ℓ spanning both rows.

    We first consider the case where there is no rule rj withj > i and ri ∩ rj 6= ∅. In this case, we create a new rule listℓ∗ from ℓ by replacing ri with rt = ri ∩Rt. Any point in Rtcovered by ri is also covered by ri ∩ Rt, so ℓ∗ still resolvesall of the columns. For any point p ∈ Rb, either some rule rk

    with k < i defines p for both ℓ and ℓ∗ or no rule in ℓ∗ does.Thus, there can be no color clash and ℓ∗ is still a resolver.This means ℓ∗ is an optimal resolver with only k − 1 rulesthat span both rows contradicting the definition of ℓ. Thus,this case is not possible.

    The case where all rj with ri ∩ rj 6= ∅ for j > i are in rowRt can be shown to be impossible using the same analysis. Thereverse case where are all intersections occur in row Rb canbe handled similarly; the only modification is that we replace

    ri with ri ∩Rb to form ℓ∗.We now consider the last case where some later rules that

    intersect ri are in row Rt and others are in row Rb. We handlethe range and prefix cases separately, starting with the range

    case. We first trim all rules rj with j > i so that both its leftand right end points are not covered by any prior rules. Since

    we merely removed the parts of rj covered by other rules, theresulting rule list ℓ∗ is equivalent to ℓ. This may reduce theproblem to one of the above cases in which case we are done.

    If not, let S be the set of rules rj in ℓ∗ with j > i that intersect

    ri and let St ⊂ S be the subset in row Rt and Sb ⊂ S bethe subset in row Rb. Assume without loss of generality thatr∗ ∈ St is the first rule in S to appear scanning from leftto right. Since r∗ intersects ri, its right end point must be tothe right of ri’s end point given the trimming operation weperformed earlier. We then apply a second trimming operation

    and update ℓ∗ by replacing all rules rj ∈ Sb with the portion ofrj that is strictly to the right of ri. This classifier still resolvesRt and Rb since rt still covers any column that was formerlycovered by rb. We have thus reduced this case to the priorcase where all rules after ri that intersect ri are from one row,and the proof is complete for ranges.

    For prefix rule lists, we observe that for any rj and j > i,either rj ⊂ ri (a contradiction since rj is totally obscured byri), or else ri ⊂ rj in the horizontal field. Let rt be the widestsuch rule from Rt and rb the same from Rb. Such rules mustalso be the last such rules since any narrower rules must lie

    entirely within their ranges and thus be obscured. Likewise,

    either rt ⊆ rb or rb ⊆ rt in this field. Assume without loss ofgenerality that rt ⊆ rb. If we remove rt, we find that rb stillresolves all of the columns within this range. Since rt musthave been the last rule in this range within Rt, there can beno other rule that is revealed by the removal of rt and thusno color clash. Thus ℓ − rt is still a resolver, a contradictionto ℓ being optimal. Thus the proof is complete for prefixes aswell.

    We next prove that there are optimal one-dimensional re-

    solvers in which no column is covered by rules in both rows.

    Theorem 2. Let ℓ be a rule list that resolves Rt and Rb. Thenthere exists a rule list ℓ′ that resolves Rt and Rb such that|ℓ′| ≤ |ℓ| and that for any i ∈ [0, nd − 1], either ℓ′(t, i) orℓ′(b, i) is undefined.

    Proof: First, by Theorem 1, we can assume that ℓ hasno rule that occupies both rows. Let ℓ be an optimal resolverwith the minimum number of rules k in one row that overlapsthe horizontal range of any rule in the other row. If k = 0,we are done. Otherwise, assume without loss of generality

  • 9

    that r∗ ∈ Rt is the first such rule when scanned left to right.We create a modified rule list ℓ∗ by trimming all rules in Rbthat overlap the horizontal range of r∗, removing any that areentirely contained within the range of r∗ (which cannot happenor else ℓ is not optimal). For prefix classifiers, any overlappingrule must by entirely within this range (and thus there cannot

    be any overlapping rules). The rule list ℓ∗ still resolves Rt andRb since ℓ

    ∗ specifies values for all columns affected in row

    Rt. However, because the modified rule r∗ does not overlap

    any rules in Rb, ℓ∗ has a smaller k value contradicting the

    definition of ℓ, and the result follows.We now prove the optimality of our general one-

    dimensional resolver.

    Theorem 3. The Optimal Range-Resolver algorithm described

    in Section IV-C1a is an optimal one-dimensional range-

    resolver.

    Proof: We prove this by induction on k, the length ofthe rows. The base case k = 0 is trivial. Now assume thismethod produces an optimal resolver ℓj for 0 ≤ j ≤ k. Wenow consider k+1. We first observe that the optimal resolverfor k + 1 has no fewer rules than the optimal resolver for k.Next, if Rt[k] = Rb[k], then ℓk+1 = ℓk is clearly an optimalresolver as it resolves all differences between the two rows in

    the first k + 1 columns. Thus, by our induction hypothesis,our one-dimensional resolver is an optimal one-dimensional

    resolver for this case.

    If Rt[k] 6= Rb[k], by Theorem 2 we restrict our attentionto optimal resolvers ℓk+1 in which no column is covered byrules in both rows. Column k must be covered by some rulesince Rt[k] 6= Rb[k]. This means column k will be coveredby a rule ri in only Rb or Rt. Without loss of generality, letus assume ri is only in Rt. By Theorem 2, it follows thatfor some 0 ≤ q ≤ k that ℓk+1 must color Rt[q, k] and ℓk+1does not color Rb[q, k]. Thus ℓ

    ∗ will be an optimal resolver

    ℓq−1 combined with an optimal coloring of Rt[q, k]. By ourinduction hypothesis, the fact that Firewall Compressor will

    compute an optimal coloring of Rt[q, k], and the fact we tryall possible values of q, our general resolver algorithm is anoptimal resolver that finds such a solution.

    We also show that our one-dimensional in-plane resolver is

    an optimal one-dimensional in-plane resolver.

    Theorem 4. The Optimal In-Plane Range-Resolver algorithm

    described in Section IV-C1b is an optimal one-dimensional

    in-plane range-resolver.

    Proof: We again induct on k, the length of the rows. Thebase case k = 0 is trivial. Now assume this method producesan optimal in-plane resolver ℓj for 0 ≤ j ≤ k. We nowconsider k+1. If Rt[k] = Rb[k], then ℓk+1 = ℓk is clearly anoptimal in-plane resolver as it resolves all differences between

    the two rows in the first k + 1 columns, and |ℓk+1| ≤ |ℓk|.Thus, by our induction hypothesis, our in-plane resolver is an

    optimal in-plane resolver for this case.

    If Rt[k] 6= Rb[k], then ℓk+1 must color Rt[k]. It followsthat ℓk+1 must color Rt[q, k] for some 0 ≤ q ≤ k. Thusℓk+1 will be an optimal in-plane resolver ℓq−1 combined withan optimal coloring of Rt[q, k]. By our induction hypothesis,

    the fact that Firewall Compressor will compute an optimal

    coloring of Rt[q, k], and the fact we try all possible valuesof q, our in-plane resolver algorithm is an optimal in-planeresolver that finds such a solution.

    Theorem 5. The Optimal General Prefix-Resolver algorithm

    described in Section IV-C1c is an optimal one-dimensional

    general prefix resolver.

    Proof: We induct on w, the word size of the rows (2w isthe length of the row). The base case w = 0 is trivial. Nowassume that this method produces an optimal in-plane resolver

    for all smaller word sizes. If ℓ(pr) = Suri(Rt) or ℓ(pr) =Suri(Rb) is an optimal in-plane resolver, then this methodfinds an optimal solution. Otherwise, the rule delimited by prcannot be included in the optimal resolver for pr. Because ofthe limitations on prefix rules, all pi are contained in either pr0or pr1. Thus, ℓopt = ℓ(pr0)·ℓ(pr1), which is the other solutionconsidered by our method. By the inductive hypothesis, both of

    the subsolutions are optimal and thus so is the overall solution.

    Theorem 6. The Optimal In-Plane Prefix-Resolver algorithm

    described in Section IV-C1d is an optimal one-dimensional

    in-plane prefix-resolver.

    Proof: This proof is essentially the same as for Theorem

    5.

    B. Two-Dimensional Approximation Results

    In [2], the authors present an iterated strip-rule algorithm

    and prove that it is a O(min(n1/3, OPT 1/2))-approximationalgorithm on range-ACLs and a O(w2 min(n1/3, OPT 1/2))-approximation algorithm on prefix-ACLs, where n is thenumber of rules in the input list, w is the word size in bits,and OPT is the size of the optimal solution. We simplify theiralgorithm and generalize it to create a class of approximation

    algorithms with the same approximation ratio. We then show

    that some Diplomat variations belong to this class.

    We first define strip rule patterns [2].

    Definition 3. A strip-rule pattern is a two-dimensional pattern

    which can be created by a rule list where each rule spans an

    entire row or column.

    Applegate et al. prove several properties about strip-rule

    patterns, the most important of which is that any 2 × 2 or3× 3 sub-array must have a monochromatic row or column.Definition 4. A forbidden rectangle is a minimal subpattern

    containing either a 2×2 or 3×3 subarray with no monochro-matic rows or columns.

    Any pattern that includes a forbidden rectangle is not a

    strip-rule pattern.

    We now define the class of strip solver compression algo-

    rithms.

    Definition 5. A strip solver is a compressor that on an m×n strip-rule pattern always returns a rule list with at mostk(m+ n) rules for some constant k.

  • 10

    Fig. 5. A forbidden rectangle

    On other m × n patterns, a strip solver may either returnsuch a rule list or it may fail. Any compressor that completes

    on all values and satisfies the bounds on strip rule patterns can

    be turned into a strip solver by checking to see if the limit was

    met and failing if it does not.

    Theorem 7. Range Diplomat with the rotational greedy sched-

    uler is a strip solver compressor with k = 1.

    Proof: Let P be an arbitrary strip-rule pattern. We provethis by induction on m+n, the sum of the number of rows andcolumns in P . If m+ n ≤ 1, then we have the empty patternwhich Diplomat solves optimally with 0 rules. If m+ n = 2,then we have a pattern with 1 row and 1 column which means

    it is a single cell which Diplomat again solves optimally with

    1 rule. If m + n = 3, then we have a pattern with either 1row and 2 columns or 2 rows and 1 column which Diplomat

    solves optimally using 2 rules.

    Now consider m + n ≥ 4 and assume the inductivehypothesis holds for all patterns where the sum of the rows

    and columns is strictly smaller than m + n. Since P is astrip-rule pattern, there must exist a monochromatic row or

    column in P . Without loss of generality, assume there exists amonochromatic row R∗. This row can be merged with eitherof its neighbors at cost 1 by coloring R∗ with a single rule.Thus, Diplomat with the rotational greedy scheduler must

    incur a cost of exactly one when merging the first two rows

    or columns. Without loss of generality, we assume Diplomat

    merges rows Rt and Rb.We now argue that P ′, the resulting pattern of merging

    Rt and Rb, is still a strip-rule pattern. Assume otherwisefor contradiction. This implies that the row created by the

    merge is part of a forbidden rectangle. However, since only

    a single rule is placed in one of the rows, the merged row

    must be identical to the other row. This implies that the

    forbidden rectangle already exists in the original pattern P ,a contradiction. Thus, P ′ is a strip rule pattern where the sumof the rows and columns is at most m + n − 1. Thus, bythe inductive hypothesis, Diplomat with the rotational greedy

    scheduler solves P ′ with at most m+n− 1 rules. Combiningthis with the one rule used to merge Rt and Rb, we see thatDiplomat with the rotational greedy scheduler colors P withat most m+ n rules and the inductive case is complete.

    We now define the class of iterated strip solver algorithms

    using ideas from Applegate et al. Iterated strip solver al-

    gorithms can be applied to arbitrary 2-dimensional patterns

    P . The first step is to partition P as follows. Let P be a2h × 2w pattern for integers h and w and assume withoutloss of generality that w ≥ h; we pad P with empty rowsand columns as needed if they are not powers of 2. For each

    integer q ∈ [1, w], do the following. Partition P into vertical

    sections of width 2q . Partition each section into a sequence ofdisjoint blocks B1, B2, . . . , where each block Bi is a maximalheight subpattern using the full width of the section in which

    the strip solver returns a value. Apply a strip solver to each

    block. For each value of q, form a solution by taking theunion of solutions for each block defined by q. Because theblocks are disjoint, the solutions for each block are completely

    independent. The final solution is the best of the w differentsolutions for each width 2q.

    Applegate et al. prove the following result.

    Lemma 1. Suppose there are r disjoint forbidden rectanglesin a 2-dimensional pattern P . Then OPT (P ) ≥ r/4.

    Proof: Applegate et al. give a proof for this in [2].

    The intuition is that the first rule to include any corner of

    a forbidden rectangle must itself have a corner within that

    forbidden rectangle.

    We now prove an approximation bound for any iterated strip

    solver algorithm.

    Theorem 8. For any two-dimensional pattern P ,any iterated strip solver with constant k is aO(min(k2/3n1/3, kOPT 1/2))-approximation algorithm.

    Proof: For a given value of q, there is a total of b+2w−q

    blocks where b ≥ 0. The first 2w−q blocks must exist sinceeach vertical section has at least one block. Additional blocks

    are added each time the strip solver cannot add another row in

    the current section. For any pair of vertically adjacent blocks,

    there must be at least one forbidden rectangle or else the two

    blocks would together be a strip rule pattern. While some of

    the forbidden rectangles may overlap, if two pairs are disjoint,

    then their forbidden rectangles must also be disjoint. Thus,

    there are at least b/2 disjoint forbidden rectangles. This leadsto three lower bounds on OPT: b/8, 2w and 2h.

    For a given value of q, an upper bound on the cost of theiterated strip solver solution is given as follows:

    TotalCost = k∑

    Bi(Width(Bi) +Height(Bi))

    = k∑

    BiWidth(Bi) + k

    ∑Bi

    Height(Bi)

    = k(b+ 2w−q)2q + k(2w−q2h)= k(b2q + 2w + 2w−q+h)≤ k(8OPT 2q +OPT +OPT 2/2q)

    We choose the value of q that minimizes the above ex-pression. If we choose q so that 2q =

    √OPT we get

    Cost ≤ k(OPT + 9OPT 3/2) = O(kOPT 3/2) leading toan O(k

    √OPT )-approximation. While we cannot know q a

    priori, by selecting multiple different q we can guarantee that2q−1 <

    √OPT ≤ 2q for some q.

    We can choose to use L if it is better than the classi-fier generated above. This means |L| ≤ kOPT 3/2 whichmeans OPT ≥ (|L|/k)2/3 = (n/k)2/3 which leads to|L|/OPT ≤ k2/3n1/3. Further, the two bounds are equal ifOPT = (n/k)2/3 and the result follows.

    Corollary 1. For any two-dimensional pattern P , range-Diplomat with the rotational greedy scheduler is a

    O(min(n1/3, OPT 1/2))-approximation algorithm.

  • 11

    Proof: The result follows immediately from Theorem 7

    and Theorem 8.

    For prefix-ACLs, we present these modifications to the

    iterated strip solver algorithm. We divide the effective grid into

    2w−q columns of width 2q as we did for the range version.We then divide these columns into vertical sections so that

    each section can be represented by a prefix. We then apply

    a prefix strip solver to each section. If the strip solver does

    not return on a section, we divide it into blocks, each half the

    height of the pattern and repeat until it returns on each block.

    The solution is again the union of the result on each block.

    Theorem 9. For any two-dimensional pattern P , any iteratedprefix strip solver is a O(min(k2/3w2/3n1/3, kw

    √OPT ))-

    approximation algorithm.

    Proof: The 2w−q columns created by the initial parti-tioning may require O(w2w−q) prefix-columns to represent.We also assume that another b blocks are added becausesome blocks are not solvable by the strip-solver. This gives

    O(w2w−q + b) blocks. As before, we note a relation betweenthe number of added blocks and the number of forbidden

    rectangles. This time, a single forbidden rectangle can force

    up to w additional blocks to be added. This is because wedivide the block in half and the forbidden rectangle can remain

    entirely within one sub-block, but no block can be divided

    more than w times. Thus, there are at least b/w forbiddenrectangles.

    TotalCost = k∑

    Bi(Width(Bi) +Height(Bi))

    = k∑

    BiWidth(Bi) + k

    ∑Bi

    Height(Bi)= k(b+ w2q)m2q + kn(w2

    q)= kmw + kmb2q + knw2

    q

    ≤ kwOPT + 8kwOPT 22q + kwOPT 2q

    Again, we let 2q =√OPT and we find that

    Cost = O(kwOPT 3/2)). The result is a O(kw√OPT ))-

    approximation. If OPT ≥ ( nkw )2/3 then we have n/OPT ≤k2/3w2/3n1/3. Again, these are equal in the event that OPT =( nkw )

    2/3. This gives us the final approximation bounds of

    O(min(k2/3w2/3n1/3, kwOPT 1/2)).We now place a bound on k for prefix Diplomat to get our

    approximation bounds.

    Theorem 10. Given a 2w × 2h strip rule pattern with aneffective grid of size m × n with 1 ≤ m ≤ 2w, 1 ≤ n ≤ 2hand w ≥ h, prefix Diplomat with a rotational greedy scheduleand optimal in-row resolver requires at most 2w(m+n) rules.

    Proof: We prove this by induction on q = m + n. As abase case, if either m = 1 or n = 1, then the single row orcolumn can be filled with at most w rules per cell. This costseither wm or wn rules.

    Now assume that this is true for all smaller q. Since P isa strip-rule pattern, there must exist a monochromatic row or

    column in the effective grid. Assume for generality that it is a

    row. Diplomat could use at most 2w− 2 rules to describe thisrow. If it chooses otherwise, it must pick another row which

    costs less to remove. Since we are using an in-plane resolver,

    the resulting pattern must only contain rows in the original

    pattern. Thus the subgrids of P ′ are a subset of those in P . Assuch, there can be no forbidden rectangles and P ′ is also a striprule pattern. The total cost then is at most 2w(q−1)+2w−2 =2w(n+m)− 2 ≤ 2w(n+m).Corollary 2. Prefix-Diplomat with the rotational

    greedy scheduler and optimal in-plane resolver is a

    O(min(w4/3n1/3, w2OPT 1/2))-approximation algorithm.

    Proof. This follows directly from Theorems 9 and 10.

    This is a little smaller than the original bounds of

    O(w2 min(n1/3, OPT 1/2)) presented in [2]. The same ap-proximation bounds can be achieved by using their maximal

    pick-up sticks algorithm as the strip solver for our revised

    iterated strip solver algorithm.

    C. Comparison with Firewall Compressor, TCAM Razor and

    ACL Compressor

    We now show that on two-dimensional patterns, Diplomat

    with a dynamic programming scheduler produces a smaller

    rule list than Firewall Compressor, TCAM Razor and ACL

    Compressor.

    Theorem 11. Given some two-dimensional pattern P , let Dbe the rule list produced by range-Diplomat with a dynamic

    programming scheduler and F be the rule list produced byFirewall Compressor. Then for all P , we have |D| ≤ |F |.

    Proof: We prove this by induction on the number of rows

    m in P . As a base case where m = 1, a 1 × n pattern takesthe same number of rules for both Diplomat and Firewall

    Compressor as both use an optimal one-dimensional solver.

    We now consider patterns P with m ≥ 2. Both Diplomat andFirewall Compressor eliminate one row from P to create anm − 1 row pattern. Let R∗ be the row removed by FirewallCompressor with cost k and P ′ be the resulting m − 1row pattern. The key observation is that Diplomat considers

    removing all rows including R∗; more precisely, Diplomatconsiders merging R∗ with each of its neighboring rows usingrules that only color R∗. When Diplomat considers removingrow R∗, its cost c for doing so is at most k because one optionthat Diplomat considers is coloring all of R∗. Furthermore,because Diplomat uses an in-plane resolver, the resulting m−1row pattern is also P ′. By our inductive hypothesis, Diplomatwill color P ′ with smaller total cost than Firewall Compressorwill color P ′. Thus, one choice for Diplomat with dynamicprogramming has cost no more than Firewall Compressor’s

    cost. Since Diplomat chooses the lowest cost solution, the

    result follows.

    Theorem 12. Given some two-dimensional pattern P , let Dbe the rule list produced by prefix-Diplomat with a dynamic

    programming scheduler and R be the rule list produced byTCAM Razor. Then for all P , we have |D| ≤ |R|.

    Proof: This proof uses the same basic ideas as the proof

    of Theorem 11. Diplomat considers removing rows in the same

    order that TCAM Razor does, and a merge never costs more

    rules than solving a row. This places an upper bound on |D|to be |R| and the result follows.

  • 12

    Theorem 13. Given some two-dimensional pattern P , let Dbe the rule list produced by mixed-Diplomat with a dynamic

    programming sheduler and A be the rule list produced by ACLCompressor. Then for all P , we have |D| ≤ |A|.

    Proof: This proof is essentially the same as for Theorems

    11 and 12. As before, Diplomat considers the same order

    to remove rows as ACL Compressor does, and merging

    rows never costs more than solving a row. Thus, this choice

    for Diplomat is no worse than ACL Compressor and since

    Diplomat takes the best possible option the result follows.

    Theorems 11, 12, and 13 do not necessarily hold when we

    include redundancy removal and other post-processing steps.

    For example, while Theorem 11 guarantees that Diplomat

    with a dynamic programming scheduler will produce a range-

    ACL with no more rules than Firewall Compressor before

    performing redundancy removal, no such guarantee can be

    made after performing redundancy removal. In our experi-

    ments, we observe that Firewall Compressor, TCAM Razor

    and ACL Compressor rarely but occasionally do outperform

    Diplomat with a dynamic programming scheduler due to this

    phenomenon.

    D. Runtime Efficiency

    We compute the worst-case runtime of the Diplomat algo-

    rithm with a dynamic programming scheduler and in-plane

    resolvers given a d-dimensional classifier with d ≥ 2. Weassume each field has size n. We analyze the algorithm inthree steps: computing the child merges, finding an optimal

    schedule, and solving the final plane.

    There are O(n) possible planes. We compute a solution foreach plane as part of the merge process which only needs

    to be done once per plane, not per merge. The total runtime

    will then be at least O(n) times the runtime for a lower-dimensional solution. Range and prefix Diplomat will have

    different solutions, covered later.

    There are O(n2) pairs of planes to try merging together.When d = 2, the planes are rows and are a special casefor both range and prefix Diplomat. When d > 2, thisrequires clipping one of the sub-solutions against a bounding

    box. Finding a minimum bounding box requires finding all

    of the differences between two planes. This can be done

    in O(nd−1) time [9]. Afterwards, it takes O(dn) time tocalculate the boundary and O(dn) time to clip the lowerdimensional solution against the box. Altogether, each merge

    takes O(nd−1) time, so all merges take O(nd+1) time.The runtime of the scheduler is not dependent on the number

    of dimensions. Thus, the total runtime is O(nd+1) plus O(n)times the runtime of a lower-dimensional solution.

    1) Range Diplomat: We now find the merge cost for range

    Diplomat when d = 2. For each row, we must make a callto Firewall Compressor, which takes O(n3) time (for O(n4)time total) [11]. This will be used as part of the computation

    for each merge. Each merge requires determining which other

    subsolution works best for each right end point. Since there

    are O(n) choices, O(n) end points, and O(n2) merges, thetotal merge cost is O(n4).

    We now derive the time required to compute the optimal

    schedule. We have n3 subproblems of the form (i, j, k) where

    we merge rows from i to j and k is the original row thatremains. There must have been a final merge that produced

    (i, j, k). There are O(n) choices for the partition point thatdefines this last merge. One half will include row k. For theother half, we have O(n) choices for the original row thatremains. Thus, it takes O(n2) time to compute the optimalschedule for (i, j, k). Altogether, this leads to O(n5) time tocompute the optimal schedule for our original problem. This

    sets the overall runtime to be O(n5) when d = 2.For range Diplomat, the total runtime when d = 2 is

    O(n5) = O(nd+3). By induction, the runtime for rangeDiplomat with d ≥ 2 is O(nd+3). This is slightly worsethan Firewall Compressor which requires O(nd+2) time in theworst case.

    2) Prefix Diplomat: For prefix Diplomat with d = 2,we make a single call to Suri’s algorithm for each row.

    Each call takes O(kw2n) time (or O(kw2n2) total) where kis the number of colors [18]. Each merge contains O(wn)subproblems: for prefix p, is it better to use Suri’s partialsolution for p or to join the solutions for subproblems p0 andp1? Deciding each subproblem takes a constant amount oftime. Thus each merge takes O(wn) time or O(wn3) time tocompute the cost of all possible merges.

    For the dynamic prefix scheduler, we have O(wn2) sub-problems of the form (p, k) where we merge prefixes p0 andp1 and row k remains. Unlike the range version, there is onlyone possible partition point. Again, one half will contain row

    k. There are then O(n) options for the original row remainingin the other half. Altogether, this leads to O(wn3) time tocompute the optimal schedule.

    For d = 2, the overall cost is O(kw2n2 + wn3) whichis approximately O(wn3) when k ≪ n. By induction, theruntime of prefix Diplomat with d ≥ 2 is O(wnd+1) in theworst case. This is slightly worse than TCAM Razor which

    requires O(kw2nd) time in the worst case.

    VI. EXPERIMENTAL RESULTS

    In this section we evaluate the effectiveness of Diplomat on

    real-life classifiers. Specifically, we assess how much Diplomat

    improves over Firewall Compressor [11], TCAM Razor [15],

    and ACL compressor [13], the current state of the art algo-

    rithms for range-, prefix-, and mixed-ACL compression. For

    Diplomat, we use the dynamic programming scheduler (using

    the solve-then-clip variant) and in-plane resolver appropriate

    to the problem. We label the different applications RDip for

    range-ACL compression, PDip for prefix-ACL compression,

    and MDip for mixed-ACL compression. We do not report

    results using a greedy scheduler (or consequently the general

    resolver) since this was determined experimentally to always

    be worse than the dynamic programming scheduler.

    A. Methodology

    All Diplomat variants, Firewall Compressor (FC), Razor,

    and ACL Compressor (AC) are sensitive to the ordering of the

    five packet fields (source IP address, destination IP address,

    source port, destination port, and protocol). We run these

    algorithms using each of the 5! = 120 different permutationsacross all of the fields and use the best of the 120 results

  • 13

    for each classifier. We also use redundancy removal [8] as a

    post-processing step.

    We now define the metrics for measuring the effectiveness

    of our Diplomat variants. Let f and g denote compressors andL denote a classifier. Let f(L) denote the resulting rule listobtained by applying compressor f to L, and |L| is the numberof rules in L. The compression ratio of f on L is |f(L)||L| , and

    the improvement ratio of f over g on L is |g(L)|−|f(L)||g(L)| . Inour results, we focus on the mean compression ratio and mean

    improvement ratio given a set of classifiers S. It is desirableto have a low compression ratio and a high improvement ratio.

    We assess the performance of our algorithms using a set of

    40 real-life classifiers which we first preprocess by removing

    redundant rules. Most of them have five fields (source and

    destination address, source and destination port, and protocol)

    although some only use a subset of these. We divide this set

    into three smaller sets based on the number of rules after

    running redundancy removal. The small set contains the 17

    smallest classifiers, the middle set contains the next 12 larger

    classifiers, and the large set contains the 11 largest classifiers.

    The classifiers in the large set all have at least 600 rules after

    redundancy removal, with the largest having more than 7600

    rules. For range and prefix tests, we treat all of the fields as

    ranges or prefixes, respectfully. For mixed tests, we treat the

    port fields as ranges and the other fields as prefixes.

    In general, larger classifiers also have larger effective grids.

    Not all of the fields strictly increase as the list size goes up.

    In particular, the protocol field only uses a few distinct values

    across all classifiers, usually 6 (TCP) and 17 (UDP). As such,

    the smallest dimension size is at most 9 for all classifiers. The

    range of the largest dimension size for each set can be seen

    in Table I.

    TABLE ISIZE OF LARGEST DIMENSION

    Min Median Max

    Small 5 14 37Medium 24 66 139

    Large 55 506 1805

    B. Compression Results

    Our results show that (i) RDip, PDip, and MDip outperform

    Firewall Compressor, Razor, and ACL Compressor, respec-

    tively, and that (ii) the performance gap between Diplomat

    and its competitors increases as the classifiers grow in size and

    complexity. Diplomat’s superior performance manifests itself

    in two ways. First, as classifiers grow in size and complexity

    from small to medium to large, the percentage of classifiers

    where Diplomat outperforms its competitors increases (Figure

    9). For range classifiers it increases from 5.9% for the small

    classifiers to 58.3% for the medium classifiers to 100% for

    the large classifiers. Second, again as classifiers grow in size

    and complexity, Diplomat’s average improvement ratio grows

    (Figure 10). For range Diplomat, it increases from 0.6% for

    the small classifiers to 14.4% for the medium classifiers to

    34.9% for the large classifiers over Firewall Compressor.

    It is especially encouraging that Diplomat performs best

    on the large classifier set as the large classifiers are the ones

    that most accurately represent the complex classifiers we are

    likely to encounter in the future and which most need new

    compression techniques. More specifically, we hypothesize

    that for the small classifiers, there is relatively little room for

    optimization and all the methods are finding optimal or near

    optimal classifiers. However, as the classifiers increase in size

    and complexity, Diplomat is able to find new compression

    opportunities missed by the earlier methods. More detailed

    compression ratio results for RDip, PDip and MDip are shown

    in Figures 6, 7, and 8, respectively.

    C. Efficiency

    We implemented our algorithms using a combination of

    C# and VB.Net. Tests were performed on a Xeon E5620

    processor at 2.4 GHz running Red Hat Enterprise Linux 6.1

    and Mono 2.10.8. We report the time for the best permutation

    for each method.

    Diplomat does require significantly more time to compress

    a classifier than both Firewall Compressor and Razor as can be

    seen in Figure 11. This is because Diplomat does explore more

    possibilities when compressing classifiers. RDip is slower

    than PDip because there are more possible cases to consider

    when any range can be used as opposed to fixed prefixes. In

    future work, we hope to find ways to maintain Diplomat’s

    compression advantage while reducing its running time.

    VII. CONCLUSIONS

    In this paper, we presented Diplomat, an algorithm for

    compressing ACLs. We first presented the general framework

    and then presented concrete algorithms for implementing this

    framework. In particular, we presented a greedy scheduler

    that has guaranteed approximation bounds and a dynamic

    programming scheduler that performs well in practice. We

    implemented Diplomat and conducted side-by-side compar-

    ison with the prior Firewall Compressor, Razor, and ACL

    Compressor algorithms on real life classifiers. The experimen-

    tal results show that Diplomat outperforms them most of the

    time, often by a considerable margin. In particular, on our

    largest ACLs, Diplomat has an average improvement ratio of

    34.9% over Firewall Compressor on range-ACLs, 14.1% over

    Razor on prefix-ACLs, and 8.9% over ACL Compressor on

    mixed-ACLs. Furthermore, we generalized the Iterated Strip

    Rule algorithm in [2] and used it to create a larger class of

    algorithms that achieve the same approximation bounds. We

    showed that Diplomat with a greedy scheduler is a member

    of this class.

    ACKNOWLEDGEMENTS

    This work is supported by the National Science Foundation

    under Grant Numbers CNS-0916044, CNS-1017588, CNS-

    1017598, and CCF-1347953, and the National Natural Sci-

    ence Foundation of China under Grant Numbers 61472184,

    61321491, and 61272546. We thank the anonymous referees

    for their constructive comments.

    REFERENCES

    [1] Topcoder statistics. http://community.topcoder.com/tc?module=Static&d1=match editorials&d2=srm150. Accessed 27/7/2012.

  • 14

    Small Medium

    Large

    All

    Classifier Size

    0

    10

    20

    30

    40

    50

    60

    70

    80

    Compression Ratio (%)

    FC

    DDip

    Fig. 6. Compression ratios on range classifiers

    Small Medium

    Large

    All

    Classifier Size

    0

    10

    20

    30

    40

    50

    60

    70

    80

    Compression Ratio(%)

    Razor

    PDip

    Fig. 7. Compression ratios on prefix classifiers

    Small Medium

    Large

    All

    Classifier Size

    0

    10

    20

    30

    40

    50

    60

    70

    80

    Compression Ratio (%)

    AC

    MDip

    Fig. 8. Compression ratios on mixed classifiers

    Small Medium

    Large

    All

    Classifier Size

    0

    20

    40

    60

    80

    100

    Classifiers Improved (%)

    GDip vs FC

    DDip vs FC

    PDip vs Razor

    MDip vs AC

    Fig. 9. Percentages of ACLs that Diplomat out-performs previous methods on

    Small Medium

    Large

    All

    Classifier Size

    0

    5

    10

    15

    20

    25

    30

    35

    Improvement Ratio (%)

    GDip vs FC

    DDip vs FC

    PDip vs Razor

    MDip vs AC

    Fig. 10. Mean improvement ratios

    Classifier

    10

    0

    10

    1

    10

    2

    10

    3

    10

    4

    10

    5

    Time (s)

    FC

    RDip

    Razor

    PDip

    Fig. 11. Runtime of each algorithm on the 8 largestclassifiers

    [2] David A. Applegate, Gruia Calinescu, David S. Johnson, HowardKarloff, Katrina Ligett, and Jia Wang. Compressing rectilinear picturesand minimizing access control lists. In Proceedings of the ACM-SIAMSymposium on Discrete Algorithms (SODA), January 2007.

    [3] P. Berman and B. DasGupta. Complexities of efficient solutions ofrectilinear polygon cover problems. Algorithmica, 17:331–356, 1997.

    [4] Richard Draves, Christopher King, Srinivasan Venkatachary, and BrianZill. Constructing optimal IP routing tables. In Proceedings of the IEEEINFOCOM, pages 88–97, 1999.

    [5] Kirill Kogan, Sergey Nikolenko, William Culhane, Patrick Eugster, andEddie Ruan. Towards efficient implementation of packet classifiers insdn/openflow. In Proceedings of the Second ACM SIGCOMM Workshopon Hot Topics in Software Defined Networking, HotSDN ’13, pages 153–154, New York, NY, USA, 2013. ACM.

    [6] V. S. Anil Kumar and H. Ramesh. Covering rectilinear polygons withaxis-parallel rectangles. In Proceedings of the ACM Symposium onTheory of Computing (STOC), pages 445–454, 1999.

    [7] Alex X. Liu and Mohamed G. Gouda. Diverse firewall design. InProceedings of the International Conference on Dependable Systems

    and Networks (DSN-04), pages 595–604, June 2004.

    [8] Alex X. Liu and Mohamed G. Gouda. Complete redundancy detectionin firewalls. In Proceedings of the 19th Annual IFIP Conference onData and Applications Security, LNCS 3654, pages 196–209, August2005.

    [9] Alex X. Liu and Mohamed G. Gouda. Diverse firewall design. IEEETransactions on Parallel and Distributed Systems, 19:1237–1251, 2008.

    [10] Alex X. Liu, Chad R. Meiners, and Eric Torng. TCAM Razor: Asystematic approach towards minimizing packet classifiers in TCAMs.IEEE/ACM Transactions on Networking, 18(2):490–500, 2010.

    [11] Alex X. Liu, Eric Torng, and Chad Meiners. Firewall compressor:An algorithm for minimizing firewall policies. In Proceedings of the27th Annual IEEE Conference on Computer Communications (Infocom),Phoenix, Arizona, April 2008.

    [12] Alex X. Liu, Eric Torng, and Chad R. Meiners. Compressing networkaccess control lists. IEEE Transactions on Parallel and DistributedSystems, 22(12):1969 – 1977, December 2011.

    [13] Alex X. Liu, Eric Torng, and Chad R. Meiners. Compressing networkaccess control lists. IEEE Transactions on Parallel and DistributedSystems, 22:1969–1977, 2011.

    [14] W. J. Masek. Some NP-complete set covering problems. 1978.

    [15] h̃ttp://razor.sourceforge.net/ Razor. Date of access: February 7, 2005.

    [16] Yu-Wei Eric Sung, Xin Sun, Sanjay G. Rao, Geoffrey G. Xie, and

    David A. Maltz. Towards systematic design of enterprise networks.IEEE/ACM Trans. Netw., 19(3):695–708, June 2011.

    [17] Subhash Suri, Tuomas Sandholm, and Priyank Warkhede. Compressingtwo-dimensional routing tables. Algorithmica, 35:287–300, 2003.

    [18] Subhash Suri, Tuomas Sandholm, and Priyank Ramesh Warkhede.Compressing two-dimensional routing tables. Algorithmica, 35(4):287–300, 2003.

    [19] Minlan Yu, Jennifer Rexford, Michael J. Freedman, and Jia Wang.Scalable flow-based networking with difane. In Proceedings of theSIGCOMM, 2010.

    James Daly received his B.S. degree in Engineer-ing and Computer Science from Hope College in2008. He is currently a Ph.D. candidate in computerscience at Michigan State University. His researchinterests are algorithms, networking, and security.

    Alex X. Liu received his Ph.D. degree in computerscience from the University of Texas at Austin in2006. He received the IEEE & IFIP William C.Carter Award in 2004 and an NSF CAREER awardin 2009. He received the Withrow DistinguishedScholar Award in 2011 at Michigan State University.He is an Associate Editor of IEEE/ACM Transac-tions on Networking. He received Best Paper Awardsfrom ICNP-2012, SRDS-2012, and LISA-2010. Heis the TPC Co-Chair of ICNP 2014. His researchinterests focus on networking and security.

    Eric Torng received his Ph.D. degree in computerscience from Stanford University in 1994. He is cur-rently an associate professor and graduate directorin the Department of Computer Science and Engi-neering at Michigan State University. He received anNSF CAREER award in 1997. His research interestsinclude algorithms, scheduling, and networking.