Efficient Resource Allocation for Protecting Coral Reef Ecosystems

Yue Yin1,2, Bo An31The Key Lab of Intelligent Information Processing, ICT, CAS2University of Chinese Academy of Sciences, Beijing, China

3School of Computer Science and Engineering, Nanyang Technological University, Singapore1melody1235813@gmail.com, 3boan@ntu.edu.sg

AbstractCoral reefs are valuable and fragile ecosystemswhich are under threat from human activities likecoral mining. Many countries have built marineprotected areas (MPAs) and protect their ecosys-tems through boat patrol. However, it remains asignificant challenge to efficiently patrol the MPAsgiven the limited patrol resources of the protectionagency and potential destructors’ strategic actions.In this paper, we view the problem of efficiently pa-trolling for protecting coral reef ecosystems from agame-theoretic perspective and propose 1) a newStackelberg game model to formulate the problemof protecting MPAs, 2) two algorithms to computethe efficient protection agency’s strategies: CLP inwhich the protection agency’s strategies are com-pactly represented as fractional flows in a network,and CDOG which combines the techniques of com-pactly representing defender strategies and incre-mentally generating strategies. Experimental re-sults show that our approach leads to significantlybetter solution quality than that of previous works.

1 IntroductionCoral reefs are precious natural resources, which form someof the world’s most productive ecosystems, providing com-plex marine habitats that support a wide range of other or-ganisms. However, some human activities, like coral mining,can severely damage the coral reef ecosystems. Once coralreefs are destroyed, it may take tens of years for them to re-store. Therefore, many countries have built marine protectedareas (MPAs) to restrict potentially damaging activities by pa-trolling in the MPAs [Bellwood et al., 2004]. It is a greatchallenge to efficiently protect the MPAs through patrollingsince protection agencies usually have to protect a large openwater area using very limited resources (e.g., the protectionagency in the Yalongwan MPA in China protects an 85 squarekilometers area with 3 patrol boats). In addition, potential de-structors can learn the protection agency’s strategies throughsurveillance, then choose the most undetectable time and themost covert path in the open water to arrive at a specific areato perform illegal activities. We aim at developing efficientpatrol strategies for protecting the coral reef ecosystems.

Though the crisis faced by coral reefs has been investigatedby many researchers [Bellwood et al., 2004; Pandolfi et al.,2003], most previous works focus on conservation planninginstead of detecting and deferring potential damage [Car-wardine et al., 2009]. Meanwhile, there has been signifi-cant progress on applying game theoretic approaches to secu-rity domains like protection of infrastructures [Tambe, 2011;Letchford and Conitzer, 2013; An et al., 2013; Yin et al.,2014; 2015; Wang et al., 2016; Shieh et al., 2012]. In ourscenario, the interaction between the protection agency (de-fender) and the potential destructor (attacker) can also bemodeled as a game, but previous work cannot be directly usedhere due to two new challenges. First, the playfield is a largeopen water area, both players’ strategies are time-dependentpaths, i.e., the defender patrols while the attacker choosessome time to sail to his target area. Second, unlike activitiessuch as igniting a bomb which can be done quickly, damag-ing activities at an MPA (e.g., coral mining) only succeed ifthey last for a relatively long time. Most previous works as-sume that at most one player takes paths [Fang et al., 2015;Basilico et al., 2009], or that time is irrelevant [Jain et al.,2013] and attack can be done immediately [Gan et al., 2015].For previous works that considered attack duration, they ei-ther consider time duration of attacks as external parame-ters but not part of the attacker’s strategy [Alpern et al.,2011], or have different goals from us [Yin et al., 2012;Bosansk et al., 2015]. The two new challenges make the strat-egy spaces of the players larger and more complicated, whichleads to great challenge in computation.

This paper makes four key contributions. First, we proposea defender-attacker Stackelberg game model to formulate theproblem of protecting MPAs, in which both game players taketime-dependent paths, and payoffs of players are affected bythe time duration of the attack. Second, we propose a com-pact linear program to solve the game, in which we compactlyrepresent defender strategies as fractional flows on graphs toreduce the number of variables in the game. To further scaleup the algorithm, our third contribution is a compact-strategydouble-oracle algorithm on graphs (CDOG) which combinesthe techniques of compactly representing defender strategiesand incrementally generating strategies. Finally, extensiveexperimental results show that our algorithms lead to signifi-cantly better solution quality than that of other algorithms inthe literature and CDOG scales up well.

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16)

531

2 Motivating ScenarioFigure 1(a) shows the landscape of the Great Barrier ReefMarine Park in Australia. There is an authority (defender) re-sponsible for the protection of the park, with offices locatedon the coast, shown as red stars in Figure 1(a). The defender,can divide the MPA into several zones to design patrol strate-gies. Figure 1(b) shows an example division. The defendercan stay at a zone or take a path among zones. Each zoneis a potential attack target. The effect of damaging differ-ent zones can be different and time-dependent. The attackerenters the park at some time, takes a path to his target zoneand determines how long to perform activities at this zone.Both agents may be limited to start and finish the path atcertain zones, e.g., office locations, peripheral zones in thepark (zones except 6 in Figure 1(b)). Strategic attackers canobserve the defender’s patrol strategies, then act consideringboth defender’s strategies and attractiveness of zones. Sinceperforming activities at a zone needs time-consuming prepa-ration (e.g., equipment setup), we assume that the attackertargets a single zone. We did not consider how the attackerescapes after attacking since the coral reefs are damaged any-way after the attack. In addition, before the attacker finishesthe attack, he needs to look for a location and operate theequipment, which can make him suspicious; while after theattack, he can easily camouflage himself and flee fast, whichmakes it difficult to catch him.

(a) Landscape and offices (b) Example division

Figure 1: The Great Barrier Reef Marine Park

3 ModelWe model the problem as a Stackelberg game [Tambe, 2011],where the defender commits to a randomized strategy first,then the attacker conducts surveillance and chooses the op-timal strategy to respond to the defender’s strategy. We firstconstruct a transition graph with a timeline. Denote an MPAas a collection of n zones Z = {1, 2, · · · , n}. We evenly dis-cretize a day as a sequence of ⌧ time points t = ht1, · · · , t⌧ iwith interval �. Assume that the time needed to travel be-tween two adjacent zones is a multiplier of � (this assumptionholds as long as � is small enough). Assume that the defenderand the attacker travel at the same speed and they only moveat time points tk 2 t. Let D = hdij 2 {1, 2, · · · }i with dijrepresenting that the time needed to move from zone i to ad-jacent zone j is dij · �. To represent players’ strategies, weconstruct a directed transition graph G = hV,Ei where a ver-tex v = hi, tki corresponds to zone i and time tk. There is anedge e = hv = hi, tki, v0 = hj, tk0ii if one of the followingtwo conditions holds:

1. j = i, k0 = k + 1. We call such edges stay edges.2. i and j are adjacent zones and k0 = k + dij . We call

such edges moving edges.Consider a simple MPA graph in Figure 2(a) which in-

cludes 3 zones. Let t = ht1, t2, t3i and dij = 1, 8i, j 2{1, 2, 3}, i 6= j. We can get a transition graph in Figure 2(b).The edge between h1, t1i and h1, t2i indicates that the de-fender can patrol in zone 1 and the attacker can perform ac-tivities in zone 1 during (t1, t2). An edge connecting h1, t1iand h2, t2i indicates that if a player moves from zone 1 at timet1, he will arrive at zone 2 at time t2.

(a) Divided MPA (b) Transition graph

Figure 2: MPA graph and transition graph

Defender strategies. Assume that the defender has mresources, i.e., m patrol boats, and each resource can keeppatrolling for time duration ✓�. Let Zd ⇢ Z be zonesthat the defender can start and end her patrol1. Since thepatrol can start at any time before t⌧�✓, we add a collec-tion of virtual source vertices to the transition graph, i.e.,S = hS1, S2, · · · , S⌧�✓i. For Sk 2 S, we add an edge fromSk to vertex hi, tki(8i 2 Zd). Similarly, we add a collectionof virtual terminal vertices T = hT1, T2, · · · , T⌧�✓i such that8Tk 2 T , there is an edge from vertex hi, tk+✓i(8i 2 Zd) toTk. Therefore, a patrol strategy Pr of a resource r is a flowfrom Sk to Tk. A pure strategy of the defender is a set of m‘patrol strategy flows’, i.e., P = {Pr : r 2 {1, · · · ,m}}.A mixed strategy of the defender is a distribution over purestrategies, i.e., x = hxP i where xP represents the probabilityof P being used.

Consider the example in Figure 2(b). Assume that Zd ={1, 3} and ✓ = 1, thus the source vertices and terminal ver-tices can be added as is shown in Figure 3(a), i.e., S1 isconnected to h1, t1i and h3, t1i, meaning that the defendercan start the patrol from zone 1 or 3 at time t1, while T1 isconnected to h1, t2i and h3, t2i, meaning that patrols startingfrom S1 should end in zone 1 or 3 at time t2. Any flow fromSk to Tk is a feasible patrol strategy, e.g., S2 ! h1, t2i !h3, t2i ! T2 represents that the defender patrols on the wayfrom zone 1 to zone 3.

Attacker strategies. Assume that the attacker can alsostart at a subset of the zones Za ⇢ Z. An attacker’s strategyincludes two parts: a path in the transition graph to go to histarget zone, and how long he attacks at the target zone. Weassume that the attack duration is a multiplier of � (this holdswhen � is small enough) and denote an attacker’s strategy asY = hHY , AY i, where HY is a path leading to his target ver-tex hi, tki and AY is a path consisting of l adjacent stay edges,representing that the attacker performs activity at zone i from

1Our model can be easily expanded to handle cases in which thedefender starts from and finishes at different sets of zones.

532

(a) Defender strategies (b) Attacker strategies

Figure 3: Strategies as paths in the transition graph

time tk to time tk+l. Consider the previous 3-zones example.Assume that Za = {1}. Figure 3(b) shows a feasible attackerstrategy where the attacker enters the MPA from zone 1 att1, arrives at his target zone 2 at t2, then attacks zone 2 tillt3. As previous work in security games, we restrict attackerstrategies to pure strategies [Kiekintveld et al., 2009].

Utilities and equilibrium. We assume that each stay edgee = hhi, tki, hi, tk+1ii in the transition graph has a value ofVe, representing the attacker’s payoff of succesfuly perform-ing activities in zone i during time (tk, tk+1). If an attackersuccessfully plays strategy Y = hHY , AY i, he gains a utilityof U(Y ) =

Pe2AY Ve, while the defender gets a utility of

�U(Y ). If the attacker is detected by the defender, he failsand both agents get a utility of 0. The attacker may be de-tected by the defender if their strategies share same edges.Specifically, if their paths share a stay edge, the attacker maybe detected when he is staying at some zone; if their pathsshare a moving edge, the attacker may be detected when heis moving from one zone to another. Naturally, the defendermay not be able to perform detection every time she meets aboat which could be a potential attacker. To describe the prob-ability that the defender detects a boat, we make the followingtwo assumptions, which are realistic enough to describe mostreal world scenarios.Assumption 1. If the defender and the attacker first meet onedge e, the probability that the defender detects the attacker,i.e., the detecting factor of edge e, is fe 2 [0, 1].

The next assumption is about the probability that the de-fender detects the attacker after they meet for the first time.Since patrol areas are usually somewhere prone to be mali-ciously damaged but not popular travel sites, a boat appearingin such areas frequently is very suspicious. Therefore, if thedefender has met a boat for many times on her way before shearrives at edge e, then the boat seems more suspicious than aboat which is seen for the first time on edge e, thus should bedetected with a higher probability than fe.Assumption 2. Assume that the defender and the attackerhave been on same edges e1, e2, · · · , ek�12 and the attackerhas not been detected. If they meet on edge ek, the probabilitythat the defender detects the attacker is min{1, fek

1�Pk�1

i=1 fei}.

The probability shown in Assumption 2 ranges in [fek , 1],which satisfies the intuition that more suspicious boat shouldbe detected with a higher probability. Based on Assump-tion 2, if the defender’s strategy and the attacker’s strat-

2The subscripts are indexes of edges in the players’ path.

egy share same edges e1, e2, · · · , ek�1, ek, let Fk�1 rep-resent the probability that the attacker is detected on anyedge in e1, · · · , ek�1, then the total probability that the at-tacker is detected given the defender strategy is Fk�1 + (1�Fk�1)min{1,

fek1�

Pk�1i=1 fei

}. Note that when k = 2, F1 =fe1 . Simple recursion results in that given a patrol strategyPr of a resource r and an attacker strategy Y , assume that theoverlapping edges of Pr and Y are Pr\Y = he1, e2, · · · , eki,then the probability that the attacker is detected by this re-source is min{1,

Pki=1 fek}.

We also assume that when several security resources andthe attacker are at the same edge, the security resources cancooperate with each other to come up with a detection prob-ability which is the sum of their respective detection proba-bility capped by 1.3 Therefore, given a pure strategy of thedefender P = {Pr} and a pure strategy of the attacker Y , theoverall detection probability for the attacker is:

dp(P, Y ) = min{1,Xm

r=1

Xe2Pr\Y

fe}. (1)

Based on Eq.(1), given a pair of strategies hP, Y i, theexpected utility of the attacker is Ua(P, Y ) = (1 �dp(P, Y ))U(Y ). Given a mixed strategy x = hxP i of thedefender and a pure strategy Y of the attacker, the attacker’sexpected utility is Ua(x, Y ) =

PP xPU

a(P, Y ), while the

expected utility of the defender is Ud(x, Y ) = �Ua(x, Y ).Our goal is to compute the Stackelberg equilibrium of the

game, hence the optimal strategy for the defender. Given thezero sum assumption, the Stackelberg equilibrium is equiv-alent to maximizing the defender’s utility when the attackerresponds the best. Technically, let X, Y be the defender’sand attacker’s strategy space respectively. Let the attacker’soptimal response function be f(x) = {Y 2 Y}. A pair ofstrategies (x, Y ) form an equilibrium if they satisfy the fol-lowing:

Ua(x, f(x)) � Ua(x, Y 0), 8Y 0 2 Y,Ud(x, f(x)) � Ud(x0, f(x0)), 8x0 2 X.

4 CLP Based on Compact RepresentationThe number of defender’s pure strategies increases exponen-tially as the game size increases. To address this challenge,we compactly represent mixed patrol strategies by marginalcoverage ce on edges e in the transition graph G, i.e., the ex-pected number of patrollers that will be on the edges. Givena mixed strategy x, we have

ce =X

PxPP (e), 8e 2 G, (2)

where P (e) represents the number of patrols in pure strat-egy P which go through edge e. Therefore, based on Eq.(2),given a mixed strategy x and its corresponding coverage vec-tor c = hce : e 2 Ei, and given a pure strategy Y of the

3If the security resources work independently, the overall detec-tion probability will be higher than that is computed by Eq. (1). Inthis case, the defender strategies computed based on Eq. (1) willlead to a utility which is a lower bound of the defender.

533

attacker, the expected attacker utility can be represented as

Ua(c, Y ) = (1�min{1,X

e2Ycefe})U(Y ). (3)

Our problem now lies in computing the marginal cover-age which corresponds to the optimal mixed strategy of thedefender. First, we need to construct marginal coverages cor-responding to feasible mixed strategies. One challenge of theconstruction lies in that a mixed defender strategy consistsof pure strategies starting from different time points. Yin etal. [2012] have proven that the problem can be solved by con-structing an extended version EG of the transition graph G,and considering the marginal coverage as the sum of severalflows on EG. Technically, EG is composed of multiple re-stricted copies of G (i.e., subgraphs of G), corresponding todifferent possible starting time points for the defender. Forthe copy corresponding to starting time points tk, we onlykeep the subgraph Gk on vertices hSk, Tk, v = hi, tk0i : i 2Z, k0 2 {k, · · · , k+✓}i. Therefore, any defender patrol strat-egy starting at tk can be represented as an Sk � Tk flow insubgraph Gk. Let zk(e) represent the expected number ofpatrollers on edge e which come from patrol strategies start-ing at time point tk (zk(e) � 0). Let �(e) represent the set ofsubgraphs which include edge e. Let ce =

Pk:Gk2�(e) zk(e).

Yin et al. [2012] show that if zk(e) satisfies conservation offlow, a defender mixed strategy which leads to the same util-ity as corresponding c = hce : e 2 Gi can be constructed inpolynomial time. Now we present a compact linear programCLP to compute the optimal marginal coverage.

CLP(G,Y) : maxc,z U (4)

U �(1�min{1,X

e2Ycefe})U(Y ), 8Y 2 Y (5)

ce =X

k:Gk2�(e)zk(e), 8e 2 G (6)

Xhv0,vi2Gk

zk(hv0, vi) =X

hv,v00i2Gkzk(hv, v00i), 8Gk (7)

XSk2S

Xi2Zd

zk(hSk, hi, tkii) = m (8)X

Tk2T

Xi2Zd

zk(hhi, tk+✓i, Tki) = m (9)

Constraint (7) enforces conservation of flow, which isclearly satisfied by any mixed patrol strategy. Constraints (8)and (9) bound the total flow entering and exiting the transitiongraph by m, the number of patrollers. Constraint (5) indicatesthat the attacker will best respond by choosing the strategyY which leads to the best utility, or equivalently, the leastutility for the defender �(1 � min{1,

Pe2Y cefe})U(Y ).

This is an overestimate of the defender’s utility since theexpression 1 � min{1,

Pe2Y cefe} is an overestimate of

the probability that the attacker is detected. This is be-cause min{1,

Pe2Y cefe} only caps the expectation (over

its pure strategies) of the detection probability at 1, but al-lows a pure strategy P = hPri in its support to achievePm

r=1

Pe2Pr\Y fe > 1, whereas according to Eq.(1), the

detection probability of each pure strategy should be at most1. As a result, the solution of this LP provides an upper boundof the optimal defender utility. Fortunately, once we generate

Algorithm 1: CDOG1 Initialize with a subgraph G0 of G and a subset Y0 ⇢ Y;2 repeat3 (c,y) CLP(G0,Y0);4 P DO(y), Y AO(c) ;5 G0 G0 [ {P},Y0 Y0 [ {Y };6 until G0 and Y0 are not expanded;7 return (c,y)

the patrols from the marginals we are able to compute the ac-tual best-response utilities of the attacker. Our experimentsshow that the differences between the actual utilities and theupper-bounds given by the LP formulation are small.

5 CDOG AlgorithmThe number of constraints in Eq.(5), being the same as thenumber of attacker strategies, increases exponentially withthe game size. This leads to the poor scalability of theLP formulation. To deal with the scalability issue, we pro-pose CDOG, a compact-strategy double-oracle algorithm ongraphs. CDOG is based on the widely used double oracleframework (e.g., [Jain et al., 2011]). The main idea is to findan equivalent small-size sub-game to avoid solving the orig-inal exponentially large game. Specifically, the frameworkstarts from solving a sub-game involving only a very smallsubset of each player’s pure strategy set. The solution ob-tained, being an equilibrium of the sub-game, is not neces-sarily an equilibrium of the original game since the playersmay have incentive to deviate by choosing strategies not inthe current sub strategy sets. Thus the framework expandsthe players’ strategy sets based on the current equilibriumand gets a larger sub-game. The process is repeated untilno player can benefit from expanding their strategy sets. Itusually ends with a sub-game with reasonable (instead of ex-ponential) size, and the final solution is provably also an equi-librium to the original game (McMahan et al. 2003).

A traditional double oracle framework can take a longtime to converge for large games, since sub-game is ex-panded slowly, while once it gets large, solving a sub-gameis also very time-consuming. For example, Jain et al’s algo-rithm [Jain et al., 2013] solves games with around 200 ver-tices, corresponding to a 10 zones, 20 time points case in ourmodel, in around 9 hours. To scale it up, CDOG exploitsthe graph structure of our problem, uses a subgraph of thetransition graph (instead of a pure strategy set) to character-ize the defender’s strategy space, and solves each sub-gamethrough compactly representing defender strategies as cover-age on edges in the subgraph. The main structure of CDOGis depicted in Algorithm 1. Here Line 1 initializes the sub-game with a random subgraph G0 of G and a random subsetY

0 of attacker’s pure strategies. Using the LP formulation,Line 3 computes the equilibrium of the sub-game, where de-fender patrols on G0 and attacker plays with strategies in Y0.Notably, c = hcei is the solution to LP, while y is an attackermixed strategy over pure strategies in Y0, which is obtainedfrom the dual variable associated with Constraints in Eq.(5).Then through Lines 4–6, CDOG implements the core of dou-

534

ble oracle framework by calling two oracles—defender or-acle (DO) and attacker oracle (AO)—to obtain the players’best responses for expanding the sub-game. Next, we presentin detail how these two oracles are implemented.

Defender oracle. The defender’s best response is an m-unit flow on G, which maximizes her utility against the cur-rent attacker strategy y. DO computes a compact represen-tation of the best response pure strategy with the followingMILP.

maxc,z

XY 2y

�Yi(1�min{1,X

e2Ycefe})U(Y ) (10)

zk(e) 2 {0, 1, · · · ,m} 8zk(e) (11)Constraints (6)–(9). (12)

Constraint (11) ensures that the coverages on edges are in-tegers, so that coverage vector c corresponds to a pure strat-egy P . Constraints (6) - (9) can be directly used here since theconservation of flow is the same for pure strategies as that formixed strategies in CLP. The objective of DO is to maximizethe defender’s expected utility given attacker’s mixed strat-egy y = hYii where Yi indicates the probability that the ithattacker pure strategy in Y0 is used. The value of Yi comesfrom the dual variable of the ith inequality corresponding toConstraint (5) of CLP(G0,Y0). If the pure defender strategycorresponding to c computed by DO goes through edges notin G0, we expand G0 by including these edges and associatedvertices.

Attacker oracle. The attacker oracle cannot be efficientlysolved by an MILP since we cannot compactly represent theattacker strategies. AO computes the attacker’s best responsethrough three steps: First, for each v 2 V , we compute theattacker’s optimal path H(v) to v. Following H(v), the at-tacker is the least likely to be detected among all paths lead-ing to v. Second, based on H(v), we compute the optimaltime duration for the attacker to perform activities if he startsthe activity at v. Finally, choose the vertex, path, and timeduration which together lead to the optimal attacker utility.We now introduce the details of the three steps.

For step (1), let �(v) =P

e2H(v) ce represent the probabil-ity of being detected if the attacker sails through path H(v).Apparently, if v = hi, tki and i 2 Za, then the attacker candirectly enter the transition graph at vertex v. Thus H(v) onlyconsists of vertex v and �(v) = 0. If i /2 Za, let ⇤(v) rep-resent the set of vertices v0 2 G such that hv0, vi is an edgein G, then the attacker has to arrive at some v0 and take edgehv0, vi to arrive at vertex v. Therefore, we have

�(v) = minv02⇤(v){1, H(v0) + chv0,vifhv0,vi}. (13)

Let vpre = argminv02⇤{H(v0) + chv0,vi}, thus H(v) =H(vpre)[hvpre, vi, and H(v) can be computed by the recur-sive function shown in Algorithm 2. Note that to find H(v)for all vertices, every edge in the transition graph only needsto be visited at most once, thus the time complexity of step(1) is O(|E|).

For step (2), if the attacker starts to perform activity atvertex v = hi, tki and chooses a time duration l, the totalprobability of being detected depends on �(v) and the prob-ability of being deleted when performing the activity, i.e.,

Algorithm 2: Find optimal path (c, v)1 Input: c, v; Output: H(v), �(v));2 if i 2 Za then H(v) = v, �(v) = 0 ;3 else4 ⇤(v) predecessors of v, pre �1, min 1;5 for v0 2 ⇤(v) do6 if H(v0) + chv0,vifhv0,vi < min then7 min = H(v0) + chv0,vifhv0,vi, pre = v

0;

8 H(pre), �(pre) Find optimal path (c, pre);9 H(v) H(pre) [ hpre, vi, �(v) min;

◆(v, l) = min{1, 1 � �(v) �P

j2{1,2,···l} cejfej}. Hereej = hhi, tk+j�1i, hi, tk+jii. The value of performing theactivity is the sum of values of edges ej , i.e., `(v, l) =P

j2{1,2,···l} Vej . Thus the expected utility is U(v, l) =(1 � ◆(v, l)) · `(v, l). Therefore, the optimal time durationfor vertex v is loptv = argmaxl2{1,··· ,⌧�k} U(v, l).

To find the optimal time duration for each vertex v =hi, tki, we need to iterate all possible time durations for v,i.e., {1, · · · , ⌧ � k}. Thus the time complexity of Step (2)is O(n⌧2). The third step is to straightforwardly choosethe vertex v leading to the optimal attacker utility, i.e., v =argmaxv02V U(v0, l

optv0 ), and construct the attacker strategy

Y based on H(v) and l. If strategy Y is not in the current setY

0, we expand Y0 by adding Y .

6 Experimental EvaluationWe evaluate the proposed algorithms in terms of (1) solu-tion quality, (2) scalability, and (3) robustness. The CLP issolved by Knitro 9.0.0. Each point in the figures is the aver-age value over 30 sample games. We test the algorithms ongraphs based on the geology of the Great Barrier Reef Ma-rine Park as is shown in Figure 1. Given that the MPA canbe divided differently due to different purposes [Watts et al.,2009], in each game in the experiments, we generate an MPAgraph based on a random division of the park. We randomlygenerate the time needed to move between adjacent zones,i.e., dij , in [1,

|t|2 ], where |t| is the number of time points in

the game. Transition graphs are then constructed based on theMPA graphs and dijs, in which each zone has |t| copies. Werandomly choose the value of each stay edge in (0, 100].

Solution quality. Solution quality of algorithms is mea-sured by attacker utility. Given the zero sum assumption,higher attacker utility indicates lower defender utility. Wecompare our algorithms with two baseline algorithms ANPand AND. ANP assumes that the attacker does not take paths,but directly attacks anywhere at any time instantaneously,which is similar as in Fang et al [2013]. AND assumes thatthe attacker can take paths in the graph to arrive at a target,but still attacks instantaneously, which is similar as in Yinet al [2012]. The baseline algorithms are under extra assump-tions since no previous algorithms can exactly solve our prob-lems. We assume 3 patrollers, 9 zones and divide the timelineinto 12 points unless otherwise specified.

In Figures 4(a), 4(b) and 4(c), the y-axis indicates the at-tacker utility, while the x-axis indicates the patrol duration of

535

a defender’s resource (i.e., the value of ✓), the number of start-ing zones for the defender, and the number of starting zonesfor the attacker respectively. Since CLP and CDOG lead tothe same attacker utility, their results are represented by onesingle bar. The solution quality of CLP and CDOG is signifi-cantly better than that of ANP and AND despite the value ofthe three parameters. It is unsurprising that as the patrol dura-tion increases (Figure 4(a)) or the number of defender’s start-ing zones increases (Figure 4(b)), the attacker utilities com-puted by all algorithms decrease. As the number of attacker’sstarting zones increases (Figure 4(c)), ANP’s performance isnot affected since it does not consider attacker’s paths, whileother algorithms show an increasing trend in attacker utilities.

Figure 4(d) depicts the percentage of the true optimal de-fender utility v.s. the theoretical upper bound returned byCLP and CDOG. The x-axis indicates the maximum valueof the detecting factor fe, i.e., 0.3 indicates that fe is ran-domly chosen in (0, 0.3]. Eq.(1) indicates that with smallerfe, CLP and CDOG are less likely to overestimate the detec-tion probability. Figure 4(d) shows a decreasing trend in thepercentage of the true defender utility v.s. CLP and CDOG’sresults. Fortunately, even when fe = 1, the percentage isstill around 90%, indicating that the upper bound computedby CLP and CDOG is very close to the true utilities.

0

200

400

600

3 4 5 6 7 8

Optim

al att

acker u

tility

Patrol duration

CLP&CDOG ANP AND

(a) Increase patrol duration

0

200

400

600

1 2 3 4 5 6

Optim

al att

acker u

tility

# of defender's starting zones

CLP&CDOG ANP AND

(b) Increase defender’s starters

0

200

400

600

1 2 3 4 5 6

Optim

al att

acker u

tility

# of attacker's starting zones

CLP&CDOG ANP AND

(c) Increase attacker’s starters

0%

20%

40%

60%

80%

100%

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

% of Upp

er bo

und

Detecting factor

(d) Percentage of upper bound

Figure 4: Solution quality

Scalability. In Figures 5(a) and 5(b), the y-axis indicatesruntime while the x-axis respectively indicates the number ofzones and time points. In both figures, the runtime of CLPshows a much more obvious increasing trend. Actually, CLPcannot solve games with more than 20 targets and 20 timepoints due to the RAM limit, while CDOG can solve gameswith 40 targets in around 6 minutes and games with 50 timepoints in less than 2 minutes. We also evaluate the runtimeof the defender oracle, attacker oracle, and CLP in CDOGin detail. Figure 6 shows an example of the runtime of thethree parts in the CDOG algorithm, which solves a game after10 iterations. The runtime of the CLP shows an increasingtrend. The size of the DO and the AO is barely affected bythe iteration, thus their runtime does not change much.

Robustness. We first consider observation noise of theattacker. We add 0-mean Gaussian noise with standard de-viations chosen randomly from U [0, 0.5] to the coverage on

0

100

200

300

400

5 10 15 20 25 30 35 40

Run

time (

secon

ds)

# of zones

CLP CDOG

(a) Increase targets

0

30

60

90

120

10 15 20 25 30 35 40 45 50

Run

time (

secon

ds)

# of time points

CLP CDOG

(b) Increase time points

0

200

400

600

3 4 5 6 7 8

Optim

al att

acker u

tility

Patrol duration

CLP&CDOG ANP AND

(c) Observation noise

0

200

400

600

3 4 5 6 7 8

Optim

al att

acker u

tility

Patrol duration

CLP&CDOG ANP AND

(d) Payoff noise

Figure 5: Scalability and robustness

each edge observed by the attacker. Figure 5(c) shows theattacker utilities considering observation noise for the sameclass of games considered in Figure 4(a). Compared withFigure 4(a), all algorithms lead to lower attacker utilities inFigure 5(c) since the observation noise prevents the attackerfrom responding the best. CLP and CDOG still significantlyoutperform ANP and AND. We also consider payoff noise ofthe defender. For each stay edge in the transition graph, weadd the same Gaussian noise as the previous setting to thedefender’s knowledge on the value of the edge. Figure 5(d)considers the class of games in Figure 4(a) with payoff un-certainties. Compared with Figure 4(a), all algorithms lead tohigher attacker utilities in Figure 5(d) since the payoff noiseaffects the defender’s judgement on the attacker’s action. Theadvantage of CLP and CDOG over ANP and AND is stillsignificant.

Iteration

1 2 3 4 5 6 7 8 9 10

CLP 4301 5172 5637 6131 6151 6231 6332 6300 6449 6547

DO 1626 1620 1650 1652 1648 1715 1639 1620 1640 1634

AO 328 352 322 323 326 319 320 334 318 320

Figure 6: Runtime (ms) of CLP and oracles in CDOG

7 ConclusionThis paper models the problem of patrolling MPAs to pro-tect coral reef ecosystems as a defender-attacker Stackelberggame, in which both players’ strategies are time-dependentpaths, and the payoffs are affected by the duration of the at-tack. We propose a linear program (CLP) to solve the game,in which defender strategies are compactly represented asflows on graphs. We also propose a more scalable algorithm,CDOG, which combines techniques of compactly represent-ing defender strategies and incrementally generating strate-gies. Experimental results show that our algorithms lead tosignificantly better solution quality than that of baseline algo-rithms, and the CDOG algorithm can scale up.

536

References[Alpern et al., 2011] Steve Alpern, Alec Morton, and Kate-

rina Papadaki. Patrolling games. Operations research,59(5):1246–1257, 2011.

[An et al., 2013] Bo An, Matthew Brown, Yevgeniy Vorob-eychik, and Milind Tambe. Security games with surveil-lance cost and optimal timing of attack execution. InProceedings of The 12th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS), pages223–230, 2013.

[Basilico et al., 2009] Nicola Basilico, Nicola Gatti, andFrancesco Amigoni. Leader-follower strategies for roboticpatrolling in environments with arbitrary topologies. InProceedings of The 8th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS), pages57–64, 2009.

[Bellwood et al., 2004] David R Bellwood, Terry P Hughes,C Folke, and M Nyström. Confronting the coral reef crisis.Nature, 429(6994):827–833, 2004.

[Bosansk et al., 2015] Branislav Bosansk, Albert Xin Jiang,Milind Tambe, and Christopher Kiekintveld. Combin-ing compact representation and incremental generation inlarge games with sequential strategies. In Proceedingsof the 29th Conference on Artificial Intelligence (AAAI),pages 812–818, 2015.

[Carwardine et al., 2009] Josie Carwardine, Carissa J Klein,Kerrie A Wilson, Robert L Pressey, and Hugh P Possing-ham. Hitting the target and missing the point: Target-basedconservation planning in context. Conservation Letters,2(1):4–11, 2009.

[Fang et al., 2013] Fei Fang, Albert Xin Jiang, and MilindTambe. Optimal patrol strategy for protecting moving tar-gets with multiple mobile resources. In Proceedings of the12th international conference on Autonomous agents andMultiagent systems (AAMAS), pages 957–964, 2013.

[Fang et al., 2015] Fei Fang, Peter Stone, and Milind Tambe.When security games go green: Designing defender strate-gies to prevent poaching and illegal fishing. In Proceed-ings of the 24th International Joint Conference on Artifi-cial Intelligence (IJCAI), pages 2589–2595, 2015.

[Gan et al., 2015] Jiarui Gan, Bo An, and Yevgeniy Vorob-eychik. Security games with protection externality. In Pro-ceedings of the 29th Conference on Artificial Intelligence(AAAI), pages 914–920, 2015.

[Jain et al., 2011] Manish Jain, Dmytro Korzhyk, OndrejVanek, Vincent Conitzer, Michal Pechoucek, and MilindTambe. A double oracle algorithm for zero-sum securitygames on graphs. In Proceedings of The 10th Interna-tional Conference on Autonomous Agents and MultiagentSystems (AAMAS), pages 327–334, 2011.

[Jain et al., 2013] Manish Jain, Vincent Conitzer, and MilindTambe. Security scheduling for real-world networks. InProceedings of the 12th international conference on Au-tonomous agents and multi-agent systems (AAMAS), pages215–222, 2013.

[Kiekintveld et al., 2009] Christopher Kiekintveld, ManishJain, Jason Tsai, James Pita, Fernando Ordóñez, andMilind Tambe. Computing optimal randomized resourceallocations for massive security games. In Proceedings ofThe 8th International Conference on Autonomous Agentsand Multiagent Systems (AAMAS), pages 689–696, 2009.

[Letchford and Conitzer, 2013] Joshua Letchford and Vin-cent Conitzer. Solving security games on graphs viamarginal probabilities. In Proceedings of the 27th AAAIConference on Artificial Intelligence (AAAI), pages 591–597, 2013.

[Mcmahan et al., 2003] H. Brendan Mcmahan, Geoffrey J.Gordon, and Avrim Blum. Planning in the presence ofcost functions controlled by an adversary. In Proceedingsof the 20th International Conference on Machine Learning(ICML), pages 536–543, 2003.

[Pandolfi et al., 2003] John M Pandolfi, Roger H Brad-bury, Enric Sala, Terence P Hughes, Karen A Bjorn-dal, Richard G Cooke, Deborah McArdle, Loren Mc-Clenachan, Marah JH Newman, Gustavo Paredes, et al.Global trajectories of the long-term decline of coral reefecosystems. Science, 301(5635):955–958, 2003.

[Shieh et al., 2012] Eric Anyung Shieh, Bo An, Rong Yang,Milind Tambe, Craig Baldwin, Joseph DiRenzo, BenMaule, and Garrett Meyer. PROTECT: An application ofcomputational game theory for the security of the ports ofthe United States. In Proceedings of the 27th AAAI Confer-ence on Artificial Intelligence (AAAI), pages 2173–2179,2012.

[Tambe, 2011] Milind Tambe. Security and Game Theory:Algorithms, Deployed Systems, Lessons Learned. Cam-bridge University Press, 2011.

[Wang et al., 2016] Zhen Wang, Yue Yin, and Bo An. Com-puting optimal monitoring strategy for detecting terroristplots. In Proceedings of the 30th Conference on ArtificialIntelligence (AAAI), 2016.

[Watts et al., 2009] Matthew E Watts, Ian R Ball, Romola SStewart, Carissa J Klein, Kerrie Wilson, Charles Stein-back, Reinaldo Lourival, Lindsay Kircher, and Hugh PPossingham. Marxan with zones: Software for optimalconservation based land-and sea-use zoning. Environmen-tal Modelling & Software, 24(12):1513–1521, 2009.

[Yin et al., 2012] Zhengyu Yin, Albert Xin Jiang,Matthew P. Johnson, and Milind Tambe. TRUSTS:Scheduling randomized patrols for fare inspection intransit systems. AI Magazine, 33(4):59–72, 2012.

[Yin et al., 2014] Yue Yin, Bo An, and Manish Jain. Game-theoretic resource allocation for protecting large publicevents. In Proceedings of the 28th Conference on Artifi-cial Intelligence (AAAI), pages 826–834, 2014.

[Yin et al., 2015] Yue Yin, Haifeng Xu, Jiarui Gan, Bo An,and Albert Xin Jiang. Computing optimal mixed strategiesfor security games with dynamic payoffs. In Proceedingsof the 24th International Joint Conference on Artificial In-telligence (IJCAI), pages 681–687, 2015.

537