A Bayesian Framework for Nash EquilibriumInference in Human-Robot Parallel Play

Shray Bansal, Jin Xu, Ayanna Howard, Charles IsbellGeorgia Institute of Technology

Email: {sbansal34, jxu81, ah260}@gatech.edu, isbell@cc.gatech.edu

Abstract—We consider shared workspace scenarios with hu-mans and robots acting to achieve independent goals, termed asparallel play. We model these as general-sum games and constructa framework that utilizes the Nash equilibrium solution conceptto consider the interactive effect of both agents while planning.We find multiple Pareto-optimal equilibria in these tasks. Wehypothesize that people act by choosing an equilibrium basedon social norms and their personalities. To enable coordination,we infer the equilibrium online using a probabilistic model thatincludes these two factors and use it to select the robot’s action.We apply our approach to a close-proximity pick-and-place taskinvolving a robot and a simulated human with three potentialbehaviors - defensive, selfish, and norm-following. We showedthat using a Bayesian approach to infer the equilibrium enablesthe robot to complete the task with less than half the numberof collisions while also reducing the task execution time ascompared to the best baseline. We also performed a study withhuman participants interacting either with other humans or withdifferent robot agents and observed that our proposed approachperforms similar to human-human parallel play interactions. Thecode is available at https://github.com/shray/bayes-nash

I. INTRODUCTION

People often perform activities in shared spaces with otherpeople achieving their own individual goals. This includesdriving to work while sharing the road with other cars, navi-gating around other shoppers when pushing a cart in a grocerystore, and sharing counter-space and utensils in a kitchen.Although, these situations are neither purely collaborative norcompetitive, however, the actions of other participants havebearing on each person’s own success or failure. We referto these activities as parallel play, related to its psychologynamesake that refers to activities in early social development,where children play besides instead of with, other children [20,19]. In the Human-Robot Interaction (HRI) context, we defineparallel play to refer to those activities where people androbots have separate individual goals but interact due to sharedspace. We aim to derive a framework that helps a robot planeffectively for parallel play with human participants, and applyit to a close-proximity pick-and-place scenario between a robotand a human.

Planning a robot’s action in HRI usually involves consid-ering the robot’s goal as well as predictions of future humanactions [23, 1, 11]. When working with others, people areoften considerate of their intents and beliefs due to Theory-of-Mind [21, 5], and so, the human’s action is influenced by theirpredicted plans of the other participant’s, including the robot.Modeling this cyclical-dependence, of the human’s predicted

plan on the robot’s and vice-versa, is important for accuratelyrepresenting the interaction dynamics in HRI.

Game Theory provides us tools to model this inter-dependence of rational interacting agents. The Nash equi-librium (NE) is a set of actions, one for each agent in thegame, which is optimal, assuming the actions of others remainfixed [13]. A Nash equilibrium implicitly captures the inter-dependence between agents, and our approach plans by findingequilibria to enable better coordination.

Although humans have been shown to play to Nash equi-librium [15], we find that multiple equilibria can exist ina game and it is not clear how the robot should choosebetween them. Figure 1 shows an example of two equilibriain a pick-and-place scenario, where each favors a differentagent by allowing them to reach their goal first. A collisionis likely if both agents choose an equilibrium that favorsthem, highlighting the importance of coordination. Humanscan coordinate social behavior in non-competitive games bylearning and following social norms [10]. Here, a norm refersto a set of abstract instructions that agents follow, and expectothers to follow. In Figure 1, a norm might favor solutionsthat allow the agent closer to their goal to reach for it firstand, if followed by both agents, would lead to coordination.However, sometimes agents can ignore the norm in favor oftheir personal preferences. For example, a selfish agent mightignore the norm and expect the other to always yield to themwhen reaching for an object. Coordinating with such agentsrequires the ability to infer this preference from experience.

We design a framework that finds Nash equilibria for par-allel play tasks; it models the strategy for choosing equilibriaas a distribution composed of two aspects - (1) a domain-specific social norm, designed by an expert apriori and (2) anagent-specific individual preference, inferred online during theinteraction. We hypothesize that this framework would lead tobetter coordination with humans in performing in such tasks,due to its modeling of the decision-making coupling betweenagents, as well as, its combination of expert knowledge withonline adaptation. To validate, we apply this to a close-proximity pick-and-place task, designed to be similar to HRItasks used to study team coordination and fluency [7, 16] witha simulated human. Our results show that this framework isable to shorten task execution time while also reducing thenumber of human collisions by half as compared to the bestbaseline when interacting with a simulated human with 3potential personalities.

(a) Nash Equilibirum (NE) Action Sequences

(b) Cost Matrix

Fig. 1. A scenario where two agents, P1 and P2, move the red and black blocks to their respective goal locations. In (a), we show two sequences of actionsstarting in the same state at t0, and ending in the same goal state at tend. While each action sequence (trajectory) is a Nash Equilibrium (NE), they both favordifferent agents. NE1 (top) favors P2 by allowing it to reach its goal first, NE2 (bottom) favors P1. A cost function is illustrated in (b), where each cell is atuple containing the cost of the actions for P1 and P2 respectively. Here, our action space has only 2 actions per agent, A1 = {a11, a21} and A2 = {a12, a22}.

We make the following contributions:1) Introduce a novel framework that models norm-

following social behavior and personality-based likeli-hood inference to interactive-planning with humans inparallel play activities. We do this by first computingthe Nash equilibria and then using this framework tofind a distribution over them.

2) Design task and metrics to benchmark the performanceof interactive-planning algorithms for parallel play. Thisincludes 3 baselines for simulating distinct human per-sonalities that have similarity to human performanceof the task, which enables testing adaptability of thealgorithms to different human behavior.

II. RELATED WORK

There has been extensive work in HRI for planning a robotto work on tasks around humans in domains like parts assem-bly [9, 7], motion planning, and autonomous driving [23, 1].Planning around people generally involves two aspects, pre-dicting the human’s behavior and finding robot actions thatachieve its goal in the presence of the human.

Human modeling. Prior work has placed emphasis onaccurately modeling the human’s rational goal-driven behavior.This includes learning the human’s preferences to predict low-level trajectories through a reward function obtained by inversereinforcement learning [29, 23] or high-level decision-makingand choice of goal or the timing of actions for part deliveryusing a probabilistic task model [9, 8]. Some models were alsoadaptable in modeling the preferences of the particular humanwith whom the robot was interacting [17].

Human adaptive planning. A common approach to plan-ning for this interactive setting is by predicting the human’sbehavior and then finding a best-response to this behavior.This approach works very well in scenarios where the robotassumes an assistive role like parts delivery [9, 28] where,apart from ensuring that the human doesn’t wait, the robot

intends to avoid interactions by keeping out-of-the-way. It hasalso been used to plan a robot to pick-up objects in a close-proximity scenario where the robot planned trajectories thatdid not intersect with predicted human plans [16, 14]. Aninherent assumption, here, is that, although the human’s plandepends on the situation, the prediction is independent of therobot’s plan. So, in situations where the agents have equalroles, like driving or navigation, the robot will choose overlyconservative behaviors which can lead it to freeze when tryingto navigate crowds [26] or fail to merge in traffic [23].

Mutually adaptive planning. Recent work has addressedthis by considering the human’s influence-ability as well. Theirmodel includes both the influence of the human and theirgoals on the robot and the influence of the robot on thehuman. Similar to us, they also utilize game-theoretic toolsto model this cyclical influence. Turnwald and Wollherr [27]modeled robot navigation as a dynamic general-sum gameand computed a Nash equilibrium to effectively plan therobot’s trajectory among a crowd of pedestrians. Sadigh et al.[23] modeled driving as a Stackelberg game where the robotplanned first and the human planned in response; they showedthat this model can successfully influence human behaviorin simulated driving tasks. Fisac et al. [6] extended this tolonger time horizons by computing a Nash equilibrium forhigh-level actions and optimizing low-level trajectories forexecuting them. Gabler et al. [7] utilized the Nash equilibriumto find an order for object pick-up in a close-proximity pick-and-place task similar to ours; they found that consideringthe mutual adaptation allowed their framework to improvesafety as well as human subjective preference. Our approachalso uses the Nash equilibrium to plan goal-driven actions forthe robot that consider the mutual adaptability between thetwo agents. However, our approach includes a strategy forselecting an equilibrium in case multiple are present, whileothers either have not mentioned this strategy [7] or only findone equilibrium due to their problem structure [23, 6].

Online model inference. Although different people performthe same task in different ways, prior work uses a singlemodel to describe all human users, with some exceptions.While in [17], Nikolaidis et. al. cluster human behavior intomultiple types and predict actions based on the inferred type,in [18], they group people by their adaptability to the robot’sactions. Chen et al. [3] explicitly model human trust on therobot’s ability during decision-making and infer this parameterduring the interaction. Sadigh et al. [22] use information-gathering actions to infer whether a driver is attentive or notand use this to coordinate better. We define a latent variablethat represents the human personality and infers this online,for each participant, in order to find plans that coordinate wellwith the human. Recently, Schwarting et al. [24] proposeda method for simulated autonomous driving around humandrivers using Nash equilibrium. Similar to us, they havea parameter that represents human-personality and infers itonline. However, in their model, this parameter is part ofthe human cost function while we use it to choose betweenNash equilibria. Also, while they define a continuous action-space and use a local approximation for computing Nashequilibrium, which finds a single equilibrium, we find multipleequilibria since we plan discrete high-level actions. Theirresults indicate that inferring human preference in combinationwith finding Nash equilibrium improves prediction and helpsachieve coordination with humans.

Coordination. The importance of coordinating with ahuman in non-competitive games was highlighted by Carrollet al. [2] where they learn human models and used them totrain reinforcement learning agents that achieve performancesuperior to self-play. Similar to us, Ho et al. [10] proposedusing social norms for improving coordination in multi-agentenvironments. While our approach does not use learning, it canbe incorporated into our framework to replace the user-definedcosts or norms.

III. INTERACTIVE PLANNING BY NASH EQUILIBRIUM

We model the multi-agent interactive planning task as a non-cooperative game represented as tuple G, G = (P,A, c) [13].Here, P = {P1, ..., PN} is a finite set of N players, A =A1 × ... × AN where Ai is the set of actions available toplayer i. We refer to the set of concurrent actions, one foreach agent, as an action profile, a, a = (a1, ..., aN ). We definea cost representing the unfavorability of an action profile foragent i as ci : A 7→ R and c = (c1, ..., cN ) includes themapping for all agents.

In our scenario, each agent p is a robot arm, each actionset Ap is a set of goal-driven trajectories, each trajectory isa sequence of joint-space positions and velocities sampledusing a planner, and the cost ci(a) encourages each robot tominimize task completion time while avoiding collisions withother agents. The goal for an agent i is to take an actionai ∈ Ai in profile a, which minimizes its cost. However,its cost depends upon the actions chosen by the other agentsin the profile a. We assume that all agents are rational andhave Theory-of-Mind, i.e., they choose actions to minimize

their own cost and are aware of the states and goals of theother agents. These assumptions allow us to utilize the NashEquilibrium (NE) solution concept for this game. An actionprofile is a NE, for a single-stage game, if no agent has anincentive to choose a different action for themselves given thatall the other actions are fixed.

a*i∈ argmin

ai

ci(a*1, ., ai, ., a*

N) ∀i ∈ N. (1)

Although generally, only one (mixed) equilibrium is guar-anteed to exist for a game [13], in our problem, one pure equi-librium is always present and we find that multiple equilibriaare frequently present. For the planning agent, some of theseequilibria can be eliminated for being Pareto-sub-optimal, i.e.,worse for all agents. For example, Nash profile, a*1 Pareto-dominates a profile a*

2, if ci(a*1) < ci(a*

2)∀i ∈ N . Next,

we present an approach that can help the agent select an actionby choosing between Pareto-optimal equilibria.

IV. OPTIMAL EQUILIBRIUM SELECTION

Our strategy chooses between equilibria using two aspectsof human social behavior, norm-following and personality-adaptation. We model the distribution over equilibria as a prod-uct of its probability under the norm, pn and its probabilitygiven the predisposed personality, pα,

p(a) = pn(a)pα(a). (2)

Here, and in the rest of this section, a refers to a NE actionprofile. Next, we explain the norm for this problem and howwe use observations to update the personality distribution.

A. Norm

Similar to [10], we define a norm to be a set of, situation-dependent, abstract, instructions that agents follow with theexpectation that others will follow them as well. They helpagents coordinate in the absence of prior knowledge of theagents they are interacting with. For example, a first-come-first-leave norm can help decide how cars navigate a four-way stop. Here, we model it as a probability distribution overNE. Different games will have different norms and the choiceof a norm should be based on expert knowledge or learnedfrom data. For our problem, we use a simple min-norm, thatprioritizes the equilibrium which achieves minimum cost forany of the agents,

pn(a) ∝ e−λnmini(ci(a)), (3)

where λn is a parameter of the exponential distribution thatwe set. This norm it encourages the agent with the shortestunobstructed path to its goal to act first.

B. Online preference estimation

Although norms can help in coordination, people sometimeshave strong preferences that guide them towards certain equi-libria regardless of the norms. For example, an aggressivedriver may decide to cross an intersection first, despite thenorm, expecting the other drivers to adapt their strategy. We

model this as a distribution over equilibria, inferred at time tusing the history Ht of the past interaction,

ptα(a) = p(a|Ht). (4)

Here, Ht refers to the history of the interaction, i.e., Ht ={({s0i∈N}), ..., ({s

t−1i∈N})}, and sti is the state of agent i at time

t. We set it to the uniform distribution at the start,

pt=0α (a) = uniform(a) ∀a. (5)

We define an exponential distribution on the distance be-tween a past trajectory, H , to an action profile, a,

p(a|H) ∝ e−λαfdist(a,H), (6)

where fdist(a,H) is defined as the euclidean distancebetween the sequence of states in H to those sampled at thesame time increments from a and λα is a parameter of theexponential distribution. Although, we would like to use Eq. 6to compute p(at|Ht), however, the distance of past trajectoriesto current action profiles is not meaningful because the laststate, st, in the history is the first state of all action profiles attime t. So, we only use Eq. 6 to find p(a0|Ht) and define alatent variable θ to help infer p(at|Ht). We use θ to representa personality-based distribution over equilibria, p(at|θ), andassume that it remains constant for every agent during aninteraction. We derive p(at|Ht) by using the personality, θ,

p(at|Ht) =∑θ

p(at, θ|Ht),

p(at|Ht) =∑θ

p(θ|Ht)p(at|θ,Ht).

We assume that the personality, θ, encodes the informationrequired for predicting the agent’s next action, which makesat conditionally independent of Ht given θ. So,

p(at|Ht) =∑θ

p(θ|Ht)p(at|θ). (7)

We define θ such that each action profile a that belongs toa personality is equally likely to be chosen,

p(a|θ) = 1θ(a)∑a′ 1θ(a

′). (8)

Next, we find p(θ|Ht) by taking its joint distribution witha0 and marginalizing it out,

p(θ|Ht) =∑a0

p(θ, a0|Ht),

p(θ|Ht) =∑a0

p(a0|Ht)p(θ|a0, Ht).

From the conditional independence between θ and Ht givena0, we get,

p(θ|Ht) =∑a0

p(a0|Ht)p(θ|a0). (9)

Since we assume a uniform prior on θ, p(θ) is a constant.Combining with Eq. 8, we get,

p(θ|a0) = p(θ)p(a0|θ)p(θ)

∑θ′ p(a

0|θ′)=

p(a0|θ)∑θ′ p(a

0|θ′)(10)

We use p(θ|a0) and p(a0|Ht) (Eq. 6) to get p(θ|Ht) in Eq.9. This allows us to find p(at|Ht) from Eq. 9 by using p(at|θ)from Eq. 8, which gives us pα(at) in Eq. 4

V. PICK-PLACE TASK

The pick-and-place task involves two 2-dof articulated armsmoving on a 2D surface with the goal to pick up theirdesignated object, by moving their end-effector close to it forgrasping, and placing it, by bringing the grasped object to thedestination area. The scenario is depicted in Figure 2, wherethe arm with a red base was controlled by our approach, andthe other one was either simulated as a human or controlledby a human. Henceforth, the former will be referred to as therobot and the latter as the human.

Fig. 2. Pick-and-place task scenario.

A. Action Planning

To plan for this task, we first sample k − 1 plans foreach agent in configuration space using a Rapidly-exploringRandom Tree (RRT) [12] and add a static plan where theagent does not move, Ai = {τj∈k}. We use them to generatean action set by taking the outer product of the trajectoriesfor each agent, A = A1 × ... × AN . We compute a cost foreach action profile by simulating it and use Eq. 1 to find theNash equilibria. We choose the NE profile a that maximizesthe distribution p(a) from Eq. 2, and select the agent’s actionfrom a. This action, along with the action taken by the human,is executed until a collision is detected or if the time beforereplanning is reached. After this, we update the history Ht

and replan. This process continues until the robot completesthe task and is depicted in Figure 3.

Fig. 3. Framework. We first plan trajectories, then use the cost to computeall Pareto-optimal Nash equilibria, then combine the norm and inferred-personality distributions to select the most likely equilibrium. This action ispartially executed before repeating the whole process until the task completes.

B. Task Costs

We define a simple cost function that encourages the robotto complete the task quickly and avoid collisions. The cost ofan action profile, a = (aR, aH), where aR, aH , are the robotand human actions respectively, is the trajectory duration if itis successful in reaching the goal and infinity if it leads to acollision. We sample goal-reaching trajectories for each agentthat are independent of the goal and state of other agents.We assume that the human is also goal-driven and collision-avoidant and so assume an analogous cost function where theircost depends on the human task completion time. Under theseconditions, one pure Nash equilibrium is guaranteed as longas there exists a trajectory for the robot to reach the goal. Forinstance, say that we select an action profile with the robot’saction being the shortest trajectory to its goal and the human’saction as the shortest trajectory that does not collide with therobot’s plan (including the static action). This profile will bea Nash equilibrium since neither agent has an incentive tomodify their actions. For the robot, the action is optimal, and,for the human, this action is optimal assuming the robot’saction as fixed due to the infinite cost of a collision.

C. Baselines

We define three baselines to compare with our approach.1) Defensive. The robot chooses an action assuming that

the human wants to maximize the robot’s cost whilestill achieving its goal leading to a maximin formulation.Thus, the agent will act defensively by preferring actionsthat do not lead to collision with the sampled humantrajectories and will often lead it to wait for the humanto complete their task.

aR = argminaR∈AR

maxaH∈AH cR(aR, aH). (11)

2) Selfish. Chooses an equilibrium profile that minimizesthe robot’s cost. This strategy selects a trajectory thatreaches the goal as quickly as possible assuming theother agent avoids collision.

aR ∈ a*, a* = argmina*

cR(a*). (12)

3) Norm-Nash. Chooses an equilibrium profile that maxi-mizes the norm distribution pn from Eq. 3. This leadsto behavior that encourages the agent closer to theirgoal to reach them first. While the first two will lead tosomewhat fixed behaviors, this strategy adapts to goal-achievability, which varies across tasks and also in stateevolution within the same interaction.

aR ∈ a*, a* = argmaxa*

pn(a*). (13)

D. Implementation Details

A 3D simulation environment was created using theopen-source Open Robotics Automation Virtual Environment(OpenRAVE) [4] with a time step of 0.1 seconds. The action-set, Ai was sampled using an RRT planner from the open-source Open Motion Planning Library (OMPL) [25]. Wesampled k = 8 plans for each agent when planning andcompute the cost as an k × k table by simulating the actionsusing OpenRAVE with a time-step of 0.8; we increased thetime-step here to allow for fast computation of the nashsolutions. Parameter λn of the norm distribution (Eq. 3) wasset to 50. We set two personality types and use a binary latentvariable θ = {0, 1}. θ = 0 selects equilibrium profiles a thatfavor agent 1, c1(a) < c2(a), and θ = 1 selects equilibriumprofiles a that favor agent 2, c2(a) < c1(a). We set λα = 10in the personality distribution (Eq. 6).

VI. SIMULATED HUMAN STUDY

We simulate human behavior to create a controlled settingfor our first experiment.

A. Simulated Human

We defined three human behaviors using the baselines:(1) Defensive, (2) Selfish-Nash, and (3) Norm-Nash. Wechose the first two behaviors because of their clear intuitivedistinctness and combine it with the third in accordance withour expectation that people also follow social norms.

B. Metrics

We measured the following task performance metrics: totaltask completion time and task time for each agent; we alsocounted safety stops, which are the number of times thesimulation stopped the agents to avoid an impending collision.To keep these measures independent, we did not have any timepenalty for a safety stop.

Fig. 4. Results from the experiment involving a simulated human. Notethat our proposed approach, Bayes-Nash, is safer than all the non-defensivebaselines and similar in task completion time to the Selfish-Nash baseline.Error bars represent standard error of the mean (SEM).

C. Results

To test how each algorithm fares with the different behav-iors, we randomly pair the robot with one of these simulatedhuman behaviors with random object locations for 30 trials.The averaged metrics for the three baselines and our proposedapproach, Bayes-Nash, are presented in Figure 4. As expected,the Defensive robot was the safest, but its safe behavior alsocaused the highest robot and total task completion times. TheSelfish-Nash was significantly faster than the Defensive robotbut also led to the highest safety stops. Both Norm-Nash andBayes-Nash performed comparably in time to Selfish-Nashbut the Bayes-Nash was marginally faster. They were bothsignificantly safer than Selfish-Nash and Bayes-Nash also hadthe fewest safety stops of the two.

D. Analysis

These results illustrate the trade-off between safety andefficiency present in the task, where the Defensive and Selfish-Nash agents sit at opposite extremes. Norm-Nash and Bayes-Nash are able to better trade-off these metrics due to theircapability to adapt to the situation and the (simulated) human,respectively. Next, we perform an experiment to validate thistrade-off in human interaction.

VII. HUMAN-HUMAN STUDY

To investigate the natural interaction between two people,we recruited 4 participants to perform the same task, in a pilotexperiment, where both interacting agents were human andcontrolled the simulated robot arm using a gamepad controller.

A. Experiment Design

We kept one of the human agents fixed throughout theexperiment and will refer to them as control. The other agent(participant) evoked different behaviors in each experimentand performed 3 trials with the control. In the first trial,the participant was asked to behave naturally, by trying to

0

1

2

3

4

5

6

7

Total-Time Control-Time Participant-Time Safety-Stops

Natural Human

DefensiveSelfish

(a) Natural (Control) Human

0

0.5

1

1.5

2

2.5

3

3.5

Total-Time Robot-Time Human-Time Safety-Stops

Bayes-Nash

DefensiveSelfish-Nash

(b) Bayes-Nash

Fig. 5. (a) Plots the interaction task metrics for the naturally acting human inthe presence of either a selfish or defensive participant in the human-humanstudy; (b) The same metrics but for the interaction of Bayes-Nash with theSelfish and Defensive baselines. The similarity in the relative trends across (a)and (b) highlight the similarity of Bayes-Nash to a real human agent. Errorbars represent SEM.

increase efficiency while reducing task time. For the othertwo trials, the participant chose either (1) Selfish - completingthis task efficiently or (2) Defensive - avoiding collisions withthe other arm strategy. The control kept the same naturalstrategy throughout the interactions and was not made awareof the strategy that the participant was employing. Also, noverbal communication was allowed during the experiment.We measured the same metrics as in the simulated humanexperiment.

B. Results

Figure 5 (a) shows that the total task completion times forthe Selfish and Defensive humans are similar when interactingwith a naturally-acting human. However, in terms of theirindividual task completion times, the Defensive agent takessignificantly longer as compared to the Selfish agent. TheSelfish human also triggers more safety stops but the safetystops in this study were much less than in simulation. InFigure 5 (b) we show the simulated Defensive and Selfish-Nash behaviors when interacting with Bayes-Nash. We findsimilar comparative trends between behavior types for bothtask completion times and safety stops. However, the robot in(b) completes the task more quickly since the arm is allowedhigher velocities in simulation.

C. Analysis

These results indicate that a naturally-acting human adaptswell to both Selfish and Defensive behavior due to similar taskmetrics for both conditions, as shown in Figure 5(a). Similartrends for Bayes-Nash, Figure 5(b), indicate that it also adaptswell to different strategies. The similarity between trends

Fig. 6. A user controlling the robot arm during the Human-Robot study.

among personalities across experiments validate our interactivetask design and metrics to benchmark performance for parallelplay. Also, since the latent variable used to parameterize theequilibrium was designed to capture the agent’s favorability inequilibria and not the specific behaviors of the baselines, weexpect that Bayes-Nash to be able to adapt well to real humanparticipants. This leads to the following two hypotheses for ahuman-robot interaction study:

H1: A robot using Bayes-Nash will have significantly fewercollisions than a robot using a Selfish-Nash planner.

H2: A robot using Bayes-Nash will have faster taskcompletion time than a robot using a Defensive planner.

VIII. HUMAN-ROBOT STUDY

We test these hypotheses in a pilot experiment by pairinghuman participants with a robot controlled by our algorithmand the baselines.

A. Experimental Design

In order to validate the proposed approach, we design awithin-subject human study. We examined the effects of threeplanning algorithms on their interaction with a human user andcounter-balanced their order. Participants were asked to controla robot arm in simulation using a gamepad controller, as shownin Figure 6. The gamepad controller allows users to movethe robot arm in eight different directions at 10 Hz. We usedthe same manipulation task as the previous two experiments,including keeping the object locations the same. Participantswere informed that they might interact with different robotsbut not what these types were. There were three rounds ofthe task and each round involved six trials. In each round, therobot used one of the following conditions: (1) Defensive, (2)Selfish-Nash, and (3) Bayes-Nash. We used the same metricsas before.

B. Procedure

After giving informed consent, participants went through anoverview of the experimental procedures. The study startedwith a pre-survey to collect demographic information. Then

Fig. 7. Human-Robot Study. Defensive and Selfish are baselines, Bayes-Nash is our approach and the Natural agent refers to the human-human studyresults where the participants acted naturally. Error bars represent SEM.

participants entered a practice session to familiarize them-selves with the user interface and the gamepad controller. Thepurpose of this session was to erase potential novelty effectscaused by the robot and the user interface. The practice roundended when the participants indicated they felt comfortablewith the control and overall task. Then the participants wentthrough three rounds of ‘pick-and-place tasks’ with the robot.Each round consisted of 6 trials and participants were askedto fill out a short survey after the last trial. After these threerounds, the participants were given a post-survey with open-ended questions about their experience.

C. Results

A total of 6 participants were recruited from a universitycampus and were randomly assigned to one possible com-bination of the experimental conditions. Figure 7 comparesthe performance of the three agents. We also compare themto a naturally acting human by including the results of twonaturally acting humans from the human-human study. Thetotal task completion time was the longest for the Defensiverobot while the other three agents were similar but significantlyfaster confirming hypothesis H2. The Defensive robot alsohad the longest robot task execution time but also led to theshortest time for the human. Bayes-Nash was the least safeand Natural the most, the selfish and defensive conditions hadsimilarly small safety stops, contrary to hypothesis H1.

D. Analysis

It took the Defensive agent 36.2% more time to completethe entire task than the Selfish agent. These results confirmthat the Defensive agent acts in an overly cautious manner.When comparing with the naturally-acting human we alsonoticed that the task completion time for the Bayes-Nashapproach performs almost equally well. However, the safetystops results are surprising in two respects: (1) the highernumber of safety stops for Bayes-Nash as compared to Naturaland other conditions, (2) the much fewer safety stops for allconditions when compared to the simulated human study. Wealso noticed the latter in the human-human study, indicating

Fig. 8. Effect of experience in the Human-Robot study. It shows thecumulative safety stops for the robot averaged over the trials. For theDefensive and Selfish-Nash conditions, the number of stops decreases withmore trials, indicating that the human learns to avoid collision. However, forBayes-Nash, the safety stops first increase and then remain constant, perhapsdue to the difficulty in adapting to an agent whose behavior is not fixed.

that people are better at avoiding collisions as compared tothe robots. We explore potential causes for these findings inthe next section.

IX. DISCUSSION, LIMITATIONS AND FUTURE WORK

Figure 5 shows that the Bayes-Nash approach and thenaturally-acting human are similar in their ability to adapteffectively to different personalities. However, in the human-robot study, Bayes-Nash had the most safety stops which con-tradicts H1. We believe there are two potential explanations.

First, due to human learning effects. Fig. 8 shows that thenumber of safety stops decrease with more trials for boththe Defensive and Selfish-Nash conditions. This indicates thatthe user might have learned a collision-avoidant response tothose agents over time. As for the Bayes-Nash condition, thesafety stops first increase over trials and then remain constant.This might be because those two strategies had a somewhatfixed behavior that the human could easily adapt to. Forexample, if the robot moves to the goal without considerationof the human’s presence every time, the human will learnthat her optimal response is to wait for the robot initially.This is similar to the observation from [23] where the robotdirectly influenced human behavior. Although this led to betterperformance, in our scenario, it is questionable whether thisfixed behavior will be desirable from a robot collaborator inthe real-world. Second, due to the mismatch in human-robotspeed. Although the maximum arm velocities were the samefor both agents, the robot was able to act faster than the human.In the Selfish condition, after a couple of trials, people mighthave realized that it was easier to complete the task by waitingfor the robot. We need further experiments that control forthese variables to confirm these and plan for it in future work.

Although our approach can generalize to more agents, itwas specifically designed for addressing scenarios involvinghuman-robot cohabitation where usually only two agents are

present [17, 7]. The time complexity of our algorithm isexponential in the number of agents, so can become intractablefor large numbers of them. However, this may not be alimitation in real-world scenarios, since, even in cases wheremany agents are present (e.g., driving on a highway), we onlyneed to consider the interactive influence of a few close-bycars to generate human-like behavior [24]. In the future wewould like to test it in the presence of more agents.

The primary contribution of our work is in developinga novel game-theoretic approach for HRI tasks. We alsoinstituted a pilot study to validate this methodology andpresent descriptive statistics of the results. However, the smallsample size did not allow us to run significance tests for thehuman experiments. Our plans for future work include a largerHRI study to provide evidence for generalization to a broadpopulation.

ACKNOWLEDGMENTS

We would like to thank Mustafa Mukadam for discussionsleading to the initial idea of this approach. We would alsolike to acknowledge Himanshu Sahni, Yannick Schroecker andSangeeta Bansal for helpful discussions.

REFERENCES

[1] Shray Bansal, Akansel Cosgun, Alireza Nakhaei, andKikuo Fujimura. Collaborative planning for mixed-autonomy lane merging. In 2018 IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS),2018.

[2] Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths,Sanjit Seshia, Pieter Abbeel, and Anca Dragan. On theutility of learning about humans for human-ai coordi-nation. In Advances in Neural Information ProcessingSystems, pages 5175–5186, 2019.

[3] Min Chen, Stefanos Nikolaidis, Harold Soh, DavidHsu, and Siddhartha Srinivasa. Planning with trust forhuman-robot collaboration. In Proceedings of the 2018ACM/IEEE International Conference on Human-RobotInteraction, pages 307–315, 2018.

[4] Rosen Diankov. Automated Construction of RoboticManipulation Programs. PhD thesis, Carnegie MellonUniversity, Robotics Institute, August 2010.

[5] David Engel, Anita Williams Woolley, Lisa X Jing,Christopher F Chabris, and Thomas W Malone. Readingthe mind in the eyes or reading between the lines?theory of mind predicts collective intelligence equallywell online and face-to-face. PloS one, 9(12), 2014.

[6] Jaime F Fisac, Eli Bronstein, Elis Stefansson, DorsaSadigh, S Shankar Sastry, and Anca D Dragan. Hier-archical game-theoretic planning for autonomous vehi-cles. In 2019 International Conference on Robotics andAutomation (ICRA), pages 9590–9596. IEEE, 2019.

[7] Volker Gabler, Tim Stahl, Gerold Huber, Ozgur Oguz,and Dirk Wollherr. A game-theoretic approach foradaptive action selection in close proximity human-robot-

collaboration. In 2017 IEEE International Conference onRobotics and Automation (ICRA), 2017.

[8] Matthew C Gombolay, Reymundo A Gutierrez,Shanelle G Clarke, Giancarlo F Sturla, and Julie AShah. Decision-making authority, team efficiency andhuman worker satisfaction in mixed human–robot teams.Autonomous Robots, 39(3):293–312, 2015.

[9] Kelsey P Hawkins, Shray Bansal, Nam N Vo, andAaron F Bobick. Anticipating human actions for col-laboration in the presence of task and sensor uncertainty.In 2014 ieee international conference on Robotics andautomation (ICRA), 2014.

[10] Mark K Ho, James MacGlashan, Amy Greenwald,Michael L Littman, Elizabeth Hilliard, Carl Trim-bach, Stephen Brawner, Josh Tenenbaum, Max Kleiman-Weiner, and Joseph L Austerweil. Feature-based jointplanning and norm learning in collaborative games. InCogSci, 2016.

[11] Hema S Koppula and Ashutosh Saxena. Anticipatinghuman activities using object affordances for reactiverobotic response. IEEE transactions on pattern analysisand machine intelligence, 38(1):14–29, 2015.

[12] Steven M LaValle. Rapidly-exploring random trees: Anew tool for path planning. 1998.

[13] Kevin Leyton-Brown and Yoav Shoham. Essentials ofgame theory: A concise multidisciplinary introduction.Synthesis lectures on artificial intelligence and machinelearning, 2(1):1–88, 2008.

[14] Shen Li and Julie A Shah. Safe and efficient highdimensional motion planning in space-time with time pa-rameterized prediction. In 2019 International Conferenceon Robotics and Automation (ICRA), 2019.

[15] George J Mailath. Do people play nash equilibrium?lessons from evolutionary game theory. Journal ofEconomic Literature, 36(3):1347–1374, 1998.

[16] Jim Mainprice, Rafi Hayne, and Dmitry Berenson.Goal set inverse optimal control and iterative replan-ning for predicting human reaching motions in sharedworkspaces. IEEE Transactions on Robotics, 32(4):897–908, 2016.

[17] Stefanos Nikolaidis, Ramya Ramakrishnan, Keren Gu,and Julie Shah. Efficient model learning from joint-actiondemonstrations for human-robot collaborative tasks. InACM/IEEE international conference on human-robot in-teraction, 2015.

[18] Stefanos Nikolaidis, Anton Kuznetsov, David Hsu, andSiddhartha Srinivasa. Formalizing human-robot mutualadaptation: A bounded memory model. In 2016 11thACM/IEEE International Conference on Human-RobotInteraction (HRI), pages 75–82. IEEE, 2016.

[19] H. W. Park and A. M. Howard. Understanding a child’splay for robot interaction by sequencing play primitivesusing hidden markov models. In 2010 IEEE InternationalConference on Robotics and Automation, pages 170–177,2010.

[20] Mildred B Parten. Social participation among pre-

school children. The Journal of Abnormal and SocialPsychology, 27(3):243, 1932.

[21] David Premack and Guy Woodruff. Does the chimpanzeehave a theory of mind? Behavioral and brain sciences,1(4):515–526, 1978.

[22] Dorsa Sadigh, S Shankar Sastry, Sanjit A Seshia, andAnca Dragan. Information gathering actions over humaninternal state. In 2016 IEEE/RSJ International Confer-ence on Intelligent Robots and Systems (IROS), pages66–73. IEEE, 2016.

[23] Dorsa Sadigh, Shankar Sastry, Sanjit A Seshia, andAnca D Dragan. Planning for autonomous cars thatleverage effects on human actions. In Robotics: Scienceand Systems, 2016.

[24] Wilko Schwarting, Alyssa Pierson, Javier Alonso-Mora,Sertac Karaman, and Daniela Rus. Social behaviorfor autonomous vehicles. Proceedings of the NationalAcademy of Sciences, 116(50):24972–24978, 2019.

[25] Ioan A. Sucan, Mark Moll, and Lydia E. Kavraki.The Open Motion Planning Library. IEEE Robotics &Automation Magazine, 2012. doi: 10.1109/MRA.2012.2205651.

[26] Peter Trautman and Andreas Krause. Unfreezing therobot: Navigation in dense, interacting crowds. In 2010IEEE/RSJ International Conference on Intelligent Robotsand Systems, pages 797–803. IEEE, 2010.

[27] Annemarie Turnwald and Dirk Wollherr. Human-likemotion planning based on game theoretic decision mak-ing. International Journal of Social Robotics, 11(1):151–170, 2019.

[28] Vaibhav V Unhelkar, Ho Chit Siu, and Julie A Shah.Comparative performance of human and mobile roboticassistants in collaborative fetch-and-deliver tasks. InACM/IEEE International Conference on Human-RobotInteraction (HRI), 2014.

[29] Brian D Ziebart, Nathan Ratliff, Garratt Gallagher,Christoph Mertz, Kevin Peterson, J Andrew Bagnell,Martial Hebert, Anind K Dey, and Siddhartha Srini-vasa. Planning-based prediction for pedestrians. In 2009IEEE/RSJ International Conference on Intelligent Robotsand Systems, pages 3931–3936. IEEE, 2009.