Dimensions of Scalability in Cognitive Models Research Team: Carnegie Mellon University - Psychology...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Dimensions of Scalability in Cognitive Models Research Team: Carnegie Mellon University - Psychology...
Dimensions of Scalability in Cognitive Models Research Team: Carnegie Mellon University - Psychology Department
Dr. Christian LebiereDr. David ReitterDr. Jerry VinokurovMichael FurlongJasmeet Ajmani
Overview
• Goal: Scaling up high-fidelity cognitive models by– Composing models– Abstracting models– Running large networks of models
• ACT-UP: a toolkit view of cognitive architectures– Same validated functionality, different form
• Lemonade game: Reusing and integrating models• Language learning: Scaling up to network cognition• The Geo-Game: Bringing it all together
– Platform for experimentation and integration
Dimensions of Scaling
ACT-R Cognitive Architecture
• Computational implementation of unified theory of cognition
• Commitment to task-invariant mechanisms
• Modular organization• Parallelism but strong
attentional limitations• Hybrid symbolic/
statistical processes
Issues with Cognitive Modeling
• High-fidelity cognitive models provide very accurate models of all observable dimensions of cognition (time, accuracy, gaze, neural) but
• They are computationally intensive as they simulate all cognitive processes in full detail
• They are labor intensive to specify all aspects of cognitive performance (knowledge, strategies)
• They are specialized to a given task in a way that makes them difficult to compose and reuse
• They usually focus on single-agent cognition
Scaling Up Cognitive Modeling
• Enable the implementation of more complex cognitive models in a more efficient manner
• Scale up the application of cognitive models to simulate learning and adaptation in communities (e.g., 1,000 models in parallel)
• Enable reuse and composition of cognitive models similar to software engineering view
• Facilitate integration of cognitive models with other modeling and simulation platforms
• Improve maintenance, update and validation
The Approach
• Difficulties: ACT-R is heavily constrained already, and models are difficult to develop, reuse and exchange
• Constraints: Architectural advances require further constraints, e.g. more representational constraints
• Scaling it up: Complex tasks, broad coverage of behavior, multi-agent cognition and predictive modeling may motivate further architectural changes
• Solution: produce models at a higher abstraction level• Retain and emphasize key cognitive mechanisms• Abstract purely mechanistic model aspects• Precisely specify model claims, underspecify/fit rest• Benefits of abstraction in efficiency, scalability, reuse
SymbolicSymbolic
SubsymbolicSubsymbolic(Learning / (Learning / Adaptation)Adaptation)
deterministicdeterministic
non-deterministicnon-deterministicexplains explains empirical empirical variancevariance
Cognitive Strategy
specify:specify:
deterministicdeterministic
non-deterministicnon-deterministicexplains explains empirical empirical variancevariance
underspecify:underspecify:
Underspecified Models
(Lisp Functions)
ACT-UP vs ACT-R 6
• Declarative memory: chunks as objects– Explicit context specification; all
activation computations• Procedural memory: productions as
functions– Explicit conflict set groups; utility
reinforcement learning• ACT-UP is synchronous with serial
execution– Parallelism in process of being
implemented• Perceptual-motor modules being planned
Validation
• Against canonical ACT-R tutorial models data
Efficiency
• Sentence production (syntactic priming) model– 30 productions in ACT-R, 720 lines of
code– 82 lines of code in ACT-UP (3 work-
days)– ACT-R 6: 14 sentences/second– ACT-UP: 380 sentences/second
Scalability
• Language evolution model– Simulates domain vocabulary emergence
(ICCM 2009, JCSR 1010)– 40 production rules in ACT-R– Complex execution paths: could not prototype– 8 participants interacting in communities
• In larger community networks:– 1000 agents– 84M interactions (about 1 min. sim.
Each)– 37 CPU hours
Related Work
• Douglass (2009; 2010) on large declarative memories– Implementation through Erlang threads– Focus on scalability
• Salvucci (2010) work on supermodels– Integrating and validating independent models– Focus on instruction interpretation for generality
• Stewart and West (2007) work on Python-ACT-R– Similar deconstructive view of architecture– Integration with neural constructs
Future Work
• Complete validation against canonical model set; currently in beta testing; full release planned for spring 2011
• Possible collaboration with AFRL Mesa on implementation of finite-state-based systems
• Potential use in other projects (Minds Eye, Robotics CTA)• Allow optional parallelism where needed and desired• Implement perceptual and motor modules• Potential implementation in other languages (C++, Java) to
facilitate code-level integration with common frameworks
Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA.
Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In Proceedings of the 2010 MODSIM conference. Hampton, VA.
Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In Proceedings of the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010.
Cognitive principles in cooperative and adversarial games:
Metacognition transfers via ACT-UP
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Networks (Distributed Knowledge)
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
ACT-UP: Rapid prototyping/Reuse
• Dynamic Stocks & Flows ACT-UP model– Winning modeling competition entry– Model written in < 1 person-month– Free parameters (timing) estimated
from example data– Model generalized to novel conditions
• Reuse of Metacognitive Strategy in the Lemonade Stand Game (BRIMS 2010)
Kevin A. Gluck, Clayton T. Stanley, Jr. L. Richard Moore, David Reitter, and Marc Halbrügge. Exploration for understanding in model comparisons. Journal of Artificial General Intelligence (to appear), 2010.
David Reitter. Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence (to appear), 2010.
David Reitter, Ion Juvina, Andrea Stocco, and Christian Lebiere. Resistance is futile: Winning lemonade market share through metacognitive reasoning in a three-agent cooperative game. In Proceedings of the 19th Behavior Representation in Modeling & Simulation (BRIMS), Charleston, SC, 2010.
Multi-agent Games
• 2x2 games such as the Prisoner’s Dilemma– Evolution of cooperation vs. competition– Memory-based expectations (Lebiere et al, 2001)
• Adversarial games (Paper Rock Scissors, Baseball)– Zero-sum competition where predictability is
fatal– Sequence-based expectations (Lebiere et al, 1998;
2003)
• Lemonade game (3 players)– Simultaneous cooperation and competition– Predictability can be desirable for cooperation
The Lemonade Stand Game
• In each iteration, each of three players chooses a location 1..12
• Payoff is proportional to the distance to left and right neighbors.
• Hidden moves (blind choice)• 1 game: 100 iterations, then reset
(no state across games)
Zinkevich (2010, unpublished)
Basic Strategies
• Random (unpredictable): choose random loc.
• Sticky (predictable): choose same location– Roll, SquareRoot
• Tournament with those four agents– Equal performance
Strategy Elements
• Offer Cooperation: Be predictable• Predict: Learn patterns of opponents• Maximize Utility: Choose highest
expected payoff• Cooperate: Pick “friendly” opponent
whose payoff is also maximized• Monitoring: analyzing own/others
performance, keep history
Strategies
offer Coop Predict maxPayoff CoopMonitorSe
lfMonitorOp
p
Sticky +
StickySmart + +
StickySharp +
CopyCat + +
Statistician +
Cooperator + + +
Strategist + + +
Metacognition
• Facility to constantly monitor performance, and to adapt behavior accordingly
• Choose the best-performing strategy out of a set of strategies (Flavell 1979, Brown 1987)
• Strategy-shifting assumed in Dynamic Stocks & Flows data (DSF Challenge)
General Metacognition
• Prediction of each opponent’s next move– Learn from agent’s history in this game– Multiple possible representations and
pattern-matching• Action: Making a move
– Optimize Utility– Suggest cooperation– Cooperate– Hurt the worst adversaries
Evaluating Strategies
• Prediction and Action strategies are learned as episodes (instances):– Each prediction strategy per iteration, per
opponent– one action strategy per iteration
• Instance-based learning (Gonzales&Lebiere 2003)
• Objective: Prediction quality/Action payoff• Blending: weighted mean (recency,
frequency, objective as above)
Metacognition in Prediction
ACT-R Activation(recency, frequency)
Expected success of strategy s and
agent a
Episode in memory:time t, actual chosen location l of
agent a, predicted probability p for l,a by strategy s
• Each prediction strategy suggests a next location for each opponent
• All past predictions are stored throughout the game: <t,l,p> (time, actual location, predicted probability of that location)
Metacognition for Actions is similar
as in Reitter (2010) - DSF model
Evaluation
• Outcome of each strategy depends on configuration of players– Some strategies
will cooperate• Metacognitive
strategy is flexible, achieves consistently high results
• Bigger circle: higher winnings. Darker circle: consistent results.
Tournament
Adaptive Multi-Agent Behavior
• Offering cooperation and cooperating with the right opponent are crucial to doing well
• Metacognitive layer allows an agent to trump all others through generality and adaptivity
• Research questions:– Human performance in cooperative games:
issues of trust, social and cultural biases– Memory activation and rational retrieval
expectations as proxy for weighing past strategy success – limits of metacognition
Future Work
• ACT-R/ACT-UP’s learning vs. more basic Bayesian models: is cognitive learning more robust through open-endedness?
• Break down current limits of cognitive models generality– Are canonical architectural parameters optimal through
coevolution for empirical clustering factors and degrees?• Key part of environment is social interactions
– Automatic acquisition of rules, strategies, structural representations rather than modeler specification
– Metacognition: accumulation of micro-strategies library into reusable, general-purpose metacognitive layer
– Combination of above provide way of breaking out of task-specific models and their assumptions: beyond task-specific parameters, representation, strategies
Scaling Up Cognitive Models from Individuals to Large NetworksThe case of communication in human communities
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Networks (Distributed Knowledge)
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
David Reitter and Christian Lebiere. Towards explaining the evolution of domain languages with cognitive simulation. Cognitive Systems Research (in press), 2010.
Dr. Christian LebiereDr. David ReitterCarnegie Mellon University
Interactive Alignment
• Garrod & Pickering 2004:
from
: G
arr
od
&P
ickeri
ng
, B
BS
20
04
Syntactic Representation
Syntactic Representation
Adaptation in Language
• Rapid decay within 8-10 secondsexperimentally, for selected constructions: Levelt & Kelter (1982),Branigan et al. (2000)
• Long-term adaptation effects, which do not decay, have also been observed (Comprehension: Mitchell et al. 1995. Production: Bock&Griffin 2000)
• ACT-R’s declarative memory decay explains the repetition probability decay
(Switchboard corpus) Reitter (2008)
Interactive Alignment
from
: P
ickeri
ng
&G
arr
od
, B
BS
20
04
Lexical Representation
Lexical Representation
Syntactic and Lexical Adaptation Predict Task
Success!(Reitter & Moore 2007)
Domain Language Experiment
• Vocabulary: Signs as meaning-signifier combinationSimple Communication System: Lewis 1989, Hurford 1989, Oliphant&Batali 1996
• Naming game: an idealized transaction between two players– Pictionary: a director draws a given target
concept using elementary drawings; a matcher has to guess the concept.
– 20 target concepts, repeated– Director/Matcher receive no explicit feedback“Brad Pitt”
Fay et al., Cognitive Science 34(3), 2010. Kirby et al., PNAS 2008; Fay et al. PhilTransRoySoc-B 2008
Pictionary Performance
Community IsolatedPairs
part
ner
swit
ch(c
om
mu
nit
ies)
part
ner
swit
ch(c
om
mu
nit
ies)
part
ner
swit
ch(c
om
mu
nit
ies)
From data by Fay et al. 2010
(empirical)
ID accuracy:proportion of signs retrieved
Broad Questions
• How does the architecture of human cognition interact with social structure?
• Have the human mind and large-scale social structures co-evolved?
• Can modeling predict the kinds of team structures that will yield optimal communication and collaboration?
Pictionary Model in ACT-UP
• Ontology shared betweendirector and matcher– abstract target concepts– concrete drawings– link weight distribution
acquired from Wall Street Journal collocations
• Director chooses three related drawings to convey a target concept
• Choice is conventionalized• Decision-making and memory
retention modeled with ACT-UP
Ontology
weig
hted
link
Pictionary and Networks
part
ner
swit
ch(c
om
mu
nit
ies)
part
ner
swit
ch(c
om
mu
nit
ies)
Reitter&LebiereJournal of Cognitive Systems Research, in press
(ACT-UP model)p
art
ner
swit
ch(c
om
mu
nit
ies)
ID accuracy:proportion of signs retrieved
100 rep.
Community IsolatedPairs
Scaling up to Networks
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Networks (Distributed Knowledge)
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
Dr. Christian LebiereDr. David ReitterCarnegie Mellon University
Reitter, D., & Lebiere, C. (2010). Did social networks shape language evolution? A multi-agent cognitive simulation. In Proc. Cognitive Modeling and Computational Linguistics Workshop (CMCL 2010), Uppsala, Sweden.
Research Questions
• Does network structure affect convergence towards a common community vocabulary?– Or: Is declarative memory robust w.r.t. a variety of
network structures?• The small-scale, empirical and modeling data
suggest that extreme networks (fully vs. disconnected) arrive at similar performance, but converge differently. How? Why?
• Larger communities that differ in their connectivity are needed to answer these questions.
Network Types
• In a network, only network neighbors play the naming game• Social: Small-World network
(low path length, high clustering coefficient, assortatively mixed by degree)
• Grid (torus)• Random Graph• Organizational: Trees
• Controlled: mean degree (except trees), number of nodes
• Here: 512 nodes, mean deg. 6., 50 rep. per condition
ID Accuracy: Neighbors
Network type: ***random<grid<smallworld<tree
MCMC on LMER log(IDacc) ~ center(round) + cond + (1|sequence)
pre
lim
inary
resu
lts
ID Accuracy: Random Pairs
Tree
RandomSmall World Grid
Indication of convergence towards common vocabulary across network (measured after round 35)
Tree vs. others: n.s. (p=0.14, MCMC on LMER log(IDacc) ~ cond + (1|
sequence))pre
lim
inary
resu
lts
Summary
• Online Linguistic Adaptation is a known phenomenon – syntactic, lexical. Between two and more
participants.– Nodes can adapt to their immediate surroundings
• Tree hierarchies function very well when stable, but are not robust to structural change– Tree hierarchies represent contemporary
organizational hierarchies and generalize typical command structures
– Small-World structures are more robust to change.
Future Work
• Which advantages do non-tree network organizational structures have in situations where environment/ground truth changes, where adversarial elements are present?
• How can temporal dynamics in network structure (gradual ramp-up in connectivity) support information convergence (domain vocabulary acquisition)?
• Do cognitive models require explicit information processing policies in non-tree hierarchies, such that accountability and reliability are preserved?
• Integration of communication with planning, control and decision-making in complex dynamic domains.
Information Exchange in Networks
• Simulation at cognitive level: Language Evolution Model
• Simulation with Bayesian LearnersWang et al. (CMU Robotics), for a Bayesian Belief Update network
• Empirical validation is rare– Real-time communication networks are rarely
studied– Most empirical datasets contain asynchronously
produced communication, lacking control over exchanged information (e.g., Enron or Twitter corpora)
Human Networks: Empirical Experiments with the Geo Game
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Networks (Distributed Knowledge)
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
Dr. Christian LebiereDr. David ReitterPsychology, Carnegie Mellon University
Dr. Katia SycaraAntonio JuarezDr. Paul ScerriDr. Robin GlintonRobotics Institute, Carnegie Mellon University
Dr. Michael LewisUniversity of Pittsburgh
MURI Team: The Geo Game
Task allocation among
humans/agents
Task allocation among
humans/agents
Probabilistic models of human decision-making in network
situations
Probabilistic models of human decision-making in network
situations
Decentralized control search and planning
Decentralized control search and planning
Information fusionInformation fusion
Network performance as a
function of topology
Network performance as a
function of topology
Communication, evolution, language
Communication, evolution, language
CornellCornell GMUGMU CMU Psychology
CMU PsychologyMITMIT PittPitt
Level 1,3Level 1,3 Level 2Level 2Level 1,2Level 1,2
Level 1-2.5
Level 1-2.5 Level 1-3Level 1-3 Level 1,3Level 1,3Level 1Level 1
Level 1,2Level 1,2 Level 1-2.5
Level 1-2.5 Level 3 Level 3 Level 2Level 2
Level 1-3
Level 1-3
Level 1,2Level 1,2 Level 1,3, 4Level 1,3, 4Level 1-3Level 1-3
Level 4Level 4 Level 2Level 2
Level 3Level 3 Level 2, 3Level 2, 3
Adaptive automationAdaptive automation Level 1,2Level 1,2 Level 1Level 1
Level 1,2 Level 2
CMURobotics
CMURobotics
Level 1Level 1Scaling of cognitiveperformance and
workload
Scaling of cognitiveperformance and
workload
Level 1,2
The Geo Game
• Information exchange in human networks– On-line (real-time) communication– Medium to large networks (15 to 1,000 nodes)– Defined information needed to execute given
task– Information is spread throughout the network
• Natural language as a means to exchange communication– Often task-specific, controlled language
(e.g. radio communication)• Trade-off: communication vs. task executionThe Geo Game platform is being developed by CMU Psychology and CMU
Robotics
Geo Game: Participant’s Task
Geo Game as Platform
• Subjects are organized in a graph– vertices define communication channels:
subjects can only communicate with network neighbors
– Currently: small-world network• The Geo Game is a platform
– Current game: foraging task– Other variants: trading agents, varied information
types (stochastic, graded, discrete, etc.), other networks (trees, adversarial networks)
– Server&web-browser based system; remotely deployable
Geo Game: Push vs. Pull
• Basic manipulation: push vs. pull of information– Relevant to practical domain and theoretical cognitive
issues• Push condition:
– Post all relevant information - items in cities; path efficiency
– Maximize information at cost of overloading attention/memory
• Pull condition:– Specify needs and only answer/forward relevant
information– Minimize overload at cost of opportunities
Push vs. Pull: Scalability
• If communication aids task success, does this effect scale?
One group of 15 participantsSeptember 2010
Geo Game: Time-to-Response
• Question-Answer Pairs: time (Q to A) is power-law distributed
cf. Barabasi (2010)
Publications
• Reitter, D., & Lebiere, C. (2009). A subsymbolic and visual model of spatial path planning. In Proceedings of the Behavior Representation in Modeling and Simulation (BRIMS 2009). Best paper award BRIMS 2009.
• Reitter, D., & Lebiere, C. (2009). Towards Explaining the Evolution of Domain Languages with Cognitive Simulation. In Proceedings of the 9th International Conference on Cognitive Modeling. Manchester, England.
• Reitter, D., Lebiere, C., Lewis, M., Wang, H., & Ma, Z. (2009). A Cognitive Model of Perceptual Path Planning in a Multi-Robot Control System. In Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics. San Antonio, Texas.
• Reitter, D., Juvina, I., Stocco, A., & Lebiere, C. (2010). Resistance is Futile: Winning Lemonade Market Share through Metacognitive Reasoning in a Three-Agent Cooperative Game. In Proceedings of the Behavior Representation In Modeling and Simulations (BRIMS 2010) Conference. Charleston, SC
• Reitter, D., & Lebiere, C. (2010). Did social networks shape language evolution? a multi-agent cognitive simulation. In Proc. Cognitive Modeling and Computational Linguistics Workshop (CMCL 2010, at Association for Computational Linguistics: ACL 2010), Uppsala, Sweden.
• Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA.
• Reitter, D., & Lebiere, C. (2010). On the influence of network structure on language evolution. In R. Sun, editor, Proc. Workshop on Cognitive Social Sciences: Grounding the Social Sciences in the Cognitive Sciences (at Cognitive Science: CogSci 2010), Portland, Oregon.
• Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In Proceedings of the 2010 MODSIM conference. Hampton, VA.
• Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In Proceedings of the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010.
• Reitter, D. & Lebiere, C. (in press). Towards explaining the evolution of domain languages with cognitive simulation. Journal of Cognitive Systems Research.
• Reitter, D. & Lebiere, C. (in press). A cognitive model of spatial path planning. Journal of Computational and Mathematical Organization Theory.
• Reitter, D. (to appear). Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence.
• Gluck, K. A., Stanley, C. T., Moore, L. R., Reitter, D., & Halbrügge, M. (in press) . Exploration for understanding in model comparisons. Journal of Artificial General Intelligence.
Future Work
• The Geo Game can be exploited as an experimental platform for years to come– Communication: Alignment of communication
standards (e.g., vocabulary)– Knowledge: Information type and acquisition and its
influence on its distribution across the network (shared knowledge is not copied knowledge)
– Network structure: influence on team performance– Trust and strategy: Adversarial networks
• Mixed human/model/agent networks– Bootstrapping methodology for model validation– Information filtering for humans