Dynamics of Learning & Distributed Adaptation
1
Dynamics of Learning & Distributed Adaptation Santa Fe Institute: James P. Crutchfield, P.I. Future Plans (6 months out) New problems: Continuous-state and continuous-time agents Adaptation to active, pattern-forming environments Dynamical theory of how learning and adaptation occur Anticipated results: Monitor emergence of cooperation in agent collectives Measure mutuality in interacting reinforcement learne Test on in-house autonomous robotic vehicle collectiv Analytical tools: Predict whether or not group cooperation can occur Agent intelligence versus group size Prediction of the rate of adaptation during collectiv Prototype models: Solvable MAS systems Software tools: Ab Initio Learning Algorithms Library for Estimating MASS Metrics Enterprise Java Platform for Robot Collectives Multi-Agent System Science (MASS) Dimension Agents learn complex environment ab initio Synchronization of agent to environment Agents adapt to nonstationary environment Strategies for agent-agent coordination Metrics for large-scale MASs Statistical Complexity: Amount of structure & organization in environ’t Individual agent knowledge v. group knowledge Mutuality: Architecture of information flow Lyapunov Spectra: Degrees of stability and instability Causal Synchrony: Detect coherent subgroup behavior CAHDE REF ACFC: Adapting to instabilities in air flow control AirOps: Emergence of spontaneous leadership Solution: Interacting reinforcement and -machine learning agents solve a group task Approach: Pattern Discovery: Beyond pattern recognition Design & analysis based on sound principles of learning Metrics for cooperation in large-scale systems Results To Date Predictive theory of agent learning: Quantify agent modeling capacity Data Set Size v. Prediction Error v. Model Complexity Pattern Discovery: The “Aha” Effect Incremental learning algorithm Quantify structure in environment: How structure leads to unpredictability for agent Define synchronization for chaotic environments: Predict required data and time to synchronize Periodic case solved in closed form Transient information: New metric of synchronization
description
Dynamics of Learning & Distributed Adaptation. Santa Fe Institute: James P. Crutchfield, P.I. Multi-Agent System Science (MASS) Dimension Agents learn complex environment ab initio Synchronization of agent to environment Agents adapt to nonstationary environment - PowerPoint PPT Presentation
Transcript of Dynamics of Learning & Distributed Adaptation
Dynamics of Learning & Distributed Adaptation Santa Fe Institute: James P. Crutchfield, P.I.
Future Plans (6 months out)New problems:
Continuous-state and continuous-time agentsAdaptation to active, pattern-forming environmentsDynamical theory of how learning and adaptation occur
Anticipated results:Monitor emergence of cooperation in agent collectivesMeasure mutuality in interacting reinforcement learnersTest on in-house autonomous robotic vehicle collectives
Analytical tools:Predict whether or not group cooperation can occurAgent intelligence versus group sizePrediction of the rate of adaptation during collective taskPrototype models: Solvable MAS systems
Software tools:Ab Initio Learning AlgorithmsLibrary for Estimating MASS MetricsEnterprise Java Platform for Robot Collectives
Multi-Agent System Science (MASS)Dimension
Agents learn complex environment ab initioSynchronization of agent to environmentAgents adapt to nonstationary environmentStrategies for agent-agent coordination
Metrics for large-scale MASsStatistical Complexity:
Amount of structure & organization in environ’tIndividual agent knowledge v. group knowledge
Mutuality: Architecture of information flowLyapunov Spectra: Degrees of stability and instabilityCausal Synchrony: Detect coherent subgroup behavior
CAHDE REF
ACFC: Adapting to instabilities in air flow controlAirOps: Emergence of spontaneous leadership
Solution:Interacting reinforcement and -machine learning agents solve a group task
Approach:
Pattern Discovery: Beyond pattern recognition
Design & analysis based on sound principles of learning
Metrics for cooperation in large-scale systemsResults To DatePredictive theory of agent learning:
Quantify agent modeling capacity
Data Set Size v. Prediction Error v. Model Complexity
Pattern Discovery: The “Aha” Effect
Incremental learning algorithm
Quantify structure in environment:
How structure leads to unpredictability for agent
Define synchronization for chaotic environments:
Predict required data and time to synchronize
Periodic case solved in closed form
Transient information: New metric of synchronization
Dynamics of reinforcement-learning agents:
Nash equilibria v. oscillation v. chaos
Dependence on system architecture and initial state
