An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish...
-
Upload
jade-malone -
Category
Documents
-
view
241 -
download
0
Transcript of An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish...
![Page 1: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/1.jpg)
An Instructable Connectionist/Control Architecture:Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale
Presented by: Jim Ries for CECS 477, WS 2000
Paper by:
Walter Schneider and William L. Oliver
University of Pittsburgh, Learning Research and Development Center
![Page 2: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/2.jpg)
Introduction Overview Task Decomposition
Gate Learning Example CAP2 Architecture CAP2 Rule Learning Authors’ Conclusions My Own Thoughts
![Page 3: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/3.jpg)
Walter SchneiderPh.D., Indiana University
Professor, Psychology
University of Pittsburgh, Pittsburgh, PA 15260
Phone: (412) 624-7061.
Fax: (412) 624-9149
Email: [email protected]
http://www.lrdc.pitt.edu/
![Page 4: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/4.jpg)
Overview
Hybrid approach to blend rules with connectionist model. Rules are “learned” (with instruction) and
represented in a connectionist manner. Learned rules are less “brittle”.
Attempts to decompose problems in order to hasten learning. Supposedly general decomposition mechanism.
Claims to model human cognition.
![Page 5: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/5.jpg)
Task Decomposition
As task complexity increases, learning times in both symbolic and connectionist systems can dramatically increase (perhaps exponentially).
Cognitive psychology indicates that basic cognitive processes can be decomposed into stages.
![Page 6: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/6.jpg)
Task Decomposition (cont.)
A good decomposition reduces the number of problem states needed for consideration. e.g., humans can do arbitrary addition by
memorizing 100 addition facts and an algorithm for adding one column at a time. w/o this decomposition, would need to learn 1010 addend combinations to solve 5 column addition problems!
![Page 7: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/7.jpg)
Gate Learning Example of task decomposition. In human version, subjects are instructed on the
rules for each gate, and then do many trials to learn.
W/o task decomposition, number of states is: 2i X g X n (where i is gate inputs, g is # of gate
types, n is # of negation states) W/ task decomposition, number of states is:
2i + (g X r) + (n X o) (where g is # of recording states, o is # of output states of gate mapping stage)
![Page 8: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/8.jpg)
Gate Learning (cont.) For six-input gates following:
w/o decomposition: 384 states w/ decomposition: 77 states
Decomposition reduces state growth from multiplicative function to additive.
![Page 9: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/9.jpg)
Gate Learning (cont.)
![Page 10: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/10.jpg)
Gate Learning (cont.)
![Page 11: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/11.jpg)
Gate Learning (cont.)
Networks trained to 100% accuracy (since there is no “noise”)
Results for 6 gate trial: w/o decomposition - 10,835 trials w/ decomposition - 948 trials human - 300 trials
![Page 12: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/12.jpg)
Gate Learning (cont.)
![Page 13: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/13.jpg)
Gate Learning (cont.)
Subjects begin by executing rules sequentially, and gradually switch to associative responses.
The stage taking the longest to converge (Recording) was the limiting factor.
The author did not mention whether real time for a “trial” differed between a net using decomposition or one without decomposition.
![Page 14: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/14.jpg)
Gate Learning (cont.)
![Page 15: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/15.jpg)
CAP2 Architecture
Controlled Automatic Processing model 2. Macro level : system of modules that pass vector
and scalar messages. Scalar messages used for control. Vector messages encode perceptual and conceptual
information.
![Page 16: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/16.jpg)
CAP2 Architecture (cont.)
Components Data Modules - transforms and transmits vector
messages (consistent with neurophysiology) Control Signals - control activity of modules
Activity report - codes whether a data module is active and has a vector to transmit
Gain control - controls how strongly the output of a module activates the other modules to which it is connected
Feedback - controls the strength of the autoassociative feedback within the module
![Page 17: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/17.jpg)
CAP2 Architecture (cont.)
![Page 18: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/18.jpg)
CAP2 Architecture (cont.)
![Page 19: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/19.jpg)
CAP2 Architecture (cont.)
Controller Module - sequential rule net Task input vector Compare Result input vector Context input vector Outputs control operations
• Attend• Compare (compare vectors from different modules)• Receive (enable a module to receive a vector)• Done
Currently implemented in C, rather than as a connectionist network!
![Page 20: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/20.jpg)
CAP2 Architecture (cont.)
![Page 21: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/21.jpg)
CAP2 Architecture (cont.)
![Page 22: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/22.jpg)
CAP2 Architecture (cont.)
Authors are committed to structural assumptions of the architecture (as related to human cognition) Processing substrate in humans akin to data
network Modular network structures serve as functional units
of processing Information passes between modules as vectors Memory associations among vectors develop
through learning similar to connectionist learning
![Page 23: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/23.jpg)
CAP2 Architecture (cont.)
Mechanisms for task decomposition Configuring the data network (# of stages, # of
modules/stage, etc. through control signals) Specifying the number of states in each stage.
CAP2 captures knowledge specified in rules However, the data network does not simply learn the
rules stored in the controller, but will learn patterns in input data as well (driving instructor example).
To achieve the same level of tuning in a production system would require a huge set of rules.
![Page 24: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/24.jpg)
CAP2 Architecture (cont.)
Chunking - “C” “A” “T” “CAT” Degree of matching (Euclidean distance)
activity report = (xi + yi)2
Does this mean that CAP2 would be unable to represent concepts that were “close” or “distant” in a different sense (e.g., Taxicab distance, or other distance measures)?
n
i
iyxi1
)(
![Page 25: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/25.jpg)
CAP2 Architecture (cont.)
![Page 26: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/26.jpg)
CAP2 Rule Learning
Rule learning should be achieved in a small number of trials (or why bother; just use connectionist learning).
Gate Learning example
![Page 27: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/27.jpg)
CAP2 Rule Learning (cont.)
![Page 28: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/28.jpg)
CAP2 Rule Learning (cont.)
![Page 29: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/29.jpg)
CAP2 Rule Learning (cont.)
Sequential network learned rules even faster than humans sequential network - 120 trials humans - 216 trials decomposed model - 932 trials single stage model - 10,835 trials
Rule knowledge is brittle, and performas much as novices perform during the early stage of rule learning.
![Page 30: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/30.jpg)
Authors’ Conclusions
Hybrid connectionist/control architecture illustrates the complementary nature of symbolic and connectionist processing. Better than connectionist learning, because it
benefits from instruction. Better than symbolic processing because it captures
rules in a connectionist network which can scale and is less brittle.
![Page 31: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/31.jpg)
Authors’ Conclusions (cont.)
Closely models human cognition. Not merely a connectionist implementation of a
symbolic architecture.
![Page 32: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/32.jpg)
My Own Thoughts
Unclear that this models human cognition, but I have no cognitive science background to truly evaluate this claim.
Is this really general? For example, they seemed to gloss over the fact that part of their rule system was implemented directly in C rather than in a connectionist manner.
The examples were generally done iteratively. How does parallelism change things (if at all)?
![Page 33: An Instructable Connectionist/Control Architecture: Using Rule-Based Instructions to Accomplish Connectionist Learning in a Human Time Scale Presented.](https://reader036.fdocuments.us/reader036/viewer/2022062517/56649eda5503460f94be9bdb/html5/thumbnails/33.jpg)
Full paper reference
Schneider, W. & Oliver, W. L. (1991). An instructable connectionist/control architecture: Using rule-based instructions to accomplish connectionist learning in a human time scale. In K. Van Lehn (Ed.), Architectures for intelligence: The 22nd Carnegie Mellon symposium on cognition (pp.113-145). Hillsdale, NJ: Erlbaum.