4. Serving Two Masters

Customer loyalty: a case studyCustomer loyalty: a case study

BAYERBAYERFrom 1898 through From 1898 through to 1910 Bayer to 1910 Bayer marketed a “non-marketed a “non-addictive morphine addictive morphine substitute” and substitute” and “cough medicine” for “cough medicine” for children. children.

““That will keep them coming”That will keep them coming”

And with this, we conclude our And with this, we conclude our case study on customer loyalty.case study on customer loyalty.

Let’s briefly review of the course, Let’s briefly review of the course, and have a first thesis proposaland have a first thesis proposal

On Computers & UnderstandingOn Computers & Understanding

• The Capyblanca projectThe Capyblanca project

• De Groot & Simon & Gobet & ChaseDe Groot & Simon & Gobet & Chase

After a 5sec presentation, subjects are After a 5sec presentation, subjects are asked to reconstruct the position.asked to reconstruct the position.

The Capyblanca ProjectThe Capyblanca Project

The Capyblanca ProjectThe Capyblanca Project

EmergenceEmergence– Small, independent processes produce the final resultSmall, independent processes produce the final result

PrimingPriming– Perception of subtle cues guide future processing; Perception of subtle cues guide future processing;

processing is NOT pre-programmedprocessing is NOT pre-programmed

UnderstandingUnderstanding– The system obtains a synthesis of the situation, in an The system obtains a synthesis of the situation, in an

information-efficient levelinformation-efficient level

FARG, 1.0FARG, 1.0

The ability interpret ambiguous, confusing, The ability interpret ambiguous, confusing, isolated cues into a meaningful, coherent, isolated cues into a meaningful, coherent, situation.situation.

This is what FARG already does now. It's This is what FARG already does now. It's what CAPYBLANCA does. But we want to what CAPYBLANCA does. But we want to replicate these, using a single framework, replicate these, using a single framework, at least in NUMBO and Copycat.at least in NUMBO and Copycat.

FARG 2.0: Next stopping pointFARG 2.0: Next stopping point

The ability to do it again, now faster.The ability to do it again, now faster.

How can the system learn and self-organize such How can the system learn and self-organize such that, in the future, things get more and more that, in the future, things get more and more efficient? efficient?

One idea here that goes beyond FARG is that of One idea here that goes beyond FARG is that of (positive or negative) feedback changing future (positive or negative) feedback changing future behavior; the other idea is the emergence of behavior; the other idea is the emergence of preference, in which the IOWA gambling task, or preference, in which the IOWA gambling task, or perhaps the N-armed bandit problem can help in perhaps the N-armed bandit problem can help in addressing.addressing.

However, things need to get faster.However, things need to get faster.

But is this of any quality?But is this of any quality?

Ron “Suki” King, a checkers world championRon “Suki” King, a checkers world champion

385 opponents—simultaneously385 opponents—simultaneously

He beat them allHe beat them all

2 seconds per move2 seconds per move 12.30mins response 12.30mins response

Capablanca’s remarkCapablanca’s remark

““I see only one move. I see only one move. The best one.”The best one.”

JosJosé Raul Capablancaé Raul Capablanca

Matheu 6:24Matheu 6:24

""No one can serve two No one can serve two masters; for either he will hate masters; for either he will hate the one and love the other, or the one and love the other, or he will be devoted to one and he will be devoted to one and despise the other.despise the other.” ”

But I think we all serve two mastersBut I think we all serve two masters

Two MastersTwo Masters

Master #1: The Hedonic Feedback systemMaster #1: The Hedonic Feedback system– pain and pleasurepain and pleasure

Master #2: The Attentional Feedback systemMaster #2: The Attentional Feedback system– Boredom & excitementBoredom & excitement

Master #1. Pain & PleasureMaster #1. Pain & Pleasure

The Hedonic Feedback SystemThe Hedonic Feedback System

• Hedonic tonalitiesHedonic tonalities

• Regulates frequency of actionsRegulates frequency of actions

• Objective of the system: Objective of the system: generate generate preferences and values for the systempreferences and values for the system

• What do I want from the world?What do I want from the world?

A thesis proposalA thesis proposal

How do these two feedback systems interact with How do these two feedback systems interact with each other?each other?

1.1. Hedonic system mostly works on the Hedonic system mostly works on the background, unconsciously.background, unconsciously.

2.2. Hedonic system “cancels out” after some time Hedonic system “cancels out” after some time (things originally intense become “boring, (things originally intense become “boring, expected, normal”—perhaps because the expected, normal”—perhaps because the attentional system has no more memory trace attentional system has no more memory trace of them?)of them?)

3.3. Attentional system triggers hedonic systemAttentional system triggers hedonic system

They usually go togetherThey usually go together

Heroin

Boredom, expected events, nothing new

Excitement, unexpected change or anticipation of change

Pain Pleasure

Child playing on power outlet

“Blind driving”

Girl feasting on box of ice creamSinus

But sometimes intense feelings become boringBut sometimes intense feelings become boring

Imagine if you had to live Imagine if you had to live next to a monster that next to a monster that screamed in your ear and screamed in your ear and threatened you every 5 threatened you every 5 minutes. minutes.

After some time, you would After some time, you would get used to it, and think it is get used to it, and think it is just “normal life”.just “normal life”.

I am not talking about marriageI am not talking about marriage

Youtube description by user Youtube description by user woodwardcars

““This is where your job went. Notice the This is where your job went. Notice the safety requirements: 60 ton press coming safety requirements: 60 ton press coming your way, no stop button, either "duck" or your way, no stop button, either "duck" or "move aside" when press comes down. If "move aside" when press comes down. If you are not on your "A" game today you'll you are not on your "A" game today you'll lose an arm at best, crushed skull at worst. lose an arm at best, crushed skull at worst. No problem, there's 20 guys standing No problem, there's 20 guys standing outside waiting to take your position.”outside waiting to take your position.”

http://www.youtube.com/user/woodwardcars

To me, this sounds just like a To me, this sounds just like a description of the US troops in Iraq.description of the US troops in Iraq.

““press[ure] coming your way, no stop button, press[ure] coming your way, no stop button, either "duck" or "move aside" when either "duck" or "move aside" when press[ure] comes down. If you are not on press[ure] comes down. If you are not on your "A" game today you'll lose an arm at your "A" game today you'll lose an arm at best, crushed skull at worst. No problem, best, crushed skull at worst. No problem, there's 20 guys standing outside waiting to there's 20 guys standing outside waiting to take your position.”take your position.” [Or at least that’s what [Or at least that’s what Bush is asking for]Bush is asking for]

Think about…Think about…

• Chinese textile “docile girls” (who are Chinese textile “docile girls” (who are ‘never going back to the farms’)‘never going back to the farms’)

• Concentration camp survivorsConcentration camp survivors

• Doctors in an emergency roomDoctors in an emergency room

• Garbage collectorsGarbage collectors

• Married peopleMarried people

If, sometimes, intense feelings become boring, If, sometimes, intense feelings become boring, what does this mean?what does this mean?

That there are That there are twotwo feedback systems. feedback systems.

What do scientific experiments show?What do scientific experiments show?

IOWA Gambling taskIOWA Gambling task2 sets of red cards2 sets of red cards2 sets of blue cards2 sets of blue cardsPlayers get $2.000 and instructions to get rich, but none Players get $2.000 and instructions to get rich, but none

of the following information.of the following information.

Game: get $100 with each red cardGame: get $100 with each red card get $50 with each blue cardget $50 with each blue card

Penalties for red cards go up to $1250Penalties for red cards go up to $1250Penalties for blue cards are less than $100Penalties for blue cards are less than $100

NOW, WHAT HAPPENS?NOW, WHAT HAPPENS?

ResultsResults

• After 50 cards, most people have a After 50 cards, most people have a feeling feeling about the good and the bad cardsabout the good and the bad cards

• After 80 cards, something has clicked, and After 80 cards, something has clicked, and people can explain what’s going on. They people can explain what’s going on. They have created a hypothesis, “tested it”, and it have created a hypothesis, “tested it”, and it seems good. They have figured out the seems good. They have figured out the game. game.

• After 10 (!) cards, people start to develop a After 10 (!) cards, people start to develop a stress response to the bad cards, 40 steps stress response to the bad cards, 40 steps before these are recognized as bad.before these are recognized as bad.

The IOWA gambling taskThe IOWA gambling task

• Tells us about where this type of feedback Tells us about where this type of feedback happens in the brainhappens in the brain

• Tells us that judgements are constantly Tells us that judgements are constantly made unconsciouslymade unconsciously

• Tells us that emotions are needed for Tells us that emotions are needed for decision-makingdecision-making

• And gives us an estimate of the time, in And gives us an estimate of the time, in terms of impressions, needed for terms of impressions, needed for judgements to arise.judgements to arise.

Experimenting with electrodes

The Hefferline experimentThe Hefferline experiment

Experimental setting:Experimental setting:– the desired response was a tiny thumb-twitch so small as to be the desired response was a tiny thumb-twitch so small as to be

subliminal, that is, the subjects could control it but only subliminal, that is, the subjects could control it but only unconsciously (it was referred to as an ‘invisible thumb-twitch’ unconsciously (it was referred to as an ‘invisible thumb-twitch’ since the subjects had no knowledge of their controlling of it).since the subjects had no knowledge of their controlling of it).

– The response was recorded by electrodes placed on the palmar The response was recorded by electrodes placed on the palmar base of the left thumb and on the medial edge of the left hand. base of the left thumb and on the medial edge of the left hand.

– Subjects heard music through headphones, and there was an Subjects heard music through headphones, and there was an aversive noise that would be mute for 15 seconds whenever the aversive noise that would be mute for 15 seconds whenever the response was obtained, or, in case of it already being mute at response was obtained, or, in case of it already being mute at the time of response, being then postponed for more 15 the time of response, being then postponed for more 15 seconds.seconds.


• Group 4. Recording electrodes and a meter that Group 4. Recording electrodes and a meter that informed them of response. Knew their behavior informed them of response. Knew their behavior would stop the sound. Straightforwardly would stop the sound. Straightforwardly developed the response.developed the response.

• Group 3. Recording electrodes, no meter. Knew Group 3. Recording electrodes, no meter. Knew their behavior would stop the sound. Some their behavior would stop the sound. Some subjects attempted to produce the response subjects attempted to produce the response voluntarily, which never occurred.voluntarily, which never occurred.


• Group 2. Recording electrodes PLUS some Group 2. Recording electrodes PLUS some dummy electrodes, so they could not know dummy electrodes, so they could not know where to attempt to respond to the noise. All where to attempt to respond to the noise. All subjects developed the response.subjects developed the response.

• Group 1. Recording and dummy electrodes. Group 1. Recording and dummy electrodes. They were not informed that had any control They were not informed that had any control on the disturbing sound. Instead, they were on the disturbing sound. Instead, they were told that the study was about “body tension told that the study was about “body tension of noise superimposed on music”. Thus, of noise superimposed on music”. Thus, they were simply told to “listen through they were simply told to “listen through earphones, and, otherwise, do nothing”.earphones, and, otherwise, do nothing”.


Impulsivity: Impulsivity: – Involuntary automatic unconscious processing Involuntary automatic unconscious processing

connecting perception and (unconscious micro) actionconnecting perception and (unconscious micro) action

HEFFERLINE, R.F. — KEENAN, B.—HARFORD, R.A.: HEFFERLINE, R.F. — KEENAN, B.—HARFORD, R.A.: Escape and avoidance conditioning in human subjects Escape and avoidance conditioning in human subjects without their observation of the response. Science, vol. without their observation of the response. Science, vol. 130, 1959, pp. 1338-1339.130, 1959, pp. 1338-1339.


One might be led to believe that group 1 was a control group, since One might be led to believe that group 1 was a control group, since subjects lacked any knowledge whatsoever of the required task, but subjects lacked any knowledge whatsoever of the required task, but this is far from the truth. Group 1 gradually developed the desired this is far from the truth. Group 1 gradually developed the desired response, as their brains were constantly scanning the senses for response, as their brains were constantly scanning the senses for associations, and, in less than an hour, they were fully conditioned associations, and, in less than an hour, they were fully conditioned to stop the noise. The immense subtlety involved in perceiving a to stop the noise. The immense subtlety involved in perceiving a causal connection between an ‘invisible’ thumb-twitch and the causal connection between an ‘invisible’ thumb-twitch and the annoying sound cannot be overstated, especially when we consider annoying sound cannot be overstated, especially when we consider the massive sensorial information constantly bombarding the the massive sensorial information constantly bombarding the subjects. subjects. After the experiments were finished, subjects in After the experiments were finished, subjects in group 1 were shocked to know they had been in control of the group 1 were shocked to know they had been in control of the noise all along, as they still believed to have had been passive noise all along, as they still believed to have had been passive listeners throughout the whole experiment.listeners throughout the whole experiment.

This experiment tells usThis experiment tells us

• That the brain constantly associates That the brain constantly associates perception with [motor] action.perception with [motor] action.

• That it quickly “picks up” the right That it quickly “picks up” the right association.association.

Master #1. Pain & PleasureMaster #1. Pain & Pleasure

The Hedonic feedback systemThe Hedonic feedback system

• Hedonic tonalities regulate Hedonic tonalities regulate frequency of actionsfrequency of actions• Objective of the system: generate preferences Objective of the system: generate preferences

and values for the systemand values for the system– What What do I want from the worlddo I want from the world??

• HOW DOES THIS SYSTEM WORK?HOW DOES THIS SYSTEM WORK?

How can we implement pain or pleasure?How can we implement pain or pleasure?

Computers with pain or pleasure?Computers with pain or pleasure?

Far from a mere technicality; this is a deep Far from a mere technicality; this is a deep philosophical questionphilosophical question

Fortunately…Fortunately…

I have a theory!I have a theory!

What do pain & pleasure feel like?What do pain & pleasure feel like?(System shutdown; global update)(System shutdown; global update)

To be executed

Previously executed

CODERACK

To be executed

Previously executed

Pain Signal !!!!

1. Interrupts processing for a global system update

CODERACK

To be executed

Previously executed

Pain Signal !!!!

1. Interrupts processing for a global system update

2. Lowers the Probability that these sequences will happen again in the future

CODERACK

Codelets (or associated nodes in the slipnet) Codelets (or associated nodes in the slipnet) are are observersobservers of what what’s going on on of what what’s going on on the hedonic feedback system.the hedonic feedback system.

Not only the hedonic feedback system Not only the hedonic feedback system throws someone away from “throws someone away from “immediatelyimmediately painful behavior”, but it also improves painful behavior”, but it also improves system speed, by changing the system speed, by changing the probabilities in a directed manner. probabilities in a directed manner.

““immediatelyimmediately painful behavior”? painful behavior”?

• Sometimes behavior is painful in short-Sometimes behavior is painful in short-term, but pleasurable/desirable in the long-term, but pleasurable/desirable in the long-run. Think, for example, of a vaccine.run. Think, for example, of a vaccine.

• Sometimes, of course, the opposite Sometimes, of course, the opposite occurs. Think about heroin.occurs. Think about heroin.

TrainspottingTrainspotting

““I chose not to choose life.I chose not to choose life.

I chose something else. I chose something else.

And the reasons? And the reasons?

There are no reasons. There are no reasons.

Who needs reasons when you’ve got heroin?”Who needs reasons when you’ve got heroin?”

A study with miceA study with mice

• Button #1. a sexy female mouse appearsButton #1. a sexy female mouse appears

• Button #2. Tasty foodButton #2. Tasty food

• Button #3. activates the brain’s pleasure Button #3. activates the brain’s pleasure centercenter

What do they do?What do they do?

And the reasons?And the reasons?

Who needs reasons when you’ve Who needs reasons when you’ve got a button linked to the pleasure got a button linked to the pleasure

center of the brain?center of the brain?

Master #2. Boredom & excitementMaster #2. Boredom & excitement

The attentional feedback systemThe attentional feedback systemRegulates memory storageRegulates memory storage

– Blind drivingBlind driving– September 11September 11thth

Objective of the system: discover how the world works and Objective of the system: discover how the world works and predict its behaviorpredict its behavior– How does the world work and How does the world work and how is it going to changehow is it going to change??

HOW DOES IT WORK?HOW DOES IT WORK?– Top-down, expectation-driven, subcognitive processesTop-down, expectation-driven, subcognitive processes– Links incoming from the brain to sensory neuronsLinks incoming from the brain to sensory neurons

The shape of things to comeThe shape of things to come

How do semantic networks evolve, reorganize, grow, and How do semantic networks evolve, reorganize, grow, and learn over time? How to avoid the handcoding problem? learn over time? How to avoid the handcoding problem? When does semantic memory touch episodic memory? When does semantic memory touch episodic memory? Can a node compute its own correlation to others as it Can a node compute its own correlation to others as it receives their signals, without global association receives their signals, without global association mechanisms? How do new nodes "pop"out? mechanisms? How do new nodes "pop"out?

How do abstract chunks come to life to enable analogy-How do abstract chunks come to life to enable analogy-making? Can there ever be composite command making? Can there ever be composite command codelets? codelets?

Can there ever be "general codelets", to handle come what Can there ever be "general codelets", to handle come what may? may?

How do feedback systems and the credit assigment How do feedback systems and the credit assigment problem fit into this picture? Finally, what does the N-problem fit into this picture? Finally, what does the N-armed bandit problem or the IOWA gambling task have armed bandit problem or the IOWA gambling task have to do with all of this?to do with all of this?

4. Serving Two Masters

Business

Transcript of 4. Serving Two Masters