1

Slide 1

Decisions and strategies

Anders Ringgaard KristensenAdvanced Herd Management

Slide 2

Outline

Decision making, information needs

Why models for decision support?

States, actions and utility (money)

Dynamics: The replacement problem

Decision processes and strategies

Dealing with uncertain knowledge

Decision hierarchies

A short overview of modeling techniques

Slide 3

From information to decision

Where focus last Tuesday was on the sub-path from data to

information, we shall now discuss the last sub-path from

information to decision.

A decision is an intention to use/not to use a factor at a given

level:

• Use 4 kg of concentrates per cow

• Cull cow no. 678

• Call for the vet!

• Build a new barn.

2

Slide 4

Necessary information

When a decision is made concerning a unit, the following information is necessary:• The present state of the unit

• The relation between factors and production

• Immediate production

• Future production

• The farmer’s personal preferences

• All constraints of legal, economic, physical or personal kind

Slide 5

Can we help? Decision support

Traditional methods• Standards, norms, recommendations

• Ignores variations in preferences, constraints and factor states.

• Forget it!

• Human experts

• Able to take individual conditions into account, but not able to combine information from different sources.

• New tools may make them able to do

Slide 6

Decision support

Future methods:

• Models for (monitoring and) decision support:

• Individual conditions (preferences and limitations)

• Representation of uncertainty

• Search for optimum

• Sensitivity analysis

• Better decisions than experts!

• What we try to learn you during this course!

3

Slide 7

Disadvantages of models

Model building is a very demanding task

Models may be very computer intensive

Lack of knowledge is not a problem relating to the model, but to the

decision problem!

Slide 8

Notation (decision graphs)

x

d

u

A variable (something that has a value)

A decision

Utility (e.g. money)

Causal influence

x1 d x2

Slide 9

The decision problem

it is the state of the system at time t

dt is the decision made at time t

ut is the utility consequence at time t given state and decision

Limitations are ignored in the figure!!!

d1

u1

i1

d2

i2

d3

i3

u2u3

4

Slide 10

The state

The state is a sufficient description of the system at time t

A description is sufficient if it contains all relevant information about the system

Defined by the value of one or several state variables each representing a trait (e.g. litter size, parity, health)

Probability distribution given previous state and decision

d1

u1

i1

d2

i2

d3

i3

u2u3

Slide 11

The decision

The decision concerns at least one factor

It is based on knowledge about the state

It influences the utility

It influences the future state

d1

u1

i1

d2

i2

d3

i3

u2u3

Slide 12

The utility

Depends on

• The output (e.g. # piglets produced)

• The value (e.g. the price of piglets)

• Farmer’s preferences (what should be measured)

d1

u1

i1

d2

i2

d3

i3

u2u3

5

Slide 13

Example: Dairy cow replacement

The state space could be defined by the state variables• Milk yield

• Pregnancy status

• Lactation number

• Stage of lactation

• Health status

The action space

• Keep the cow

• Replace it by a heifer

i

Milk Preg.

Stage

Health

d d

Lact#

Slide 14

Capacity

Test day 1* Test day 2* Test day 3* Test day 4* Test day 5* Test day 6*

Genetype Permanent

Temp 1

Pregnancy

Temp 2 Temp 3 Temp 4 Temp 5 Temp 6

Diagnosis*

Heat Obs. Heat*

Observing the state

”Milk yield” – the best possible basis for prediction

”Pregnancy status”

None of them are observable!

Slide 15

Dynamics: Time horizon

Consider the dairy cow replacement problem: The present cow is a 2nd lactation cow:• If we keep it, we will at next stage (year) have a 3rd lactation cow. In two

years we will have

• A 4th lactation cow, if still kept

• A 1st lactation cow if then replaced

• If we replace it, we will at next stage (year) have a 1st lactation cow. In two years, we will have

• A 2nd lactation cow, if new heifer kept

• A 1st lactation cow, if new heifer replaced

When making a decision for the present 2nd lactation cow, how far into the future shall we look? What is the time horizon?

The time horizon is not well defined.

6

Slide 16

A strategy (or policy)

Let Ω be the set of all possible states and D be the set of all possible decisions

A strategy s is a function s: Ω→D. For any state i∈Ω, the strategy sspecifies the decision d∈D to make.

A general rule: ”If state i is observed, decision d should be made”.

Problem: To determine a strategy that maximizes the utility of the farmer (under the limitations).

d1

u1

i1

d2

i2

d3

i3

u2u3

Slide 17

Feeding of dairy cows

True energy content of silage is unknown

The precision of the observed content depends heavily on the observation method (standard value from table, laboratory analysis etc.)

Silage obs.* Silage true

Concentr.*

Ration Milk yield*

Herd size*

Slide 18

Feeding of dairy cows, V

Decision graph for the full problem (student’s project).

Silage obs.* Silage true

Concentr.*

Ration Milk yield*

Herd size*Method

MixPrice Cost Rev.

7

Slide 19

Uncertainty

Uncertainty is not the opposite of knowledge

Uncertainty is a property of knowledge

Reduction of uncertainty is often possible at some cost!

Reducing uncertainty is not always profitable.

Slide 20

Decision Hierarchies

Time• Strategic

• Tactical

• Operational

Level• Herd

• Group

• Animal

In both cases decisions at different ”levels” interact

Slide 21

Survey of methods

Linear programming

• October 5th – 8th

• 3rd mandatory report

• Chapter 10

Bayesian networks

• September

• 2nd mandatory report

• Jensen (2001) – Chapters 1-2.

Decision graphs

• October 11th – 12th

• Chapter 12

Dynamic programming – MDP

• October 15th – 22nd

• 4th mandatory report

• Chapter 13

Simulation

• October 22th – 29rd

• Chapter 14

8

Slide 22

Methods: Linear programming

Minimize a linear cost function given a set of linear constraints.

Well known from ration formulation

Also applied for whole farm planning

Excellent for representation of constraints

Ignores uncertainty

Assumes linearity

Static method

Slide 23

Methods: Bayesian Networks

The ideal tool for representation of uncertainty

Graphical model description with well defined elements: Ellipses are random variables and arrows represent a causal relation

Combination of information from many sources

Silage obs.* Silage true

Concentr.*

Ration Milk yield*

Herd size*

Capacity

Test day 1* Test day 2* Test day 3* Test day 4* Test day 5* Test day 6*

Genetype Permanent

Temp 1

Pregnancy

Temp 2 Temp 3 Temp 4 Temp 5 Temp 6

Diagnosis*

Heat Obs. Heat*

Slide 24

Methods: Decision graphs

Baysian networks with decisions and utilities added.

Silage obs.* Silage true

Concentr.*

Ration Milk yield*

Herd size*Method

MixPrice Cost Rev.

9

Slide 25

Methods: Decision graphs

Same advantages as Bayesian networks

Static model

No forgetting

Computationally very demanding

Slide 26

Methods: Dynamic programming – MDP

Basic setup:

i1 i2 i3 i4 i5

d1

r1

d2

r2

d3

r3

d4

r4

d5

r5

Markov property: No memory of the past

Slide 27

Methods: Dynamic programming – MDP

Dynamic method

Many kinds of uncertainty may be represented

State representation less flexible than in decision graphs

Hope for the future: A combination of decision graphs and advanced variants

of dynamic programming.

10

Slide 28

Methods: Simulation

Monte Carlo simulation:• Random numbers

• Excellent for representation of herd restraints

• Excellent for representation of uncertainty

• No good methods to use in search for optimal strategies

Probabilistic (“Markov chain”) simulation• Dynamic programming without decisions

Slide 29

Properties of methods for decision support

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Perfect method

Lousy method

Slide 30

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Properties of methods for decision support

11

Slide 31

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Bayesian networks

Properties of methods for decision support

Slide 32

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Bayesian networks

Decision graphs

Properties of methods for decision support

Slide 33

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Bayesian networks

Decision graphs

Dynamic programming

Properties of methods for decision support

12

Slide 34

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Bayesian networks

Decision graphs

Dynamic programming

Simulation

Properties of methods for decision support

Slide 35

Herd constraints Optimization

Biologicalvariation

Uncertainty

Functionallimitations

Dynamics

Linear programming

Bayesian networks

Decision graphs

Dynamic programming

Simulation

Properties of methods for decision support

Slide 36

Properties of methods for decision support

Conclusion

• No single method is perfect

• Methods have very different strengths and weaknesses

• Combination of methods is a challenge for the future