Optimal Control applied to life sciences: a numerical method based presentation

92
Optimal Control applied to A numerical method based presentation Talk under the framework of the project “Stochastic models in medicine and life science: mathematical analysis, model identification, validation and stability properties” sponsored by CAPES Foundation, Ministry of Education of Brazil. Jorge Guerra Pires Information Engineering and Science PhD program University of L’Aquila/ Institute of Systems Analysis and Computer Science [email protected] Italy, 2014

Transcript of Optimal Control applied to life sciences: a numerical method based presentation

Page 1: Optimal Control applied to life sciences: a numerical method based presentation

Optimal Control applied to

A numerical method based presentation

Talk under the framework of the project “Stochastic models in medicine and life science: mathematical analysis, model identification, validation and stability properties” sponsored by CAPES Foundation, Ministry of Education of Brazil.

Jorge Guerra PiresInformation Engineering and Science

PhD programUniversity of L’Aquila/

Institute of Systems Analysis and Computer Science

[email protected], 2014

Page 2: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

2

Optimal Control applied to

Abstract: Optimal Control Applied to Life Sciences

Life Sciences might be seen as the connections between medicine, biology, mathematics, physics, and computer sciences. Further, optimal control might be defined as the extension of static optimization, or even as some comments, the new face of Variational Calculus.

In this talk we present several examples from life sciences analyzed with the support of optimal control. We might apply basically three approaches to optimal control problems, with their own weaknesses and strengthens: the Pontryagin’s Maximum Principle, Dynamic Programming, or Static Optimization. All the problems treated here were analyzed by the Pontryagin’s Maximum Principle.

The problems are solved using numerical schemes implemented in a computer. Topping the bill, we present two cases from a paper on process of publication: phototherapy for infants affected by neonatal jaundice and Feed-Forward Loop Network. We leave as future works comparisons with other approaches such as dynamic programming, or works on constraints on state space. Furthermore, we have concentrated on continuous-deterministic problems.

Keywords: Life Sciences, applied optimal control, numerical schemes, Runge-Kutta Method, forward-backward sweep method.

Page 3: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

3

Optimal Control applied to

Cases Considered throughout the endeavor

Toy models;

Mold and Fungicide;

Bacteria ;

Tasmania Devil facial tumour disease;

Optimal production of Protein;

Drug Administration in one-compartment model;

Applied Optimal Control Theory in Phototherapy of Infants;

Cancer;

Page 4: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

4

Optimal Control applied to

Cases Considered throughout the endeavor

Bioreactors;

Predator-Prey Model;

Discrete Time Models; Cancer Therapy with Gompertzian Growth and one-compartment model; Fish Harvesting;

Epidemic Model;

HIV Treatment;

Bear Populations: metapopulation;

Glucose Model;

Timber Harvesting;

Page 5: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

5

Optimal Control applied to

Cases Reported

Cancer Therapy with Gompertzian Growth and one-compartment model; Fish Harvesting; (Excluded, but interesting)

Epidemic Model;

HIV Treatment;

Bear Populations: metapopulation; (Excluded, but interesting)

Optimal production of Protein;

Drug Administration in one-compartment model; (Excluded, but interesting)

Applied Optimal Control Theory in Phototherapy of Infants;

Page 6: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

6

Optimal Control applied to

1. Introduction

A straightforward definition of life sciences is no longer simple. It might be said that in the past, this scientific domain comprised of a set of united field such as medicine and biology that rarely interfered with each other; nonetheless, in the present it is a “unique” branch comprised of researches from a variety of field such as mathematics, medicine, and biology.

The inclusion of mathematics and other fields such as computer science (and information sciences) came as a “rebirth” of the field.

Page 7: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

7

Optimal Control applied to

1. Introduction

Page 8: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

8

Optimal Control applied to

1. Introduction

Optimal control might be defined as a sub-area of control system in which we look for the best policy (control) over a period of time. It was born in the 50s from concerns on the aerospace industry, instead of optimizing a finite set of variables, today named static optimization, they wanted to optimization a dynamical system behavior, dynamic optimization.

In the majority of applications we take as granted important properties of the system such as controllability.

The similarities are such as the fact that we can show necessary and sufficient conditions based upon the same mathematical framework.

Page 9: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

9

Optimal Control applied to

1. Introduction

Page 10: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

10

Optimal Control applied to

1. Introduction

Each box is defined as:

The tradeoff function represents the balance between doing something and something else, such as letting the system evolves on its own dynamics or control. In general, this is a integral over the whole period of optimization and a payoff function for the final state;

The dynamical system represents the real system itself, using state variables and state equations;

Further details comprise further details such as bounds on control or even maximum total amount of the same all of the period.

In the scheme it was not represented a arrow between the further details’ box and the tradeoff’s once the demands on the tradeoff are in general mathematically demanded such as limited tradeoff function on the optimal policy and pathway.

Page 11: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

11

Optimal Control applied to

1. Introduction

Page 12: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

12

Optimal Control applied to

1. Introduction

Page 13: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

13

Optimal Control applied to

1. Introduction

Page 14: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

14

Optimal Control applied to

2. Ordinary Differential Equation and Systems

The term dynamic refers to phenomena that produce time-changing patterns, the characteristics of the pattern at one time being interrelated with those at other times. The term is nearly synonymous with time-evolution or pattern of change. It refers to the unfolding of events in a continuing evolutionary process. The term system was originated as a recognition that meaningful investigation of a particular phenomenon can often only be achieved by explicitly accounting for its environment [7].

Dynamical system might be defined as a single or set of “systems” that evolve in time, in the case of more than one, they are in general coupled, for example in social networks. The most common methodology applied to model those systems are differential equations. It is not clearly defined, but other areas such as biology work sometimes with their own methodology for modeling dynamical system. We might say that one of the challenge of complex networks theory is to unify those field, mainly using dynamical systems theory.

Page 15: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

15

Optimal Control applied to

2. Ordinary Differential Equation and Systems

Several issues are treated in the theory of dynamical systems, the most common are bifurcations, stability, and modeling. Bifurcation is the study of quantitative behavior of a system close to some special values of the parameters of the model, called bifurcation parameters. Basically several systems might “change” their dynamical close to several parameterizations. Stability is a quite important matter, it is related to the fact that a system can be kept on a particular state, or at least close to it, called equilibrium points. Modeling is related to the “ability” of a system to represent real plants, this is extremely important in control theory, once the study is done on the model, then it must be applied to the real plant, if the model is too far from the real plant, quite strange behavior will come up.

Predominantly, we have continuous and discrete systems. They differ on the tools, for continuous systems, we apply differential equations, whereas for discrete we apply difference equations. Example of discrete systems is birth and death modeling, and continuous is density modeling.

Page 16: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

16

Optimal Control applied to

2. Ordinary Differential Equation and Systems: gene expression dynamics

Source: Simulations for BRICS-CCI Brazil 2013

Page 17: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

17

Optimal Control applied to

2. Ordinary Differential Equation and Systems: gene expression dynamics

Source: Simulations for BRICS-CCI Brazil 2013

Page 18: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

18

Optimal Control applied to

2. Ordinary Differential Equation and Systems: gene expression dynamics

Source: Simulations for BRICS-CCI Brazil 2013

Page 19: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

19

Optimal Control applied to

2. Ordinary Differential Equation and Systems: gene expression dynamics

Source: Simulations for BRICS-CCI Brazil 2013

“Type-1 FFL is a sign-sensitive delay element that can protect against unwanted responses to fluctuating inputs. The magnitude of the delay in the FFL can be tuned over evolutionary timescales by varying the biochemical parameters of regulator protein y [middle gene], such as its lifetime, maximum expression, and activation threshold ” Alon (2007, p.57).

Page 20: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

20

Optimal Control applied to

3. Numerical Schemes

Numerical Scheme can be defined as an human-written sequence of orders (algorithm) with the purpose to solve problems based upon numbers.

Examples go from a simple root finder scheme to a more complicate scheme for teaching networks (learning machine), called learning paradigms.

A quite famous application of numerical schemes came with Gauss, the Mean Squared Error (MSE), when he predicted the trajectory of an unknown planet based upon observations, this is today the base for numerical schemes, from optimal control to learning machine and regression.

The second famous application came with Feynman and Collaborators, when they have been surprised by the result of a numerical scheme, this had confirmed the corpuscular nature of heat, no chaotic.

It is said that Hodgkin–Huxley had studied their famous model on dynamics of neuron using a hand-like calculator, the model is based on dynamical systems.

Page 21: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

21

Optimal Control applied to

3. Numerical Schemes

Nowadays we have an almost unanimous consent regarding the importance of numerical methods; nonetheless some still “resist” such as an old professor of mine from Russia.

If we compare the simple method of Guass to the ones used today, we get somehow surprised, such as the studies of an old professor of mine from Russia, he was modeling the sun’s dynamics by numerical schemes.

Besides we always make use of computer for numerical schemes, the majority of them, even the new ones are based upon those, were developed before the computer could have been even dreamt of. The consequence is that we need to improvise, see the variations of the famous method of Newton. Maybe this time for us develop our own schemes!

It is rare a case where we have a single numerical scheme for solving a problem, the variations go around the aim “cost-simplicity-accuracy”. The choice in the majority of cases is a matter of schools or preferences.

Page 22: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

22

Optimal Control applied to

3. Numerical Schemes: Euler Method or Tangent Line Method

Page 23: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

23

Optimal Control applied to

3. Numerical Schemes: Euler Method or Tangent Line Method

Page 24: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

24

Optimal Control applied to

3. Numerical Schemes: Euler Method or Tangent Line Method

Page 25: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

25

Optimal Control applied to

3. Numerical Schemes: Euler Method or Tangent Line Method

Page 26: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

26

Optimal Control applied to

3. Numerical Schemes: Euler Method or Tangent Line Method

Page 27: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

27

Optimal Control applied to

3. Numerical Schemes: The Runge-Kutta Method

Page 28: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

28

Optimal Control applied to

3. Numerical Schemes: The Runge-Kutta Method

Page 29: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

29

Optimal Control applied to

3. Numerical Schemes: The Runge-Kutta Method

Page 30: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

30

Optimal Control applied to

3. Numerical Schemes: Error

Page 31: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

31

Optimal Control applied to

3. Numerical Schemes: Shooting Method

Page 32: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

32

Optimal Control applied to

3. Numerical Schemes: Shooting Method

Page 33: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

33

Optimal Control applied to

3. Numerical Schemes: The Newton’s Method

Page 34: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

34

Optimal Control applied to

3. Numerical Schemes: The Secant’s Method

Page 35: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

35

Optimal Control applied to

4. Numerical Solutions for optimal control: algorithms

All the numerical schemes presented here are result of the application of the necessary conditions based on the theory of Hamiltonian, as a result of applying the Pontryagin’s Maximum Principle. The necessary conditions used here are those presented in [11].

All the derivations will be presented before any coding is discussed (omitted). In order to try to simplify the presentations, we will organize the problems into prototypes.

The prototypes suppose to serve as a reference model. We have chosen to present those in prototypes once they will demand different schemes for each of them.

The good news is that they all might be done with an extension of single code for the prototype 1: Forward-Backward Sweep Method, for short FBSM.

Page 36: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

36

Optimal Control applied to

4. Numerical Solutions for optimal control: algorithms

All the models treated follow a common pattern given by the following picture (next slide).

The drawback of this approach is the red arrow: one must first transform the optimal control problem into a system of differential equation, this is hand-calculation, and for complicate system it might be cumbersome.

One potential approach is directly integrating the system. One possible way of doing that is integrating the Hamiltonian directly, but it might introduce further error on the approximations of the derivatives for the optimality condition and the adjoint equation.

Furthermore, it might be complicate once we are demanded to optimize the Hamiltonian function.

Page 37: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

37

Optimal Control applied to

4. Numerical Solutions for optimal control: algorithms

Page 38: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

38

Optimal Control applied to

4. Numerical Solutions for optimal control: algorithms

This is the simplest problem we can formulate in optimal control that could be applied to real problem. As we will see, this simple approach sometimes might not be appropriate and further details should be added, this is the next prototype.

Page 39: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

39

Optimal Control applied to

4. Numerical Solutions for optimal control: algorithms

Page 40: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

40

Optimal Control applied to

4. Numerical Solutions

for optimal control:

The Forward-Backward

Sweep Method

Page 41: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

41

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method

Page 42: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

42

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method

Page 43: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

43

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method for Prototype 4

Page 44: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

44

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method

Page 45: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

45

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method for prototype 5

Page 46: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

46

Optimal Control applied to

4. Numerical Solutions for optimal control: The Forward-Backward

Sweep Method

Page 47: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

47

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Page 48: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

48

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

The case study discussed here is the treatment by phototherapy for newborns affected the syndrome called Neonatal Jaundice. The neonatal jaundice is caused by the immaturity of the liver, turning impossible the conjugation of byproduct of the natural process of hemoglobin breaking down that happens after birth, therefore the elimination of the undesired wastes, which are toxic, see for example Pires et al (2009) and references therein.

According to (Pires et al, 2009; Schoof et al 2012), and further references, jaundice might be seen as the discoloration of skin, mucous, and sclera due to the accumulation of bilirubin, a waste of the hemoglobin breaking down, they turn the skin, mucous, and sclera yellowish.

Page 49: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

49

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

We know from studies with light properties that frequency is what matters, what differentiate the colors, the properties of x-ray compared to gamma-rays. We also know that the visible spectrum is relatively small, something from 200 nm to 700 nm in wavelength.

The sunlight is the most complete illumination system, containing from ultraviolet to more rare wavelengths. Nonetheless, we must recollect that what heats the earth is present in the sunlight, namely, infrared, or even the so-feared ultraviolet. Therefore, side effects might turn it impossible to use it depending on the phototherapy intensity demanded.

In the case of the neonatal jaundice, the wavelength corresponding to the “blue” is the demanded one. Thus, a good system of illumination for this phototherapy must “possess the blue spectrum in abundance.”

Page 50: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

50

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Page 51: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

51

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Page 52: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

52

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Before going on, some remarks might be useful. In Pires et al (2009), it is not discussed on the real correlation between the sensitivity of the bilirubin and the polymeric solution to the radiation, just taken as granted the correlation, based on theory and independent experiments.

In the case of the bilirubin, we have a circulatory system, that is, just the portion of the blood in the skin will be exposed to the blue light, whereas in the polymeric solution, the expose is higher. Nonetheless, this type of issue had not limited the application of optimal control.

In Lenhart e Workman (2007), this is presented a bear-control policy designed by optimal control. In Lenhart e Workman (2007), it has been presented a control of bear over regions sharing boundaries, called metapopulations. The control of a region affects the other by the dynamic on the boundaries.

Page 53: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

53

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Simulations for the neonatal phototherapy. The green curve depicts a “bad case” of the syndrome, that is, the body of the newborn eliminate the bilirubin in a low rate compared to the “production” rate; the red curve represents an arbitrary case, but better than the green curve; the blue curve represents the case in which the degree of importance to diminish the concentration of bilirubin is increased by 10 compared to the risks of the phototherapy. We simulate an “intermittent phototherapy;” we apply the therapy for 1.5 units of time and then stop for another 1.5 units of time. We have used 10 units of concentrations as initial state. The dashed rectangle in red is the therapeutic window. Source: own elaboration.

Page 54: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

54

Optimal Control applied to

5. Optimal Phototherapy Regimen: the neonatal jaundice case

Simulations for the phototherapy in newborns using a drug X in junction. The blue curve represents the simulations using just the phototherapy, whereas the red is the simultaneous use of phototherapy and the drug X. The dashed line represents the goal (superior limit). We apply the treatment for 1.u units of time. Source: own elaboration.

Page 55: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

55

Optimal Control applied to

6. Optimal Protein Production: the feed-forward loop network motif

Page 56: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

56

Optimal Control applied to

6. Optimal Protein Production: the feed-forward loop network motif

The topic treated here is quite intriguing or even provocative, and indeed the treatment herein overlooks several important topics.

For instance, from an engineering viewpoint, one might think of a bioreactor and we aim to optimize the workings the same; but from a biological viewpoint, we might want to understand how a certain gene or even its circuit was selected under evolution.

This topic has been exploited in the literature, see for instance Alon (2007) for several discussion from a different angle. For the mathematical model used here, see Cacace et al (2012), or even Pires (2012), for more details. For discussion on potential application of this topic in engineering, see for example Pires and Palumbo (2012), or as alternative Pires (2013).

Page 57: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

57

Optimal Control applied to

6. Optimal Protein Production: the feed-forward loop network motif

On this part of the paper, we discuss on the optimal control of protein production. We suppose that we are lucky enough to reveal the secrets of god, and we find a function that is always optimized in the protein production process. See that Alon (2007) comments that one difficulty in optimization theories, such as the one used here, is that we may not know the fitness function in the real world. Let’s neglect this issue.

In simple terms, we have three genes. Further, just one produces what we need, call it Z, but surprisingly, it cannot be controlled directly, a second gene must be activated for that, call it X. Thus, the more we have activated X, the more we will have producing, but something starts to go wrong, then we identify a third gene, call it Y. We find out that Y is activated by X, but it inhibits Z, our goal. Then we must activate X as much as possible, but keep Y as silent as possible. It must have an equilibrium point. The communication between genes is done by a special group of proteins called transcription factors, something like papers in a company, functionless, but necessary. What we have described is called a network motif, namely, it is a feed-forward loop network motif, an incoherent type I.

Page 58: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

58

Optimal Control applied to

6. Optimal Protein Production: the feed-forward loop network motif

Page 59: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

59

Optimal Control applied to

6. Optimal Protein Production: the feed-forward loop network motif

Controls for different situations, parameterizations: protein degradation rate, mRNA degradation rate, protein production rate, mRNA production rate, degree of importance to produce protein, degree of importance for minimizing the control, and threshold for the Hill functions “K” on the mathematical model. The lines on continuous style are different values for the parameters, on the dashed lines we have increased by 10 the importance of producing proteins compared to the cost, the cost is applied just on gene X, input gene. The green function depicts what happens if we add the final value of protein to be optimized (a payoff term). The initial condition for all genes are ‘0’, but gene X which is given a small amount of mRNA in time ‘0’. Source: own elaboration.

Page 60: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

60

Optimal Control applied to

7. Epidemic: optimal policy

Page 61: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

61

Optimal Control applied to

7. Epidemic: optimal policy

Epidemics can be defined as the breakout of an infection disease. Related terms are pandemic (world-spread-out) and endemic (steady-state).

Epidemics is in general a complex network issue.

Source: http://1.bp.blogspot.com/-WSrc1yadP2U/TzFs3dHA-TI/AAAAAAAABuo/bVbX1lgTnd8/s1600/epidemic_diffusion.jpg

Page 62: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

62

Optimal Control applied to

7. Epidemic: optimal policy

Page 63: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

63

Optimal Control applied to

7. Epidemic: optimal policy

Page 64: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

64

Optimal Control applied to

7. Epidemic: optimal policy

Page 65: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

65

Optimal Control applied to

7. Epidemic: optimal policy

Page 66: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

66

Optimal Control applied to

7. Epidemic: optimal policy

Page 67: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

67

Optimal Control applied to

7. Epidemic: optimal policy

Page 68: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

68

Optimal Control applied to

7. Epidemic: optimal policy

Page 69: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

69

Optimal Control applied to

8. HIV and AIDS

Page 70: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

70

Optimal Control applied to

8. HIV and AIDS

Acquired immunodeficiency syndrome (AIDS) is medically devastating to its victims, and wreaks financial and emotional havoc on everyone, infected or not.

Keywords: Viral Replication; Immunology.

“Viruses are very small biological structures whose reproduction requires a host cell.” They are not considered living things.

Helper T-lymphocytes play a key role in the process of gaining immunity to specific pathogens.

HIV is an especially versatile virus. It not only inserts its genetic information into its host’s chromosomes, but it then causes the host to produce new HIV.

A virus cannot reproduce outside a host cell, which must provide viral building materials and energy.

Page 71: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

71

Optimal Control applied to

8. HIV and AIDS

The immune system monitors the body for dangerous pathogens.

When it detects pathogens, the immune system computers and mobilizes the appropriate responses.

The immune system is made of a vast collection of cells that communicate and interact in myriad ways.

One of the major tools of the immune system is antibodies.

One of the important roles of the immune system is to scan the cells of the body for antigens – foreign proteins made by pathogens –.

The scanning process is carried out by T-cells.

Page 72: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

72

Optimal Control applied to

8. HIV and AIDS

Page 73: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

73

Optimal Control applied to

8. HIV and AIDS

Viral nucleic acid enters the host cell and redirects the host cell’s metabolic apparatus to make new viruses.

“Many RNA viruses do not use DNA in any part of their life cycle.”

Living Systems and some virus

Retrovirus

Page 74: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

74

Optimal Control applied to

8. HIV and AIDS

Page 75: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

75

Optimal Control applied to

8. HIV and AIDS

Key-point 1 : To find an optimal chemotherapy strategy in the treatment of the human immunodeficiency virus (HIV).

Key-point 2: The model used herein describes the interaction of the immune system with HIV.

Key-point 3: It is assumed the treatment acts to reduce the infectivity of the virus for a finite time.

Antiretroviral drugs are medications for the treatment of infection by retroviruses, primarily HIV. Different classes of antiretroviral drugs act on different stages of the HIV life cycle. Combination of several (typically three or four) antiretroviral drugs is known as highly active anti-retroviral therapy (HAART).

Page 76: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

76

Optimal Control applied to

8. HIV and AIDS

Each T cell attacks a foreign substance which it identifies with its receptor. T cells have receptors which are generated by randomly shuffling gene segments. Each T cell attacks a different antigen. T cells that attack the body's own proteins are eliminated in the thymus. Thymic epithelial cells express major proteins from elsewhere in the body. First, T cells undergo "Positive Selection" whereby the cell comes in contact with self-MHC expressed by thymic epithelial cells; those with no interaction are destroyed. Second, the T cell undergoes "Negative Selection" by interacting with thymic dendritic cell whereby T cells with high affinity interaction are eliminated through apoptosis (to avoid autoimmunity), and those with intermediate affinity survive.

Page 77: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

77

Optimal Control applied to

8. HIV and AIDS

Page 78: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

78

Optimal Control applied to

8. HIV and AIDS

Page 79: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

79

Optimal Control applied to

8. HIV and AIDS

Page 80: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

80

Optimal Control applied to

8. HIV and AIDS

Page 81: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

81

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

Tumor is a state of the body where cells divide (mitosis, multiply) on an uncoordinated way. This is a type of cancer in some cases. Tumors might be classified as benign, premalignant, or malignant (cancer). Cancer is so feared for spreading out, invading neighboring tissues, tumors (premalignant cancer) does not invade neighboring tissues. On this section we present the simplest model possible to build applying the theories presented so far.

The model presented herein is relatively simple. We apply the theory of optimal control for the treatment of cancer. Using the mentioned theory, we can optimize the behavior of a set of differential equations, on our case, ordinary differential equations.

The model discussed here was published by Fister e Panetta (2003) and studied by (Lenhart e Workman 2007; Swan, 1984); we try to make it better by adding the dynamic of the drug.

Page 82: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

82

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

Page 83: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

83

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

See that the first term of the differential equation represents a ‘growth,’ a natural process, using a dynamics called Gompertz model, and the second represent our control by means of the treatment.

It should be pointed out that we can use just this equation on the study of optimal control applied to tumor therapy and this is what is done on Lenhart e Workman (2007), but we can do better!

What about the dynamics of the drug? This is what we add here. Drugs might exhibit peculiar behavior and assuming that we can control the amount exactly of drug that reaches the site might be a mistake.

As Lenhart e Workman (2007) highlights studies of drugs is a rich and state of the art field, and one of the models lacking are the ones that studies multiple drugs taken at the same time. That is, models that takes into account interactions between different drugs; see that the model presented here is limited in the sense that it is still kinetics, not dynamics, models for dynamics are much complicate, we skip them.

Page 84: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

84

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

Just to provoke these studies on the literature, let’s consider an ideal case of two drugs taken for eliminating this tumor.

We consider a simple case, the second drug just increase the absorption of the first, the drug one is the one that really can eliminate the tumor.

This can increase the possibility to maintain the plasma concentration within the therapeutic window; see that the second drug is increasing the absorption, then it should eventually increase the concentration above the therapeutic window of the drug we need to monitor, and this exactly the trick of optimal control, the optimal control policy will just use what is needed given a goal (therapeutic window).

This is known as bioavailability; this is similar when you are suggested to take a drug together with milk, milk is not a drug, just increase the bioavailability of the take drug, or even when you are asked to use a drug with empty stomach. This type of analysis is concern of noncompartmental models.

Noncompartmental models are considered easier to use due to several reasons such as it does not change from individual to individual so much as compartment models. See Rosenbaum (2011) for more details.

Page 85: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

85

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

Page 86: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

86

Optimal Control applied to

9. Optimal Tumor Treatment : Cancer Therapy with Gompertzian Growth and one-compartment model

Page 87: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

87

Optimal Control applied to

References

[1] George W Swan, Applications of optimal control theory in biomedicine, Pure and Applied Mathematics. Marcel Dekker Inc, 1984. [2] Wendell H Fleming; Raymond W Rishel. Deterministic and Stochastic Optimal Control. Applications of Mathematics 1, Springer-Verlag: 1975. [3] Boyce, William E.; Diprima, Richard C. Elementary differential equations and boundary value problems seventh edition. John Wiley & Sons, Inc.: 2001. [4] Betts, J. T. Practical Methods for optimal Control and Estimation Using Nonlinear Programming. Second Edition. Advances in Design and Control, Society for Industrial and Applied Mathematics. 2010. [5] Wikipedia. The Secant Method Online: http://en.wikipedia.org/wiki/Secant_method. Accessed on May/2014. [6] Wikipedia. The Newton’s Method Online: http://en.wikipedia.org/wiki/Newton%27s_method. Accessed on May/2014.[7] David G. Luenberger, introduction to dynamic systems: theory, models, and application. John Wiley & Sons, 1979.[8] Lynch, Stephen. Dynamical systems with applications using Mathematica®. Birkhäuser Boston, 2007.[9] Wikipedia. Logistic Map http://en.wikipedia.org/wiki/Logistic_map. Accessed on May/2014.

Page 88: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

88

Optimal Control applied to

References

[10] Wikipedia. Tent Map http://en.wikipedia.org/wiki/Tent_map. Accessed on May/2014.[11] Lenhart, S.; Workman, J.T, Optimal Control Applied to biological models, Chapman & Hall/ CRC, Mathematical and Computational Biology Series, 2007.[12] Press, William H. ;Teukilsky, Saul A. ;Vetterling, William T. ;Flannerg, Brian P. Numerical recipes in C: the art of scientific computing. Second edition. Cambridge University press: 1992.[13] Ruggiero, Márcia A. Gomes; Lopes, Vera Lúcia da Rocha. Cálculo numérico: aspectos computacionais. 2° edição. São Paulo: Pearson Markon Books, 1996.[14] Devries, Paul L. ; Hasbun, Javier E. A first course in computational physics. Second edition. Jones and Bartlett Publishers: 2011.

Page 89: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

89

Optimal Control applied to

References

ALON, U. An Introduction to systems biology: design principles of biological circuits. Chapman & Hall/CRC, 2007.CACACE, F., GERMANI, A., PALUMBO, P., The state observer as a tool for the estimation of gene expression, Journal of Mathematical Analysis and Applications, Vol.391, pp.382-396, 2012.LENHART, S.; WORKMAN, J. T, Optimal Control Applied to biological models, Chapman & Hall/ CRC, Mathematical and Computational Biology Series, 2007.PIRES, J. G. Desenvolvimento de programa baseado no problema da mistura . Pesquisa Operacional: programação matemática. Simpósio de Engenharia de Produção. XVI: 1-12, 2009.PIRES, J. G.; DUARTE, A. S.; BIANCHI, R. F.; SANTOS, Z. A. DA S.; BIANCHI, A. G. C. Projeto e desenvolvimento de Produto: proposta e desenvolvimento de dispositivo eletrônico para auxiliar no tratamento da icterícia. Gestão do Produto: engenharia do produto. Simpósio de Engenharia de Produção. XVI: 1-12, 2009.PIRES J. G. On the applicability of Computational Intelligence in Transcription Network Modeling. Thesis of master of science. Faculty of Applied Physics and Mathematics, Gdansk University of Technology, Poland. 74:1:46. 2012.PIRES, J. G.; PALUMBO, P. Engenharia de Software: Planejamento e desenvolvimento de programa baseado em Inteligência Computacional aplicada a Redes de Expressão Genética . Gestão do Produto: engenharia do produto. Simpósio de Engenharia de Produção. XIX: 1-12, 2012.PIRES, J. G. Na importância da biologia e em engenharias: biomatemática e bioengenharias . Educação em Engenharia de Produção: estudo do ensino de engenharia de produção. Simpósio de Engenharia de Produção. XX: 1-12, 2013.ROSENBAUM, S. E Basic pharmacokinetics and pharmacodynamics: an integrated textbook and computer simulations, John Wiley & Sons, 2011. Visite: http://www.uri.edu/pharmacy/faculty/rosenbaum/basicmodels.SCHOOF, C. P.: Zschocke, J.: Potocki, L. Human Genetics: from molecules to medicine. Lippincott Williams & Wilkins: 2012.WINSTON, W. L. Operations Research: application and algorithms. Third Edition, 1996.

Page 90: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

90

Optimal Control applied to

What is next....?

Dynamics programming;

Optimization based schemes;

Discretization based approaches;

Stochastic counterparts;

Extension of the models presented herein for stochastic framework;

Pharmacogenomics;

Test some in PK/PD;

Pharmacokinetic/pharmacodynamic interactions;

Receptor Interactions;

Report in September;

Java Library for Optimal Control in Life Sciences (JAR Executable)!? !?

Page 91: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

91

Optimal Control applied to

http://ijcai-15.org/index.php/important-dates

Page 92: Optimal Control applied to life sciences: a numerical method based presentation

June/2014 Jorge G Pires

92

Optimal Control applied to

Thank you.