SOMChE 2016 Miri, Sarawak, Malaysia, 1-3 December 2016 …...4.0 Results & Discussions Modelling,...

3

1.0 Introduction

Modelling, Simulation & Computing Laboratory (mscLab)Faculty of Engineering, Universiti Malaysia Sabah, Malaysia

SOMChE 201629th Symposium of Malaysian Chemical Engineers

Miri, Sarawak, Malaysia, 1-3 December 2016

Substrate (S) Undesired products (E)

Desired microbes (X)

• Challenges• Fermentation – complicated

dynamics and nonlinearsystem

• Control failure – increasingundesired side-products

• Mostly depending on priorknowledge / experience

4

2.0 Objective

The purpose of this paper is to optimize fed-batch production(maximize desired products and minimize undesired products) usingQ-learning.

a. Derive and simulate fed-batch processb. Develop and compute multi-objective Q-learning algorithmc. Carry out multi-objective optimization process and performance

testing of developed algorithm in fed-batch process




5

3.0 Methodology (Q-Learning)

Q-Learning (QL) – an unsupervised learning algorithm• Interacts with process environment (get states, reply in actions)• Gain experience (updates Q-Table)• Decides actions (input feed flow) based on process goal (reward)




6





Curse of Dimensionality

Q-LearningExploration

Exploitation

In large size problem (large states and

actions space)

The challenge of QL

7


Schematic diagram of QL in fed-batch system




8


• Multistep action (MSA) Q-Learning is applied for fed-batch process

• Reward function




9


• Initialization of the learning process – no experience (new toenvironment) – exploration (in action space)

• Continue learning (more rounds), the most chosen route will havehighest weight




a1 a2 a3 a4 a5 a6 a7 .. an

s1

s2

s3

s4

s5

:

sp

5 7 6 4 3 3 4 3 2

5 7 8 7 4 4 3 3 2

4 6 6 9 7 8 6 6 5

4 5 6 7 8 5 4 3 2

3 4 5 6 7 8 9 6 5

3 3 4 6 7 7 8 7 6

10

4.0 Results & Discussions




0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

Concentr

ation o

f G

lucose(g

/l),

Eth

anol(g/l)

and o

xygen(m

g/l)

glucose

ethanol

oxygen

0 1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

80

Concentr

ation o

f Y

east(

g/l)

Time(h)

Substances concentration profile at F = 500*exp(0.05*t)(l/h)

yeast

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

Concentr

ation o

f G

lucose(g

/l),

Eth

anol(g/l)

and o

xygen(m

g/l)

glucose

ethanol

oxygen

0 1 2 3 4 5 6 7 8 9 100

10

20

30

40

50

60

70

80

Concentr

ation o

f Y

east(

g/l)

Time(h)

Substances concentration profile using Q-Learning

yeast

Conventional: Exponential Feeding Proposed: Q-Learning

14 %

11

4.0 Results & Discussions




12

5.0 Conclusions

• QL suggested a substrate feed flow rate that is able to improve the yeastproduction by 14 % compared to exponential feeding while minimizingethanol significantly throughout the process.

• The results show the potential of implementing unsupervised self-learningalgorithm in fed-batch fermentation to achieve multi-objectiveoptimization: maximizing desired product and minimizing undesiredproduct.




SOMChE 2016 Miri, Sarawak, Malaysia, 1-3 December 2016 …...4.0 Results & Discussions Modelling,...

Documents

Transcript of SOMChE 2016 Miri, Sarawak, Malaysia, 1-3 December 2016 …...4.0 Results & Discussions Modelling,...