Approximate Dynamic Programming and Reinforcement Learning for Nonlinear Optimal Control of Power...
-
date post
20-Dec-2015 -
Category
Documents
-
view
221 -
download
2
Transcript of Approximate Dynamic Programming and Reinforcement Learning for Nonlinear Optimal Control of Power...
Approximate Dynamic Programming and Reinforcement Learning for Nonlinear
Optimal Control of Power Systems
November 4, 2003
Ronald HarleyGeorgia Institute of Technology
ECS-0231632ECS-0080764
Kumar VenayagamoorthyUniversity of Missouri-Rolla
Adaptive Critic Design: Nonlinear Optimal Control
PlantInformaton
UtilityFunction (U)
Optimal cost-to-gofunction (J)
Critic Networks :To minimize the value (of derivatives)
of J with respect to the states
Derivatives via BP
Model Network(Identifier) : To learnthe dynamics of plant
Model Network
Action Network :To find optimal control
u
Plant
Control
Reinforcement Learning
0
* )()(k
k ktUtJ
STATCOM Control
Simulation Results
100ms SC at PCC,
Line Voltage , Generator Terminal VoltageV
The simplified schematic of the SSSC (160 MVA, 15KV VL-L)
Optimal control for FACTS devicesInternal control for static series synchronous
compensator (SSSC)
Series VSI
SynchronousGenerator
V dc
Inf. bus
v s v rv c
is v x
Turbine-Governor
AVR -Exciter
rexe
SSSC
+
GTO
Control
DHPNCCONVC
idc
Optimal control for FACTS devicesInternal control for SSSC (CONVC)
PI Based internal controller (CONVC) for the SSSC
Synchrnouslyrotating reference
transformations
kk ip
+
+
)ˆ
ˆ(tan
ˆˆ
1
22
cd
cq
dc
cqcd
v
vα
V
vvm
Vdc *
Vdc
+-
id
PI- Vdc
iq
- s
kk ip
iq
PI- iq
cqv̂
+iq*
Vdc
s
kk ip
PI- ip
ip
cdv̂+id*
-
ia ib ic
m
+
GTO gate controlof series VSI
+
P*Q*
Vdc
id
Vectorphase-locked
loop
va
vb
vc
Real and reactivecurrent computation
V r
V'dc
Publication: N.G. Hingorani and L. Gyugyi, “Understanding FACTS-Concepts and Technology of Flexible AC Transmission Systems”, IEEE Press, New York, 2000.
Optimal control for FACTS devicesCase study: 100 ms three phase short circuit test at
receiving-end (infinite-bus)
Rotor angle
0 1 2 3 4 5 620
30
40
50
60
70
80
90
100
110
120
130
Time [s]
d [D
egre
e]
Uncompensated
CONVC
DHPNC
Schematic single-line diagram showing an SCRC with external controller (160 MVA, 15KV VL-L)
Optimal control for FACTS devicesExternal control for series capacitive reactance
compensator (SCRC)
SynchronousGenerator Inf. bus
v s
is
SCRC
v r 0
re2xe2
re1xe1Turbine-
Governor
AVR -Exciter
CsT1
1
Internal Controlof SCRC
VoltageSourceInverter
V dc
v c
+GTO
W
WC
sT1
sTK
X C
Filtering DampingController
CsT1
1
External Control
+
+
Line #1
Line #2
*CX
CX
Optimal control for FACTS devicesDHP based external controller (DHPEC)
Schematic single-line diagram showing the DHP based external controller (DHPEC)
SynchronousGenerator
Inf. bus
vs
is
SCRC
re2 xe2
re1 xe1Turbine-Governor
AVR -Exciter
Internal Controlof SCRC
VoltageSourceInverter
Vdc
vc
+GTO
XC
+
+
Line #1
Line #2
*CX
CX
vr
DHP basedexternal controller
(DHPEC)
Optimal control for FACTS devicesCase study: Step changes X*C [pu]
0 5 10 15 20 25-0.1
-0.08
-0.06
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
Time [s]
[ra
d/s
]
Fixed XC*
KC=1.5
DHPC
Speed deviation
Application in Multi-Machine power system
FACTS(SCRC)
Gen 31600 MVA
Gen 15000 MVA
Gen 22200 MVA 200 km
AREA 1 AREA 2
T1
T2
T3
T4
T5
T6
500 kV 500 kV 500 kV500 kV 13.2 kV
13.2 kV
13.2 kV
13.8 kV
115 kV
Line 1
Line 2
Line 3
Line 4
Line 5
Z 1
Z 2
1
2
34 5
610
7
DHPNC-G
CONVC-GS1
DHPEC-S
CONVEC-SS2 3
11Industrial load
Residentialload
115 kV 13.8 kV
8 9
Large-scale multi-machine power system
A UPFC in the POWER SYSTEM
InfiniteBusShunt
InverterSeries
InverterVd
c
SeriesInverterControl
ShuntInverterControl
V1
Vdcref
R1, L1
V2V1
V1ref
Z1Synch
Generator
Governor
AVR
Exciter
+
-
UPFC
Z1
V1ref
Vdc
Pref
Pinj
Qinj
Q ref
1 2
Pout,
Qout
Verr
Vdcerr Perr
Qerr
R2, L2
Vr
TurbinePref
Neurocontroller
Neuroidentifier
Q
P
ed
eq
Neurocontroller
Neuroidentifier
Vdc
V
epd
epq
5 6 7 8 9 10 11-50
0
50
100
150
200
250
Time (sec)
Load
Ang
le(°
)
UPFC
PI
5 6 7 8 9 10 11-50
0
50
100
150
200
250
Time (sec)
Load
Ang
le(°
)
UPFC
PI
NC
PI
Responses of the Generator for a 180 ms 3- phase Short Circuit at bus 2 at P=0.8 p.u & Q=0.15 p.u
Load angle
Speed response
4 5 6 7 8 9 10 110.96
0.98
1
1.02
1.04
1.06
1.08
1.1
Time (sec)
Spe
ed (
Pu)
UPFC
PI
4 5 6 7 8 9 10 110.96
0.98
1
1.02
1.04
1.06
1.08
1.1
Time (sec)
Spe
ed (
pu)
UPFC
PI
NC
PI
Micro-Machine Research Lab. at the University of Natal, Durban, South Africa
Gen. #1: Trans. Line Impedance Increase
10 10.5 11 11.5 12 12.5 13 13.5 14 14.5 1520
25
30
35
40
Time in seconds
Load
ang
le in
deg
rees
DHP_CONV
CONV_PSS_CONVCON_CONV
10 10.5 11 11.5 12 12.5 13 13.5 14 14.5
0.97
0.98
0.99
1
1.01
Time in seconds
Ter
min
al v
olta
ge in
pu
DHP_CONV
CONV_PSS_CONV
CONV_CONV