The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2....
Transcript of The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2....
![Page 1: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/1.jpg)
The parallelization of geoscience apps at C3L with Azure
Prof J Craig Mudge FTSE
Collaborative Cloud Computing Lab (C3L)
![Page 2: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/2.jpg)
Outline
1. A no-machines lab
2. Magnetotelluric method for imaging sub-surface
3. Cleaning and processing the data - weeks
4. Results
5. Future work
![Page 3: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/3.jpg)
A no-machines Lab
eScience enabled by cloud computing
Seed funding from -- department of mines www.pir.sa.gov.au -- MSFT Research Jim Gray Seed Grant
Started June 2010
![Page 4: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/4.jpg)
Magnetotelluric (MT) imaging 1. Using the magnetic and electric
fields of the earth, MT imaging determines the resistivity structure of a sub-surface area of interest.
2. It goes deeper (hundred or so Km) than seismic (<2 Km) but does not have the same resolution
3. Applications 1. mineral exploration, 2. water management in mining, 3. geothermal exploration, 4. carbon storage, 5. aquifer research and management 6. earthquake and volcano studies.
[email protected] 27 sep 2010
CO2 in depleted gas field
(Heinson and Mudge, 2010)
4
![Page 5: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/5.jpg)
Electrical resistivity
![Page 6: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/6.jpg)
Electromagnetic methods
![Page 7: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/7.jpg)
Data logging by University of Adelaide Geophysics, on a geothermal site – Paralana, SA,
Australia
7
![Page 8: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/8.jpg)
Cle
an
Bro
adb
and
pro
cess
ing
E fi
eld
co
nve
rsio
n t
o s
tan
dar
d u
nit
s
Forward Modeling And Inversion
Station 1
Station n
Station 2
MT Station data from logging in the field
Outputs from BIRRP are (a)impedance Z, where E=ZB (b)coherence
data (c) Apparent
resistivity and phase
BIRRP
Co
nve
rt
to E
DI
Co
nve
rt
to E
DI
Co
nve
rt
to E
DI
8
Time series Apparent resistivity
3D
![Page 9: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/9.jpg)
![Page 10: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/10.jpg)
Forward smoothness
Sensitivity
Forward misfit
Sequential Parallel
0 82
0 82
163 60
339 46
0 0
1 1
Total 584 189
,000 secs
[email protected] 10.2.2011
MT 3D inversion Gawlor Craton 6 frequencies
No use of smoothness
![Page 11: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/11.jpg)
![Page 12: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/12.jpg)
![Page 13: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/13.jpg)
![Page 14: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/14.jpg)
MT Inversion Code Reconstruction
Sequential code: FORTRAN 77 22,000 lines
1. Find computational blocks amenable to easy parallelization Sensitivity & Forward modelling computations independent for each
frequency.
2. Separate control logic from numerical computations.
3. Optimization taken over by a parallel workflow control script (Python)
In other words: FORTRAN code split into multiple computational sub-processes driven by the control script
Email [email protected] for TN 2011-3
![Page 15: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/15.jpg)
Adelaide
Enhanced Geothermal System
Hot rocks
![Page 16: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/16.jpg)
Current Highlights Groundwater
Nepabunna, Flinders Ranges
Perth Geothermal
Direct heat – urban
Potential: Extraction of deep heat; perhaps part of Square Kilometre Array
![Page 17: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/17.jpg)
Future areas
1. Near real-time feedback during data logging
2. Improvements to current MT FORTRAN program
e.g., Parallelise the matrix operations
3. Genetic algorithm – alternative to conjugate gradient
4. Joint inversion – seismic + MT + gravity
5. Our modelling accessible as a cloud application available to explorers world-wide
17
![Page 18: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/18.jpg)
(Rachel Maier, 2010)
Renmark Trough
5 10 15 20 25 30 35
0
1
2
3
4
5
resis
tivity
(ohm
.m)
depth
(km
)
distance (km)
RMSMT
=2.30
0.5
1
1.5
2
5 10 15 20 25 30 35
0
1
2
3
4
5
resis
tivity
(ohm
.m)
depth
(km
)
distance (km)
RMSJI=5.3 RMS
MT=5.3 RMS
GV=4.5
0
0.5
1
1.5
2
D
epth
(km
)
Distance (km)
Devonian D=2360
Basement D=2800
NE SW
Seismic constrained Gravity
MT Inversion
Joint Inversion 5 10 15 20 25 30 35
0
1
2
3
4
5
resis
tivity
(ohm
.m)
depth
(km
)
distance (km)
RMSMT
=2.30
0.5
1
1.5
2
5 10 15 20 25 30 35
0
1
2
3
4
5
resis
tivity
(ohm
.m)
depth
(km
)
distance (km)
RMSJI=5.3 RMS
MT=5.3 RMS
GV=4.5
0
0.5
1
1.5
2
5 10 15 20 25 30 35
-40
-30
-20
-10
gz (
mG
als
)
distance (km)
100
101
102
TE data
TE model response
TM data
TM model response
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
100
101
102
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
5 10 15 20 25 30 35
-40
-30
-20
-10
gz (
mG
als
)
distance (km)
100
101
102
TE data
TE model response
TM data
TM model response
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
100
101
102
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
5 10 15 20 25 30 35
-40
-30
-20
-10
gz (
mG
als
)
distance (km)
100
101
102
TE data
TE model response
TM data
TM model response
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
100
101
102
10-2
100
102
0
30
60
90
App R
es (
.m)
phase
period (s)
![Page 19: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/19.jpg)
Future areas
1. Near real-time feedback during data logging
2. Improvements to current MT FORTRAN program
e.g., Parallelise the matrix operations
3. Genetic algorithm – alternative to conjugate gradient
4. Joint inversion – seismic + MT + gravity
5. Our modelling accessible as a cloud application available to explorers world-wide
19
![Page 20: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/20.jpg)
Acknowledgements
Ed Lazowska
Gordon Bell
Dennis Gannon
Catharine van Ingen
John Warren
![Page 21: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/21.jpg)
Acknowledgements
Graham Heinson
Pinaki Chandrasekhar
Wei Wang
Stephan Thiel
National geophysics facility - magnetotellurics
![Page 22: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/22.jpg)
Summary
1. A no-machines lab - June 2010
2. Magnetotelluric method
3. Processing the data
4. Results -- weeks down to days
5. Future work
Production and development: almost 100% Azure
eScience – geothermal, aquifer, as well as minerals
Modelling on demand for explorers world-wide
![Page 23: The parallelization of geoscience apps at C3L with Azure ... · A no-machines lab 2. Magnetotelluric method for imaging sub-surface 3. Cleaning and processing the data - weeks 4.](https://reader034.fdocuments.us/reader034/viewer/2022042017/5e752d17711195521721ad44/html5/thumbnails/23.jpg)
Thanks and
questions
www.cloudinnovation.com.au
+61 417 679 266 +1 650 224 2111