PyCon PH 2014 - GeoComputation
-
Upload
engr-ranel-padon -
Category
Technology
-
view
857 -
download
2
Transcript of PyCon PH 2014 - GeoComputation
GEOCOMPUTATIONEngr. Ranel O. Padon
PyCon PH 2014 | [email protected]
http://www.gctours.net/product_images/uploaded_images/grand-canyon-hd720.jpg
Full-Time Drupal Developer (CNN Travel)
Lecturer, UP DGE (Java/Python OOP Undergrad Courses)
Lecturer, UP NEC (Web GIS Training Course)
BS Geodetic Engineering in UP
MS Computer Science in UP (25/30 units)
Involved in Java, Python, and Drupal projects.
ABOUT ME
The role of Python in implementing a rapid and
mass valuation of lots along the Pasig River
tributaries.
This is the story of what we have done.
ABOUT MY TOPIC
I• PRTSAS BACKGROUND
II• VALUATION COMPONENT
III• AHP MODELING
IV• RECOMMENDATIONS
TOPIC FLOW
OF FLOOD AND MEN
http://www.reynaelena.com/wp-content/uploads/2009/09/ondoy-aftermath-by-wenzzo-pancho.jpg
http://1.bp.blogspot.com/-sdUQ_XBc5o8/TnfOuNASgjI/AAAAAAAAAug/u-OQ1Cv5oEg/s1600/Ondoymissionhospital.jpg
http://filsg.com/download/ondoy16.jpg
Man is a complex being:
he makes deserts bloom - and lakes die.
GIL SCOTT-HERON
http://i.dailymail.co.uk/i/pix/2011/05/28/article-0-0C4E40E200000578-673_468x301.jpg
http://d2tq98mqfjyz2l.cloudfront.net/image_cache/1254443971159430.jpeg
PASIG RIVER | BEFORE
http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html
PASIG RIVER | AFTER
http://ourss14blog.blogspot.com/2011/10/article-xii-national-economy-and.html
PRTSAS = Pasig River Tributaries Survey and Assessment Study
PRTSAS = PRRC + UP TCAGP
Aims to gather baseline information on the physical
characteristics of major and minor tributaries of the Pasig River.
The gathered information will be used to properly manage the
river and correctly steer its rehabilitation.
BACKGROUND | PRTSAS
“To transform
Pasig River
and its environs
into a showcase
of a new quality
of urban life.”
BACKGROUND | PRTSAS | PRRC
http://www.prrc.gov.ph/
Restore the Pasig River to its
historically pristine condition by
applying bio-eco engineering and
attain a sustainable socio-economic
development.
Relocation of formal and informal
settlers.
Regulate the 3-m easement.
BACKGROUND | PRTSAS | PRRC
BACKGROUND | PRTSAS | UP TCAGP
http://dge.upd.edu.ph/dge/about/about-tcagp/
Research and extension arm of UP DGE.
Large-Scale Projects:
DREAM (DOST NOAH)
PRTSAS
PRS 92 R&D and Implementation Support
BACKGROUND | PRTSAS | UP TCAGP
PRTSAS has 5 major components:
Parcel/As-Built Survey
Hydrographic Component
Water Quality/Environmental Impact
Easement and Adjoining Lots Valuation
Web GIS
BACKGROUND | PRTSAS | COMP.
BACKGROUND | PRTSAS | COVERAGE
BACKGROUND | PRTSAS | COVERAGE
To perform individual valuation work of the PRRC proposed
relocation sites.
To perform a rapid appraisal of the 3-meter easements and
adjoining lots for all tributary locations.
To develop and perform an automated GIS-assisted valuation
of the lots adjoining all tributaries.
VALUATION | DUTIES
VALUATION | THE TEAM
Develop a GIS-assisted valuation model and
perform automated valuation of lots
adjoining the tributaries.
VALUATION | OVERVIEW
VALUATION | EASEMENT CONDITION
Fully-Developed
Partially-Developed
Undeveloped/
Depreciated
determined by the highest price a property can command
if put up for sale in an open market
determinations are made from market evidence or
transactions and found on published market listings or
information from market participants.
VALUATION | MARKET VALUE
The ultimate question is: how do you value a land?
And how do you value lands with huge coverage rapidly?
VALUATION | MARKET VALUE
http://blog.melvinpereira.com/wp-content/uploads/2011/04/man-thinking.jpg
http://e.peruthisweek.e3.pe//ima/0/0/0/1/5/15908/624x468.png
AHP Model Formulation
Geospatial Data Buildup
Market Value Geoprocessing
Market Value Map
GENERAL PROCESS FLOW
ArcPy
http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png
http://www.lic.wisc.edu/training/Images/arcgis.gif
http://www.logilab.org/
Analytic Hierarchy Process is a decision-making method
based on mathematics and psychology developed by Prof.
Thomas L. Saaty in the 1970s.
The input can be obtained from actual measurements such
as price, weight, etc. and from subjective opinion such as
satisfaction feelings and preferences.
AHP
http://www.nae.edu/File.aspx?id=41107
used in scientific and business contexts
useful in situation with scarce, but high-quality or high-
importance data
80/20 Principle: essential information (80%) could be
expressed by just a small but important set of data (20%)
unlike the case of face recognition problem which
requires voluminous data to be stable
AHP
http://www.nae.edu/File.aspx?id=41107
AHP | CHOOSING A LEADER
http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process
AHP | CHOOSING A LEADER
BRAIN
http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process
AHP | CHOOSING A PARTNER
1. Parameters
II. Weights of Parameters
AHP | MURPHY’S LAW OF LOVE
BEAUTY AVAILABILITY
BRAIN
B· B· A = k
AHP | I. PARAMETERS
Intelligence
Values
Humor
Beauty
Wealth
Religion
Health
Interests
Sports
Zodiac Sign
and so on
Choosing a partner
AHP | I. PARAMETERS
Use statistical software to evaluate if some factors
could be eliminated, values to watch out:
1.) Kaiser-Meyer-Olkin (KMO) Coefficient –
tests whether the partial correlations among variables are small
2.) Barlett’s Test for Sphericity (BTS) –
tests whether the correlation matrix is an identity matrix
Choosing a partner
AHP | I. PARAMETERS
Why Dimensionality Reduction?
To simplify data structures
Conserve computing and/or storage resources
Examples: Face Recognition, MP3 and JPEG file formats,
Douglas-Peucker Algorithm
AHP | I. PARAMETERS
Dimensionality Reduction | EigenFaces
Principal vectors used in the problem of human face recognition
http://cognitrn.psych.indiana.edu/nsfgrant/FaceMachine/faceMachine.html
AHP | I. PARAMETERS
Dimensionality Reduction/Factor Analysis
Is the strength of the relationships
among variables large enough?
Is it a good idea to proceed a factor analysis for the data?
Choosing a partner
AHP | II. WEIGHTS OF PARAMETERS
Possible major components after Factor Extraction
1. Humor
2. Beauty
3. Intelligence
Choosing a partner
AHP | II. WEIGHTS OF PARAMETERS
Sample Preference Matrix (3 Parameters)
Choosing a partner
Criteria More
ImportantIntensity
A B
Humor Beauty A 5
Humor Intelligence A 7
Beauty Intelligence A 3
AHP | II. WEIGHTS OF PARAMETERS
Choosing a partner
AHP | II. WEIGHTS OF PARAMETERS
As you might observed, we need to reduce the
number of parameters so that the respondents/evaluators
will just have to evaluate the smallest preference matrix possible.
Choosing a partner
AHP | FINAL PARAMETERS’ WEIGTHS
Apply the AHP algorithm to compute the relative weights,
possible result:
0.60 Humor
0.25 Beauty
0.15 Intelligence
Choosing a partner
AHP | FINAL PARAMETERS’ WEIGTHS
Optimum Partner (among alternatives/suitors)
= 0.60 Humor + 0.25 Beauty + 0.15 Intelligence
Choosing a partner
AHP | VALUING A LAND
1. Parameters
II. Weights of Parameters
III. Weights of Sub-Categories
http://i.domainstatic.com.au/b432bfa9-1e06-4d69-812e-ea14e22d0112/domain/20108120961pio04192711
AHP | I. PARAMETERS
Lot Shape
Topography
Easement Condition
Neighborhood Classification
Accessibility to Main Roads
Corner Influence
Land-Use Type
Proximity to Commercial Area
Proximity to Churches
Proximity to Markets
Proximity to School
Proximity to LGUs
Existing Improvements
Public Utilities
and so on
Obtaining the optimal land value
AHP | I. PARAMETERS
AHP | I. PARAMETERS
We used SPSS for computing the KMO and BTS
Coefficients.
1.) KMO > 0.5
2.) BTS < 0.001
SPSS also provides validation values that could be used
when we decide to automate the process in pure Python later.
Choosing a partner
Factor Analysis (18 raw & unordered variables)
AHP | I. PARAMETERS
Extracted Factors
AHP | I. PARAMETERS
Land-Use
Accessibility
Lot Size
Lot Shape
Neighborhood
AHP | II. WEIGHTS OF PARAMETERS
Sample Preference Matrix (4 Parameters)
Choosing a car: 4 Params, 6 Comparisons
Criteria More
ImportantIntensity
A B
Cost Safety A 3
Cost Style A 7
Cost Capacity A 3
Safety Style A 9
Safety Capacity A 1
Style Capacity B 7
AHP | II. WEIGHTS OF PARAMETERS
Actual Data
Obtaining the Optimal Value : 5 Params, 10 Comparisons
The CSV File
AHP | II. WEIGHTS OF PARAMETERS
AHP Algorithms (Ishizaka & Lusti, 2006)
1. The Eigenvalue Approach (Power Method)
2. The Geometric Mean
3. The Mean of Normalized Values
AHP | II. WEIGHTS OF PARAMETERS
3. The Mean of Normalized Values
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
Effective AHP parameters
Parameter Weight
Land Use 0.372
Location/Accessibility 0.276
Lot Size 0.125
Lot Shape 0.111
Neighborhood Classification 0.116
AHP | II. WEIGHTS OF PARAMETERS
Some issues for the computation of our
AHP parameters:
1.) Assumes all respondents have
consistent preference matrices
2.) Uses the arithmetic mean for computing the
effective parameter weights across
all the respondents.
AHP | II. WEIGHTS OF PARAMETERS
consistency means that if A>B and B>C then A>C,
where A, B, and C, refer to the criteria/parameters
of the land value.
It also means that if A > 2*B and B > 3*C then A > 6*C,
as the number of criteria increases, it's more difficult
to be consistent
AHP | II. WEIGHTS OF PARAMETERS
We have implemented the proposed Saaty's
Consistency Measure of the preference matrix of the
respondents but we have found it to be too limiting.
AHP | II. WEIGHTS OF PARAMETERS
Pelaez and Lamata (2002) proposed a new way of
computing the Consistency Index and that is by using
the concept of determinants.
We implemented their paper using Python and
NumPy and we obtained a better filtering for the
consistent survey answers.
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
However, [Aragon, et al (2012)], shown that it is
better to use the geometric mean than the
arithmetic mean of the AHP parameters' weights.
We re-implemented the effective parameters' weights
using the geometric mean of all weights across all
respondents.
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
AHP | II. WEIGHTS OF PARAMETERS
There are two approaches [Aragon, et al (2012)]
for solving the effective parameters:
(1) EIW: Effective Individual Weights
computes the individual parameters' weights and
get their geometric mean
(2) WEPM: Weights of the Effective Preference Matrix
get the geometric mean of all the preference matrices
and compute the parameters' weights.
AHP | II. WEIGHTS OF PARAMETERS
We implemented both approaches in combination
with the 3 AHP algorithms for comparison and validation.
AHP | II. WEIGHTS OF PARAMETERS
Finally, we will use the following result
(using the Weights of the Effective Preference Matrix
of the Mean of Normalized Values AHP Algorithm)
AHP | II. WEIGHTS OF PARAMETERS
AHP allows hierarchies/subcategories
Phase III for gathering the sub-categorical weights or
adjustment factors
AHP | III. SUBCATEGORY WEIGHTS
AHP | III. SUBCATEGORY WEIGHTS
Geometric Mean of all survey data
AHP | III. SUBCATEGORY WEIGHTS
(Context is Per Estero)
Computed Unit Market Value =
Average Market Value * (
Land-Use * (Commercial|Industrial|Residential…)
+ Accessibility *(Proximity to POIs and Access to Roads)
+ Lot Area * (Preferred|Not-Preferred)
+ Lot Shape * (Quadrilateral|NonQuadrilateral)
+ Neighborhood Classification * (Formal|Informal)
)
AHP | FINAL PARAMS AND WEIGHTS
(Context is Per Estero)
Computed Unit Market Value =
Average Market Value * (
0.4287 * (1.5148l|1.1308|1.1288|1.0080|1.0000)
+ 0.2809 *(0..1)
+ 0.1119 * (1.5599|0.3338)
+ 0.0988 * (1.3831|0.5997)
+ 0.0797 * (1.4082|0.5696)
)
AHP | FINAL PARAMS AND WEIGHTS
AHP | GIS
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Key_aspects_of_GIS/00v20000000r000000/
AHP | ArcPy
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/
AHP | ArcPy
AHP | ArcPy
http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/Working_with_geometry_in_Python/002z0000001s000000/
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | ArcPy
AHP | VALIDATION
AHP | WELCH’S TEST
AHP | MARKET VALUE MAP
AHP | MARKET VALUE MAP
AHP | MARKET VALUE MAP
Pure Python Pipeline:
SPSS <= RPy2 or Pandas (Python Data Analysis Library)
ArcGIS (ArcPy) <= QGIS (PyQGIS)
RECOMMENDATIONS
AHP Model Formulation
Geospatial Data Buildup
Market Value Geoprocessing
Market Value Map
GENERAL PROCESS FLOW
ArcPy
http://ithelp.port.ac.uk/images/SPSS-logo-32F23C8B51-seeklogo.png
http://www.lic.wisc.edu/training/Images/arcgis.gif
http://www.logilab.org/
AHP Model Formulation
Geospatial Data Buildup
Market Value Geoprocessing
Market Value Map
RECOMMENDATIONS
PyQGIS
http://pandas.pydata.org/
http://rpy.sourceforge.net/rpy2/doc-dev/html/index.html
http://trac.osgeo.org/qgis/chrome/site/qgis-icon.png
RECOMMENDATIONS | BOOKS
http://locatepress.com/
This comprehensive article demonstrates the tight integration
of Python’s data analysis and geospatial libraries:
IPython
Pandas
Numpy
Matplotlib
Basemap
Shapely
Fiona
Descartes
PySAL
RECOMMENDATIONS | MASHUP
There are two types of expertise.
One is the type you already know – content expertise,
immersing yourself deeper and deeper in a subject,
practicing for 10,000 hours and all of that.
But I think there’s a connection expertise too.
That comes from going horizontal rather than vertical.
It’s about knowing a little about a lot,
and finding wisdom in how things connect in new and different ways.
MICHAEL STANIER
http://www.speakers.ca/wp-content/uploads/2012/12/Michael-Bungay-Stanier_Feb2-760x427.jpg
Python could be a valuable tool for expanding your knowledge
vertically, as well as horizontally. And, it’s a must have tool for
connectionist experts.
END NOTE
http://fc01.deviantart.net/fs25/i/2009/022/1/a/inject_knowledge_question_mark_by_CHIN2OFF.jpg
Aragon, T., et al (2012). Deriving Criteria Weights for Health Decision Making: A Brief
Tutorial, http://www.academia.edu.
Forman, E. & Selly, M. (2001). Decision By Objectives: How to Convince Others That
You Are Right. World Scientific Publishing Co. Pte. Ltd. Singapore.
Griffiths, D. (2009). Head First Statistics. O’Reilly Media, Inc., 1005 Gravenstein Highway
North, Sebastopol, CA 95472. USA.
Ishizaka, A. & Lusti, M. (2006). How to Derive Priorities in AHP: A Comparative Study.
Central European Journal of Operations Research, Vol. 14-4, pp. 387-400.
Lamata, M. & Pelaez, J. (2002). A Method for Improving the Consistency of Judgements.
International Journal of Uncertainty, Fuzziness, and Knowledge-Based
Systems. Vol. 10, No.6, pp. 677-686. World Scientific Publishing Company.
Pelaez, J. & Lamata, M. (2002). A New Measure of Consistency for Positive Reciprocal
Matrices. Computers and Mathematics with Applications, 46 (8), pp. 1839-1849.
Pornasdoro, K. & Redo, R. S. (2011). GIS-Assisted Valuation Using Analytic Hierarchy Process
and Goal Programming: Case Study of the UP Diliman Informal Settlement Areas
(Undergraduate Thesis).
Uysal, M. P. (2010). Analytic Hierarchy Process Approach to Decisions on Instructional
Software. 4th International Computer & Instructional Technologies Symposium,
Selçuk University, Konya, Turkey, pp. 1035-1040.
REFERENCES
Thank You!