Post on 08-Apr-2020
SDI assessment
Questions
Why there is a need to do an assessment?
How the assessment results are further used?
Assessment/Evaluation• A study designed and conducted to assist some audience to assess
an object’s merit and worth (Stufflebeam, 2000)
• The fundamental purpose of evaluation is to create greater
understanding (Taylor-Powell, et al., 1996)
2
Assessment framework
• System of methods and processes to support
evaluation/assessment.
Assessment purposes (Chelimsky, 1997)
Accountability – to test if the
program works
Knowledge – to better understand
the program
Developmental – to improve the
program
Formative
evaluation
Summative
evaluation
„The STIG – A new SDI assessment method” (2015)
Bujar Nushi, Bastiaan Van Loenen, Joep Crompvoets
„Efficiency, Effectiveness, and User Satisfaction of Web-Based Ontology
Browsers” (2015) Turkan Karakus Yilmaz, Umut Durak, Halit Oguztuzun,
Kursat Cagiltay
„Assessing Corporate Spatial Data Infrastructures” (2013) LP de Vries
„Towards an online Self-Assessment Methodology for SDIs” GA Giff, J
Jackson
„Assessing the Development of Kenya National Spatial Data Infrastructure
(KNSDI)” (2014). J Okuku, A. Bregt, L Grus
„Multi-view SDI assessment of Kosovo” (2012) B Nushi, B Van Loenen, JWJ
Besemer, J Crompvoets
Inspire monitoring and reporting
7
Inspire and assessment
9
Monitoring
Quantitative
Yearly
Reporting
Qualitative
Every 3 yrs
Some principles of assessing SDIs13
Complex nature of a SDI;
Truly complex problems can only be approached
with complex resources (Cilliers, 1998);
Multi-faceted view is needed in understanding
concrete SDI initiative (De Man, 2006);
Use multiple assessment methods and
approaches;
Do not oversimplify;
Incorporate different views/understandings;
Flexibility;
Assessment result
Framework applied in 21 countries
Data collected by means of survey
Survey filled in by SDI coordinators (on behalf of
authorized SDI institution)
4 SDI assessment approaches were used:
Clearinghouse, SDI readiness, INSPIRE State of Play,
Organizational
SDI assessment
Clearinghouse suitability approach (in%)
49
0
100
50
76
60
0
47
0
46
76
75
49
0
36
0
96
0
52
41
0 20 40 60 80 100 120
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
SDI Readiness approach (in%)
53
56
64
59
66
53
65
42
41
58
39
58
32
59
48
37
70
37
55
50
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
INSPIRE State of Play approach (in%)
52
50
74
50
76
59
59
59
27
65
44
73
55
59
39
55
71
32
52
56
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
Organizational approach (in%)
50
75
100
75
100
75
75
75
50
100
50
75
50
75
50
50
75
50
50
70
0 20 40 60 80 100 120
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
Average results in percentages from all assessment approaches
85
79
78
70
67
62
58
56
54
52
52
51
50
48
46
45
43
35
30
29
0 10 20 30 40 50 60 70 80 90
Canada
Colombia
Spain
Mexico
Jamaica
Cuba
Chile
Ecuador
Average per sample
Malaysia
Uruguay
Argentina
Denmark
Netherlands
Nepal
Brazil
Poland
Serbia
Turkey
Guyana
Assessment results
One year later another measurement of the Dutch SDI
was performed
Clearinghouse suitability approach (in%)
49
0
100
50
76
60
0
47
0
46
76
75
49
0
36
0
96
0
52
41
0 20 40 60 80 100 120
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
December
2007
Clearinghouse suitability approach (in%)
49
0
100
50
76
60
0
47
0
46
76
75
49
79
36
0
96
0
52
45
0 20 40 60 80 100 120
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
October
2008
Differences in Clearinghouse suitability SDI
approach
National Georegister “almost” ready
Clearinghouse suitability indicators measured
georegister
www.nationaalgeoregister.nl
SDI Readiness approach (in%)
53
56
64
59
66
53
65
42
41
58
39
58
32
59
48
37
70
37
55
50
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
December
2007
SDI Readiness approach (in%)
53
56
64
59
66
53
65
42
41
58
39
58
32
71
48
37
70
37
55
51
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
October
2008
Differences in SDI readiness approach results
Indicators December 2007 October 2008
Politician vision regarding SDI 0.35 0.8
Institutional leadership 0.5 0.65
Umbrella legal agreement(s) 0.65 0.8
Digital cartography availability 0.65 0.8
Metadata availability 0.5 0.5
Human Capital 0.99 0.99
SDI culture 0.65 0.8
Individual leadership 0.65 0.65
Web connectivity 0.73 0.73
Telecommunication infrastructure 0.68 0.68
Geospatial software availability 0.65 0.8
Own geoinformatics development 0.65 0.8
Open source culture 0.35 0.5
Government central funding 0.5 0.65
Return on investment 0.35 0.65
Private sector activity 0.5 0.5
INSPIRE State of Play approach (in%)
52
50
74
50
76
59
59
59
27
65
44
73
55
59
39
55
71
32
52
56
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
December
2007
INSPIRE State of Play approach (in%)
52
50
74
50
76
59
59
59
27
65
44
73
55
71
39
55
71
32
52
57
0 10 20 30 40 50 60 70 80
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
October
2008
INSPIRE State of Play approach - changes
INDICATOR Dec 2007 Oct 2008
The SDI-initiative has a long-term and clear vision about the national SDI 0,5 1
There is documented data quality control procedures applied at the level of the national
SDI
0 0,5
Concern for interoperability goes beyond conversion between data formats (e.g.
hardware/software/data definitions)
0,5 1
One or more standardized metadata catalogues are available covering more than one data
producing agency
0,5 1
One national on-line access service for metadata (clearinghouse) is available providing
metadata of more than one data producing agency
0 0,5
There are one or more web mapping service available for core spatial data 0,5 1
Organizational approach (in%)
50
75
100
75
100
75
75
75
50
100
50
75
50
75
50
50
75
50
50
70
0 20 40 60 80 100 120
Argentina
Brazil
Canada
Chile
Colombia
Cuba
Denmark
Ecuador
Guyana
Jamaica
Malaysia
Mexico
Nepal
Netherlands
Poland
Serbia
Spain
Turkey
Uruguay
Average per sample
Average results in percentages from all assessment approaches
85
79
78
74
70
67
62
58
56
54
52
52
51
50
46
45
43
35
30
29
0 10 20 30 40 50 60 70 80 90
Canada
Colombia
Spain
Netherlands
Mexico
Jamaica
Cuba
Chile
Ecuador
Average per sample
Malaysia
Uruguay
Argentina
Denmark
Nepal
Brazil
Poland
Serbia
Turkey
Guyana
Assessing the Dutch SDI
Embedding GEO in eGovernment
Key GEO registers
Implementing INSPIRE
Creating added value
Knowledge, innovation and edu.
Organization, steering, directing
Cooperation
Supply optimization
Assessing GIDEON – Dutch GII
Ministrie I&M asked for
GIDEON implementation progress reporting and
monitoring (yearly)
1st GIDEON assessment type
2nd GIDEON assessment type
3rd GIDEON assessment
type
Phase 2 Assessment approachIndicator 1. The number of visitors of the Dutch national georegister.
Indicators 2. Availability of datasets and services
Indicator 3. The use of view and download services
Indicator 2. The percentage of datasets from GIDEON annex 1
that are available without any restrictions
Indicator 1. General governmental policy terms for (re)use of
geographical information.
}
Indicator 3. Yearly turnover of the geo-information
business in the Netherlands
Indicator 1. The level of cooperation within 5 chains of the GIDEON.
Indicator 2. The use of geo information within
e-government processes
Indicator 1. The number of Geo-information events
Indicator 2. The number of unfulfilled vacancies in geo-sector
Indicator 3. Expenditure of Geo ICT sector in the Netherlands
on research and development.
Indicator 4. Expenditure of research sector on R&D.
}
}
{
Indicator 1.1 Gemiddelde aantal bezoekers NGR per dag
Hotspots van het aantal terugkerende gebruikers van het NGR
Indicator 1.2 Beschikbaarheid van datasets en services
Indicator 2.1: Algemeen overheidsbeleid voor het (her)gebruik van geo-informatie
Indicator 2.2: Het % datasets dat beschikbaar is zonder gebruiksbeperkingen
Indicator 2.3: Jaarlijkse omzet van de geo-bedrijfssector in Nederland
Indicator 4.1: Het aantal geo-events in Nederland per jaar
Conclusions
Assessment often an integral part of SDI
Difficulties due to SDI complexity
Helps to monitor the implementation and
achievement of goals
53
Questions?
54