InSPIRe Australian initiatives for standardising statistical processes and metadata Simon Wall...
-
Upload
rose-banks -
Category
Documents
-
view
214 -
download
0
Transcript of InSPIRe Australian initiatives for standardising statistical processes and metadata Simon Wall...
InSPIRe
Australian initiatives for standardising statistical processes and metadata
Simon Wall
Australian Bureau of Statistics
December 2012
1
2
Increasing cost & difficulty of
acquiring data
New competitors &
changing expectations
Rapid changes in the environment
Competition for skilled resources
Reducing budget
Riding the big data wave
Challenges facing NSIs
• Reduce the cost and time of doing business• Grow the business through new statistical products
and services• Deliver the first large scale digital Census (2016) on
time, to budget and quality
while delivering Business As Usual.
3
Challenges facing ABS
• Through large scale Innovation across the whole ABS
• We will:
– radically transform the way we acquire, collate, use, reuse and disseminate statistical information
• By:
– industrialising, modernising and reengineering our business processes
– reengineering our statistical infrastructure and the way we manage information
– develop capability needed to meet future needs
– Collaborating with other international NSIs4
How will we get there?
InSPIRe
5
What is InSPIRe?
• Infrastructure for Statistical Process and Information Management Re-engineering
• Building core infrastructure in the ABS for:– Business Process management
– Information management
• SWM – Statistical Workflow Management System
• MRR– Metadata Registry and Repository
6
SWM
• The Statistical Workflow Management System
• An environment that will be used to develop and manage common business processes – These processes can drive and/or be used by
tools and applications.
7
MRR
• The Metadata Registry and Repository consists of two parts:
– Repository • The centralised ‘bucket’ to store standards
based metadata.
– Registry• The catalogue that lets you find out what is in
the repository.
8
MRR
• Registers and stores a wide range of information, including:– Metadata (i.e. Classifications, Variables, etc)
– Data – registered in MRR, stored in EDW
– Process Definitions and Paradata – information to run a process
– Process Metrics – information about a process that was run.
9
MRR
• The MRR– Enforces registration of the information, ensuring consistent
documentation– Reads information in a range of standard supported metadata
formats• (initially DDI and SDMX)
– Re-issues metadata in whatever standard format is required by a given process.
• Regardless of the format in which the metadata was originally created in.
– This information is stored for use and later re-use by the metadata driven processes in SWM
– Enables the searching and discovery of metadata for re-use.
10
Interfaces
• Processes and Systems will interact with the MRR through a web service interface that supports a controlled set of standards.– Currently limited to DDI 3.1 and SDMX 2.1
11
InSPIRe
SWMSWMSearchSearch
Business ProcessBusiness Process
Users
User InterfaceUser Interface
MRRMRR
RetrieveRetrieve RegisterRegister
Benefits of InSPIRe
• Basis for greater automation and reuse of processes and metadata
• Faster to market– For individual collections– For building new collections
• Cheaper to market (long term)– For individual collections– For building new collections
13
Progress so far…
• Proof of Concept Project (2010/2011)– A group of simple use cases to prove the idea of the
MRR and SWMs.
• Demonstrating:– Benefits
– Potential functionality
– What is technically achievable
• Building capability
14
Pathfinders
• Pathfinder Integration (July 2012)– Expanded on PoC
– ‘Pathfinder’ projects to produce further use cases for the MRR and SWMs.
• Not end-to-end processes, but isolated ‘snap shot’ projects to demonstrate the capabilities of InSPIRe.
• Four pathfinders chosen– Including web data capture (e-forms) and REEM (Remote
Execution Environment for Microdata)
REEM and Web Data Capture Pathfinders
• Proved integration with real ABS systems
• Demonstrated the benefits of integration with InSPIRe, namely:
– Re-use of processes (in SWM)
– Standards based metadata (DDI in MRR) driving business tools
– Automating business processes (in SWM)
Early Adopter Projects
• Projects which are positioning to integrate with InSPIRe within the next 12 months.– Includes projects focusing on administrative data,
e-forms and the 2016 Census.
• Suitable candidates to prove the operation InSPIRe.
– implementing processes in different phases of the GSBPM.
17
What we have achieved so far
• MRR– Design and review, mappers, shredders, database,
automated generation of metadata types, basic search, registry model
• SWM– Environment setup, test cases with projects
• Capability Development– Best practice guidelines for implementing DDI, metadata
content guidelines, InSPIRe integration information packs
18
What we still have to do
• MRR– Production version
• SWM– Further process definition, integration with other projects
• Metadata Authoring Environment– A mechanism for:
• Taking existing metadata content and registering it in the MRR
• Creating new metadata content via a content creation tool and registering it in the MRR
19
The future…An example
20
21
Survey Designer
Retrieve Store
Content Creation
Tool
e-form Transformation
EnvironmentWeb form
MRR
DDI DDI
Register
Data Collection
DDI
EDW
Retrieve StoreSWM