Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess...

13
Modernization of Statistical Information Systems Speaker presentation: Australian Bureau of Statistics: Transforming International Statistical Systems /1 1 This document was contributed by the Australian Bureau of Statistics. It has been reproduced without formal editing. The views expressed are those of the authors and do not necessarily reflect the views of the United Nations 09 December 2010 English only Economic and Social Commission for Asia and the Pacific Side event to the Committee on Statistics, second session Bangkok, 17 December 2010

Transcript of Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess...

Page 1: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

Modernization of Statistical Information Systems

Speaker presentation: Australian Bureau of Statistics: Transforming International Statistical Systems /1

1 This document was contributed by the Australian Bureau of Statistics. It has been reproduced without formal editing. The views expressed are those of the authors and do not necessarily reflect the views of the United Nations

09 December 2010 English only

Economic and Social Commission for Asia and the Pacific Side event to the Committee on Statistics, second session Bangkok, 17 December 2010

Page 2: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

1

Transforming International Statistical Systems

Australian Bureau of Statistics

The environment National Statistical Offices operate in has changed

• Complex interlinked problems

• Expectations of speed

• Expectations of detail

• Expectations of dynamic (personalised) information services

Page 3: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

2

The business of NSIs is evolving

• Was about collection and compilation of particular statistics

• Is now about transforming large quantities of data from multiple sources into cogent, coherent bodies of detailed statistical information

• Quality assurance, transparency, trustworthiness of the information services will remain crucial

Managing information throughout it's lifecycle

• Superficially the demand is for websites, dissemination systems, visualisation tools– That's a symptom, the generation of the necessary

information with rich metadata is the strategic issue

• Preparation needed to describe and disseminate detailed data responsively and flexibly

• Transparency and detailed descriptions cannot be added in at the end of the process, during dissemination

• Assembling consistent metadata during, in fact as a central part of, the production process is the way forward

• “Active” metadata driving automated end to end production systems

Page 4: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

3

Industrialisation / practical level harmonisation amongst NSIs

• No individual NSI can afford the funds, or the human resources to achieve the change in a reasonable timeframe

• Co-operation, collaboration is a solution and that requires genuine harmonisation of methods, processes and IT systems

• Statistical Information Management is essential :– so NSIs can truly industrialise / collaborate, and– because of the nature of the challenge we are facing

Page 5: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

Statistical Journal of the IAOS 26 (2009/2010) 125–133 125DOI 10.3233/SJI-2009-0712IOS Press

The case for an international statisticalinnovation program – Transforming nationaland international statistics systems

Brian Pinka, Jenine Borowikb and Geoff Leec

aAustralian Statistician, Australian Bureau of Statistics, AustraliabChief Information Officer, Australian Bureau of Statistics, AustraliacChief, Information Management Transformation Program, Australian Bureau of Statistics, AustraliaE-mail: {brian.pink,jenine.borowik,geoff.lee}@abs.gov.au

Abstract. National Statistical Institutions (NSIs) face significant challenges in a rapidly evolving information environment.Although considerable networking and information sharing occurs amongst NSIs, there are further opportunities to work togetherto improve the relevance of statistical frameworks and standards and to more rapidly evolve, share and connect our technologyinfrastructures. Action in these areas has the potential to help NSIs improve the reach and agility of official statistical services.

1. Introduction

National Statistical Institutions (NSIs) face manycritical challenges in the 21st Century. This paper out-lines these challenges, presents a vision for 21st Cen-tury NSIs, explores the future capabilities required andoutlines issues and approaches for working as a com-munity of organisations.

2. Challenges facing National StatisticsInstitutions in the 21st Century

With the constraints of limited funding, full workprograms and aging infrastructure, and the challengesof a rapidly evolving environment, NSIs need to worktogether to succeed. Two key things NSIs need to doto speed up response to change are to increase the rele-vance of statistical frameworks, standards and classifi-cations to contemporary issues and to work out a wayto more rapidly evolve, share and connect technologyinfrastructures.

Governments and international organisations aroundthe world make large investments to manage critical is-sues. They need information to help the formulation of

good policy, assess the impact of the investments andrefine government programs. The range of informationrequired has expanded significantly and the urgencywith which it is needed has accelerated, driven by anincreasingly complex and interconnected world. Someof the critical areas relate to recognition by society ofurgent social problems, global financial systems fail-ure, globalisation, terrorism, climate change and a newemphasis on sustainability.

Complex problems have multiple causes, and requiredifferent information views. However, the differentviews need some coherence to reduce confusion andsupport effective evidence-based decisions. NSIs havea key role in assisting coherent and logical analysis ofdata from multiple sources. The fundamental businessof NSIs is to assist governments, business and com-munities to make informed decisions. As institutions,individually and collectively, they work to provide acoherent base of information with accurate measure-ment of change over time in key economic, social, de-mographic and environmental indicators. NSIs pro-vide additional value through attention to coherence ina number of dimensions: particular areas of nations;key issues and groups (the young, the aged, indigenouspeople, disadvantage, health, the economy, the envi-

1874-7655/09,10/$27.50 2009/2010 – IOS Press and the authors. All rights reserved

Page 6: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

126 B. Pink et al. / The case for an international statistical innovation program

ronment, etc.); comparison of individual nations withsimilar nations; and international statistics (either forregions of the world or the entire world).

Citizens are expecting more from government.There is a resurgence of social conscience in particu-lar fields, perhaps enabled by improved education andaccess to information and like-minded people throughtechnology. There is often strong community inter-est in assessing the success of government in tacklingproblems and in working with government to designeffective programs. The mechanisms for the commu-nity to raise issues with government are also expand-ing in range and power with the increasing use of theinternet to circumvent traditional media and public re-lations mechanisms. The debates are becoming moreopen, visible and volatile. Governments are increas-ingly looking at how to use web 2.0. to transform gov-ernment services and interaction.

The revolution in the speed with which informationis becoming available is creating new information prod-ucts and new ways of combining and using information.These alternative sources of information are sometimesupdated in real time and are available with fewer con-straints and with greater detail than the informationprovided by NSIs. In addition to the wealth of admin-istrative data generated by business activity, there is anever-increasing pool of data generated from personaldevices, sensors, instruments and computers. Exam-ples of sources are retail scanners, scientific equipment,imaging systems, transport systems, telecommunica-tions networks and even metrics collected about peo-ple’s use of the Internet. There is a blurring of theboundaries of official statistics. A number of organ-isations are publishing statistics, sometimes releasedmore frequently and often widely used to complementor pre-empt the official versions.

Many governments are unwilling to provide addi-tional investment to maintain and expand the core ex-isting authoritative baseline data, but are often willingto invest in rapid responsive approaches which provideanswers to new challenges and questions they are con-fronting.

The current slow and considered approach to devel-oping international and national statistical frameworks,standards and classifications is a significant barrier torapid change. While the approach suits the need forcomparable statistics over time, it can take decades andgreat expense for new versions of existing standards tobe implemented across the world, and for many prag-matic reasons some countries do not ever implementthem. As a result, current official statistical data is not

always seen as fully relevant to new areas of informa-tion enquiry, particularly when compared to some ofthe more agile and informal information sources. Theoutdated nature of frameworks can also complicate theanalysis task, as users attempt to make their own adap-tations to suit the real world. There is a need to con-sider approaches which preserve the comparability ofstatistics but enable rapid ways of producing addition-al statistical models to reflect contemporary changestaking place in economy and society and in differentnational interests, international interests, contemporaryactivities and issues.

Tools and approaches that have worked well in thepast may not serve NSIs so well in the future. Muchof the technology effort of NSIs has been focusedon supporting individual processing models and, al-though NSIs share international frameworks, informa-tion about approaches and sometimes share technology,many of the systems are developed for a single organ-isation’s purposes. Limited technology resources areconsumed by cycles of development and modernisationattempts.

Predicting an environment dominated by the knowl-edge economy and the internet in 10 or 20 years timeis complex. As a result, NSIs need to work togetherto work out what they need to do in order to surviveand prosper in this evolving world. Unless there iscollaborative effort among NSIs, it is not certain thatthey will remain at the heart of official information forsocieties. Facing a similar challenge, newspapers whofail to adapt to new business models are disappearing.

3. A Vision for 21st Century NSIs

NSIs collectively have an important role in evidence-based policy making through enabling comparison ofdata across regions, countries and the world. By sup-porting international statistical standards, they providegovernments with the opportunity to assess the suc-cess of similar policy interventions in other countries,whether prior to making significant investment, dur-ing implementation or once programs are established.For global issues, the importance of coherence acrossthe international statistical system is even more pro-nounced.

The effective NSIs of the future will continue to pro-vide a framework of stable, trusted, regular and coher-ent base of key national and international statistics. Inaddition, they will be able to rapidly combine data fromofficial statistics and a wide variety of other sources to

Page 7: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

B. Pink et al. / The case for an international statistical innovation program 127

produce coherent information relevant to current andemerging issues. They will be responsive and agile atmounting new collections to fill gaps and to answer newquestions. They will maximise the availability of infor-mation through the innovative use of methods and toolswhich allow data to be combined and analysed whileensuring privacy and confidentiality requirements arepreserved. They will further develop and share stronginformation management capabilities and work effec-tively with others to encourage innovation in informa-tion management and use.

As new information sources are developed or dis-covered they will incorporate the most useful into theenduring fabric of information available to their gov-ernments for informed decision making and into thebase of important international statistics. NSIs will takesteps to shape how new information sources are devel-oped and used. They have tools, people and credibilityto be a creative partner in this field.

4. Future capabilities

The Australian Bureau of Statistics (ABS) has beenconsidering what new capabilities might be requiredto meet these challenges, how NSIs and others mightwork together to obtain them, and what actions shouldbe taken now.

4.1. From static data products to “commoninformation services”

Typical NSI websites contain a mix of informationproducts. These include electronic publications sup-ported by more detailed data products in a variety offormats (spreadsheets, data cubes, data warehouses)and other products such as confidentialised microda-ta files and information about standards and classifica-tions. The standards and classifications are often notexplicitly linked to the data products which use them,and searching and finding specific data is often a man-ual process, unique to each organisation’s website.

In the future it should be possible to bring togetherinformation automatically and repeatedly, in ways wecannot imagine and at speeds we can’t envisage. NSIsneed to support the demand for assembly of data fromdifferent sources, including their own, and at the sametime protect the confidentiality of the data. They have,individually and collectively, significant data assets ofvalue to others. Society and governments are exploringways to unlock public sector information and a shared

approach by NSIs has the potential to assist this drive.They should consider how to establish common infor-mation services to attract development investment fromthe commercial, government and non-government are-nas to support this future. If enough momentum isachieved through a common approach, other organisa-tions and individuals will choose to connect their ser-vices and innovations as well. By working as a groupthis will not only make it easier for these organisationsto deal with NSIs, but should result in richer servicesand reduced costs.

To be successful, with a “common information ser-vices” business model:

– NSIs would have common metadata services ontheir websites. The metadata services would pro-vide descriptions of the data elements in eachinformation product including definitions of therows and columns of each table or file, links be-tween data element definitions and data products,links to data quality information, descriptions andconcepts. Visitors to NSI websites would be ableto browse or search the repositories to find infor-mation about which data elements are available,in which combinations, and from which specificdata products. It is important that the informationis also machine searchable and provides metadataservices to other applications both on NSI web-sites and elsewhere, allowing clients to rapidly as-semble data from multiple sources.

– As a companion to a common metadata service,NSIs would implement one or more common dataservices. NSIs are already struggling to produceand quality assure the many different cross tabula-tions that users require and the process of individu-ally crafting these takes resources and time, slow-ing down the ability to respond quickly enough.Many NSIs have recognised that the solution is toinvest in dynamic data services which allow clientsto specify the information they require and gener-ate this dynamically from ‘raw’ data. Examplesof this already exist (for example, the ABS CensusTableBuilder product).

– As well, NSIs would have common confidential-ity services. Because of the need to protect con-fidentiality, NSIs typically hold a great deal moreinformation than they can release. There are com-plex approaches for ensuring that data releasedmeets confidentialisation requirements. For ex-ample, the Census TableBuilder product appliesa perturbation method which ‘works’ for simplecount data but is not directly applicable to other

Page 8: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

128 B. Pink et al. / The case for an international statistical innovation program

types of data. Other types of data might includecontinuous variables, longitudinal data or compos-ite data sets. There is a need for several differ-ent confidentiality methods, depending upon thecharacteristics of the data being accessed.

– Integrating common data, metadata and confiden-tiality services, NSIs also need to design approach-es which support the ability to dynamically linkdata and services from elsewhere. Dynamic datalinking services offered by NSIs and though theirwebsites should allow comparison and combina-tion of data from multiple sources.

– NSIs would partner with industry and governmentto stimulate and enable new ways and means forindividuals, businesses and communities to gainaccess to and to use publicly available data.

The combination of these services would also pro-vide important functionality for future access to moresensitive microdata. ABS’ current Remote AccessData Laboratories (RADL) provides a batch submis-sion process, which allows researchers to submit pro-grams against data that is not available to them in rawform. It has a heavy reliance on assessing and pre-confidentialising the unit data being analysed. Futuredevelopment aspirations are to move this towards aninteractive service, with the focus on dynamically con-fidentialising outputs before it is provided to the client.This is far from a trivial challenge. However, NSI ex-pertise in methods for “confidentialisation” of data isrelevant to all data custodians. Further development ofcommon approaches could be used in many organisa-tions to support information sharing, to increase accessto statistical data and potentially to support new datalinking approaches across multiple organisations’ data.

There will be many very important cases where theinformation required by clients will still be built frompre-assembled data products. Some obvious examplesare time series of data, products such as the nation-al accounts where the information is assembled frommultiple sources within a standard framework,productslike price indices where the assembly process involvescomplex calculations such as chain linking, and verylarge datasets where the processing cost of creating ev-erything dynamically might be too high. NSIs shouldalso share investigations into approaches for managingthese cases and providing them to clients through thesame metadata and data services mentioned above.

4.2. From publications to communication

Publications from specific data collections will re-main important, but there will be a subtle change in their

role. They will become alerts, to warn data users thatnew information has become available. These alertswould announce the key headline stories in the data ina way that other agents, including the media, can easilyconvey to the community. These alerts should also di-rect clients via active links to the richer data availablethrough the data and metadata services.

This separation of the data services from the presen-tation process will support the use of new ways of ex-posing and communicating the meaning stored in sta-tistical data, such as those described by Professor HansRosling, Professor of International Health at Karolin-ska Institute and Director of the Gapminder Founda-tion, which developed the Trendalyzer software sys-tem. There are many other examples of innovationin this area (for example, mashupaustralia.org). Thetechniques available will continue to evolve. Success-ful NSIs in the future will provide their information inways which support and encourage these different waysof communicating the underlying messages in the data.

4.3. Support for transaction data flowing at a muchhigher volume

The volume and sources of real-time or near real-time data are increasing exponentially. Different in-dustries and sources use different metadata and datastandards, but these are often common within areasof interest across the world. For example, to supportbanking transactions for customers everywhere, thereare standards used by many banks. Major manufactur-ers of business equipment such as cash registers andscanners operate globally and use standards such asbarcodes and RFID. This is also true for telecommuni-cations, travel, and manufacturers of other equipmentused within societies (such as traffic lights) and for spa-tial information (including satellite data, Google mapsand GPS data). A collective approach to determiningthe best ways for NSIs to incorporate this data into theinformation stores for their countries would be useful,as would discussion about how to judge which datamight be retained to inform future policy issues. Thiswork could include determining common approachesfor data discovery, data exchange and, perhaps, datawarehousing/archiving and access to data.

4.4. Ability to rapidly incorporate new issues andviews of data into standards and classifications

The use of standards and frameworks will continueto be important to support coherence. However NSIs

Page 9: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

B. Pink et al. / The case for an international statistical innovation program 129

need a way to rapidly incorporate new issues and viewsof the data, without perturbing the base. As well assupporting international standards, NSIs have to oper-ate in different environments with some different areasof key concern. Users of the data also need to adjust oraugment the standard classifications to better suit theiranalyses, situation, environment or to compare the datawith pre-existing data provided by others. Classifica-tions need to be dynamic and supported by automaticways of transforming the data quickly, such as auto-matic coders. Automatic approaches are important be-cause they enable multiple views without significantlyincreasing cost or time. This includes rapidly recodingexisting datasets, coding large amounts of “un-coded”data, “multi-coding” data or recoding data on demand.ABS has undertaken research into some promising newtechniques, using machine learning. Because codingtechniques have broad applicability and the functioncan be quite discrete, ABS would prefer to collaboratein the development of new coders based on these tech-niques. Use of common facilities in multiple organisa-tions (including those who provide data to NSIs) wouldalso support consistent coding and higher data quality.

4.5. ‘Rapid-response’ capability

Often existing collections of data are not sufficient toanswer new questions. If statistical planning processesare operating well, we should have the basic baselinedata to shed light on an underlying social, economic, orenvironmental issue. However, when a policy directionis set, and an “experiment” is underway, governmentswill need up to date information about the outcomesand how these relate to their investment. They willneed to make comparisons with what has happened inthe past, or is happening elsewhere.

Successful NSIs will be required to mount new col-lections, or adapt existing ones, quickly and respon-sively. These new collections may be ‘traditional’ datacollections run by an NSI, or they may involve div-ing into data which is available through other organi-sations’ systems or websites, or both. Culturally, NSIshave a strong bias towards stable data series, and de-veloping collections using careful evaluation of a widerange of user needs, thought into the underlying con-ceptual framework, careful development and testing ofquestions and questionnaires, samples designed for op-timal efficiency, processing systems tailored and test-ed in advance, data edited comprehensively, informa-tion analysed carefully and publications constructed,

presented etc. All these are valuable approaches, butadditional and different approaches will also be crucial.

Technology or methodological changes will help toachieve more rapid response, but changes to the mindset, the capability and the capacity of skilled staff arealso fundamental. NSIs need to be able to form agileteams with a combination of relevant expertise (statis-tical, analytical, technical and policy related) and equipthem with the mandate and tools required to respondquickly. In addition to the capabilities already men-tioned, some of the tools and infrastructure requiredare question modules, web survey facilities, call cen-tres for follow-up or interview, ETL (extract, transform,load) tools, additional confidentiality approaches, datawarehouses to store the results and web site facilities toshare the results. We need agile legal approaches thatcan help to remove barriers to data exchange such astemplate data sharing agreements and licences such asCreative Commons. We need engagement mechanismswhich make it easy to involve people (with academic,commercial, NSI or other backgrounds) in the work ofthese teams and if possible shared approaches to thedevelopment of these people.

4.6. Connecting processes and passing metadata anddata easily between them

Once an initial rapid response has proved its worth,provided that the underlying demand remains, success-ful NSIs will have to improve and institutionalise thenew information stream into their existing core produc-tion processes.

A commitment to a standard way of describing in-formation using a format (such as SDMX and DDI)would allow NSIS to connect statistical process stepstogether more easily over time as key parts of systemsare developed or re-developed. It would also enableothers to provide functionality more easily.

4.7. Analysing assemblies of data

Tools being developed in the Web 2.0 / semantic webspace hold out the promise of new and more efficientways of analysing metadata stores. The use of a stan-dard approach such as SDMX/DDI containing links be-tween particular aspects of information (eg. data collec-tions which use a specific data element, questionnairesand forms used to collect the information, data qual-ity statements about the collection) will support im-proved and better directed investigations by skilled sta-tistical staff into any discrepancies between disparatedata sources. It will also provide insights into how thedata should or should not be used.

Page 10: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

130 B. Pink et al. / The case for an international statistical innovation program

5. Working together

The areas mentioned above are examples of whereNSIs and others might profitably collaborate and in-novate as a community. Each organisation is likely tohave a view of where investment in change is requiredand the development of a program for collaboration oninnovation would require NSIs to explore these viewsand agree on some initial candidate areas.

As individual organisations, current approaches arenot as adaptive as needed. Common strategies havebeen to either attempt large “whole of system” trans-formation programs, or to deal with particular partsof the process at particular points in time. NSIs havetended to engineer processes and systems for currentproblems rather than looking at what is needed for thefuture. This strategy is only appropriate for a relatively“steady state” environment. The level of change re-quired is profound and it is unlikely that any NSI willbe able to make the investments required to facilitatechange at the rate required on its own. The technologyenvironments that are in use today are likely to be total-ly irrelevant in the future, so the approach of redevel-oping large components of infrastructure every 5–20years will not serve organisations well. Instead, NSIsneed an approach that allows them to evolve small-er components of statistical infrastructure much morerapidly, and which allows them to inject and attach newfunctionality as it appears in the environment.

NSIs need to encourage innovation in their organ-isations but also excite and use interest in statisticalinnovation in academia, other information providers,the ICT industry and in creative individuals, so thatthey don’t have to fund all of the change directly them-selves. They need the ability to easily combine theseinnovations along with other tools from the knowledgeeconomy, as they emerge. By acting as a group, itshould be possible to spark interest amongst others inthe research and academic communities and in the ICTvendor community. Researchers, academics and stu-dents probably have the time, energy and interest toinnovate and ICT vendors tend to move into areas thataddress common issues. There are many groups ofdedicated developers/enthusiasts who like to innovatein their area of interest and who are connected via theweb, including group GGobi who like to develop newways of analysing data and numerous groups of opensource developers with specific interests (see source-forge.net). These enthusiasts are a vast untapped work-force. In the commercial arena, Apple has publishedthe details of how to write small applications for the

iPhone/iPod and allows developers to place their appli-cations in an on-line store. Every time a particular ap-plication is downloaded, the developer receives a smallroyalty fee. In the short time since the iPhone was re-leased, many thousands of applications have been writ-ten. Many are trivial, but there are increasing numbersof useful applications becoming available. Use met-rics are collected so that royalties can be paid and as aconsequence, the popular applications are well known.Apple have stated that there are now more than 85,000applications available in the App Store and more than2 billion iPhone and iPod Touch applications have nowbeen downloaded. Although these examples may notwork for NSIs, they should explore these approachesto stimulate contributions from a wider community.

As an international statistics producing community,NSIs need to try to build standard approaches togetherwhile they still have the relevance to force a standard.It is important to make some decisions as a communityabout what needs to be standard, choose some standardseven if they do not represent the best “technical” ap-proach and then evolve these over time if required. Asa community of organisations the “buying power” canbe amplified if agreement can be reached on commonrequirements and design. It may be difficult to reachagreement on the requirements for large components(such as an entire input data warehouse), but if a startis made with components for some of the smaller, rela-tively discrete functions of statistical processing, thesecould be built into larger components over time.

To be effective, the program needs to support changethrough processes, methodology, technology and peo-ple. The people dimension could include opportunitiesfor sharing staff across NSIs (either through short ormedium term secondments or through the use of virtualteams). It is likely that the skills, capability and careersof staff in the organisations would be enhanced throughthe program, particularly if collaboration on the de-velopment and delivery of relevant training is includ-ed. Many NSIs also have active international statisticalaid programs and there is potential to link these to aninnovation program, with benefits for both programs.

Although there are significant institutional and otherbarriers, NSIs should be encouraged by the success of anumber of open source and cross government projects.More effective collaboration has the potential to reducethe costs for each statistical organisation of buildingand maintaining software and could allow stronger in-vestment in particular areas of work. There is alsothe potential to accelerate the availability of statisticalinfrastructure above what could be achieved individu-

Page 11: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

B. Pink et al. / The case for an international statistical innovation program 131

Fig. 1. Generic Statistical Business Process Model (top level) showing some areas of current ABS interest.

ally and at the same time to enhance support for thestatistical systems of developing nations. Combinedwith efforts to enhance statistical skills, a stronger baseof statistical infrastructure shared with other organisa-tions should assist statistical leadership aspirations, re-

duce development and cost duplication, and could ulti-mately drive the emergence of a truly integrated globalstatistical information system.

International organisations, particularly the OECD,the UN Economic Commission for Europe (UNECE)

Page 12: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

132 B. Pink et al. / The case for an international statistical innovation program

Fig. 2. An incremental business transformation approach.

and EUROSTAT, have driven frameworks such as theGeneric Statistical Business Process Model (www.unece.org/stats/gsbpm) and the Common MetadataFramework, and recommended metadata standardssuch as DDI and SDMX through active engagementwith NSIs. These are a very useful starting point pro-viding a top-down view of NSIs’ business and impor-tant guidance regarding the ways to link processes andcomponents. So that components will integrate, there isa strong need for a standard way for the components toshare data and metadata. A number of NSIs and inter-national statistics organisations support the use of DDIand SDMX for this purpose. Part C of the CommonMetadata Framework http://www.unece.org/stats/cmf/provides further information.

What is now needed is a practical way to work togeth-er to address some of the challenges outlined above.

Many NSIs have endorsed the Generic StatisticalBusiness Process Model (Fig. 1) as a reference model.This model provides a useful description of the busi-ness in a way that could assist discussions about the de-velopment of shared approaches. It is also useful with-in NSIs for discussing the areas (“hot spots”) whereinvestment is most required (the highlighted boxes inthe Fig. 1 are examples of current interest to the ABS).

Rather than commencing a huge high risk project todevelop a single solution covering the entire statisticalbusiness process model, an incremental approach isrecommended. An approach is outlined in the Fig. 2.

The approach could commence by establishing asmall set of projects to develop the first group of com-mon components. A set of criteria could be establishedto assist this selection and might include attributes suchas: contributes to ‘future proofing’ of NSIs; required asearly ‘building blocks’ for future collaboration; com-mits to a required standard; delivers tangible benefitsnot only to individual NSIs but to the broader statisticalcommunity; usefulness to more than one participant;ability to complete in defined time period; ability toincorporate easily into existing approaches, speed ofreturn on investment, etc.

Each project could progress through a consistent pro-cess of determining shared requirements, preparing aproject proposal, common design, cost and benefit anal-ysis, team establishment (virtual or co-located), projectplan and milestones for delivery. Participants wouldprobably not contribute directly to all of the projects.

It would be important to select an initial set ofprojects that have a good chance of success. It is en-visaged that, each year, National Statisticians acting as

Page 13: Modernization of Statistical Information Systems Speaker ... Internation… · good policy, assess the impact of the investments and ... istrative data generated by business activity,

B. Pink et al. / The case for an international statistical innovation program 133

‘venture capitalists’, could on the basis of results, as-sess whether to continue and expand the program andcould also determine which new projects to support. Asthe program gained momentum, there would be manyother potential collaborators for this work. These in-clude other government, non-governmentand commer-cial organisations within our nations, the full range ofNSIs, international statistical organisations, academia,global ICT organisations and creative individuals. Allof these are likely to have capabilities, components orplans that might be useful. The development of com-ponents of the statistical process by one NSI whichare then shared with others sometimes works, but inthe past this has tended to be opportunistic rather thanplanned. There will be many solutions that fit into eachbox of the GSBPM and this is good – competition be-tween NSIs, commercial providers and others will onlyserve to accelerate the change and innovation required.

6. Summary

In summary, NSIs need to consider the critical chal-lenges they face and determine the types of organisa-tions they need to become to meet the future success-fully.

The challenges include the increasing range of infor-mation needs, revolution in the amount of informationavailable, declining budgets, full work programs andaging infrastructure.

By exploring the new capabilities and standards thatmight be required, and by working together, there areopportunities for NSIs to share and develop practicalsolutions which improve the reach and agility of officialstatistical services.

References

[1] TrendalyzerSoftwarehttp://en.wikipedia.org/wiki/Trendalyzer.[2] MashupAustrlaia http://mashupaustralia.org/.[3] GGobi Data Visualization System http://www.ggobi.org/.[4] Sourceforge- Open Source Software http://sourceforge.net/.[5] Apple App Store http://www.apple.com/iphone/apps-for-

iphone/.[6] Generic Statistical Business Process Model http://www.unece.

org/stats/gsbpm.[7] Common Metadata Framework http://www.unece.org/stats/

cmf.