Sdmx at us federal reserve

The FRB and SDMX:National data and

International standards

San CannonFederal Reserve Board

SDMX Conference 9-11 January 2007

SDMX Conference9-11 January 2007

2

Background

• The Fed is a statistical agency as well as a central bank and regulatory agency.

• Lots of data and information are available on the public website.

• Statistical data are varied: monthly industrial production indexes, daily interest and exchange rates, quarterly financial flows for various sectors of the economy, surveys of small businesses and consumers, etc.


3

Serving our users betterTo some, it may appear that the statistical

agency role is secondary.• Data are not always easy to find.• Downloads are not customizable.• Example: Trying to extract one industrial

production series requires retrieving two text files, cutting and pasting, reformatting….

• Complete – yes. User Friendly – no.


4

Data Download Program (DDP)• XML designated as key format but project

team wrestled with implementation details.

• Staff weighed a homegrown DTD setup against the new SDMX standard.

• SDMX looked to have greater benefits and was adopted.

• Good decision: additional internal applications as well as interagency projects using SDMX are in the works.


5

A lot to learn• SDMX is based on data structure

definitions (‘key families’) and codelists, with every concept represented by a code with a corresponding definition.

• We were unfamiliar with this type of data modeling, so it proved challenging.

• Two of our pilot datasets translated easily to this new format; others needed more work.


6

Data structures differ

Some data structures are readily adapted to the concept/codelist representation. Series “keys” have no real mnemonic value.

HBBA Int. Rate, Official, Discount rate/Base rate

HBCA Int. Rate, Official, Intra-day loans

SCBA Indust. Production, Motor vehicles, NSA

SCBB Indust. Production, Motor vehicles, SA


7

Hierarchical relationshipsWe allow data to be modeled hierarchically,

and use mnemonics that reflect this.

RIFSPFF_N.B

R.*:Rate

R.I.*:Rate of interest in money and capital markets

R.I.F.*:Federal Reserve System

R.I.F.S.*:Short-term or money market

R.I.F.S.P.*:Private securities

R.I.F.S.P.FF.:Federal funds

_N.:Not seasonally adjusted

.B:Business (Five days, Monday-Friday)

JQI_I02Y3361T3_N.M:

J.*:Indices except of prices

J.Q.*:Production

J.Q.I.:Industrial

_I.*:NAICS-based industry classification

02Y:codes from year 2002

3361.:Motor Vehicle Manufacturing

T:thru

3363:Motor Vehicle Parts Manufacturing

_N.:Not seasonally adjusted

.M:Monthly


8

Applying the SDMX model• Data represented by a concrete number of concepts

are much easier to represent with key family dimensions and attributes:JQI_I02YMF_N.M → Topic_Industry_SA.FreqFA156900005.Q → Prefix (2 digits), Sector

(2), instrument type (5), series type (1), frequency

• Hierarchical relationships and varying number of concepts makes life more difficult:RIFSPPNA2P2D30_N.B → Topic?_SA.FreqRIFLGFCY20_XII_N.B → Topic?_Inflate_SA.Freq


9

Decisions we made

• Allow a variable number of data structure definitions per dataset.

• Use the compact format for internal exchange and external downloads.

• Stick with SDMX 1.0, for now.• Use a relational database to store

data and XML information for retrieval.


10

Final product: DDP!• We have a flexible application whose

interface is entirely driven by the data structure definitions.

• “We store the XML as carefully sliced text in a relational database and we can build an index structure that allows us to respond to ad-hoc queries very efficiently, even for large volumes of data.”

Data Download Program

http://www.federalreserve.gov/datadownload


11

Strengths and weaknesses• Because interface is entirely data driven, it

is easy to add new data. • Internal architecture is complex, due to

security and the data workflow:– SDMX files are generated by data staff and

transmitted to public website staff for processing.

– These files are made available on the website and “shredded” for database entry and lookup.

• Current structure is not set up for codelist sharing.


12

What do the users say?• “Really excellent. One of the

best I’ve used on the web.”

• “This Data Download thing is better than sliced bread.”

• “I downloaded the XML files, but I cannot run them. All I see is the xml code.”


13

What do the numbers say?• More than 250,000 unique visits since

April – about 50,000 per month.

• Data Download Program is the 6th most visited area on the Federal Reserve website.


14

Next steps• Add more data: reserves, exchange rates,

consumer credit.

• Continue working with other central banks and statistical agencies on common framework.

• Prepare to move to SDMX 2.0 to take advantage of additional features.


15

The last slide…

Questions? Comments?

Thank you for your attention!

San [email protected](202) 452-3710

mailto:[email protected]

Sdmx at us federal reserve

Technology

Transcript of Sdmx at us federal reserve