Measuring Retail Trade with Administrative Data...2016/06/10 · data” Web scraping/ application...

1 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

U.S. Bureau of Labor Statistics

Measuring Retail Trade with Administrative Data:

David FriedmanAssociate Commissioner for Prices & Living

ConditionsFederal Economic Statistics Advisory Committee

June 10, 2016

2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov2 — U.S. BUREAU OF LABOR STATISTICS • bls.gov

Data Sources

Administrative/Publicly available data

Purchased data sets

Company provided data – “corporate level data”

Web scraping/ application program interface (API)


CPI Data Uses

Create sample frames

Benchmark samples

Supplement collected data to support hedonic modeling (quality adjustment)

Replace/supplement current data collection methods


Summary: Replacing Collection Initiatives

Almost complete

CorpY – company provided dataset

In progress

CorpX – company provided dataset

JD Power – purchased data

Nielsen – purchased data


Corporate Level Data: CorpY

Great Opportunity

maintain respondent cooperation

reduce respondent burden

work with transaction level data

receive insurance prices

Challenges

Average prices for broader category and aggregated

Data received in format difficult to process

Status: 1st production use is May 2016 Index for monthly quotes


Corporate Level Data: CorpX

Receive sales data monthly by 5th of following month

Great Opportunity

maintain respondent cooperation

reduce burden

work with sales data


Corporate Level Data: CorpX

Challenges

mapping the CorpX item categories to the CPI structure

melding the sales level data into our methodology and current systemo in particular, accommodate seasonality & item substitution

including new methodology

o achieve constant-quality price change w/a big data set

lack of characteristic detail

having enough history to validate method


CorpX Current Status

I. Received data for all CPI Primary Sampling Units (PSU’s) beginning with October 2014

II. Testing various methodologies

III. Will develop necessary CPI system changes to be ready to use


JD Power Project

Purchase JD Power dataset as source for replacement in New Vehicles index

Prime example of benefits and challenges of “big data”

Breadth of information

Challenge of integration with current systems

Methodological issues


New Vehicle Observations

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

CPI JDPower

Num

ber

of

Obse

rvations


Model Year Price Indexes


Unit Prices Increase

40

50

60

70

80

90

100

110

120

130Ja

n-0

7

Apr-

07

Jul-07

Oct

-07

Jan-0

8

Apr-

08

Jul-08

Oct

-08

Jan-0

9

Apr-

09

Jul-09

Oct

-09

Jan-1

0

Apr-

10

Jul-10

Oct

-10

Jan-1

1

Apr-

11

Jul-11

Oct

-11

Jan-1

2

Apr-

12

Jul-12

Oct

-12

Jan-1

3

Apr-

13

Jul-13

Oct

-13

Jan-1

4

Apr-

14

Jul-14

Oct

-14

Jan-1

5

Index (

100=

1/2

007)

UnitPriceInx MatchedModelTorn


Ways to Treat the Price Declines

Show the drop

Show price change across model years

Create “Changeover” price relatives

Use Year-Over-Year Index


Price DynamicsAverage Prices (Source: Aizcorbe, Bridgman and Nalewaik(2010))

16 — U.S. BUREAU OF LABOR STATISTICS • bls.gov 16

JDPower vs CPI

90

92

94

96

98

100

102

104

106

108

110

20

07

_1

20

07

_3

20

07

_5

20

07

_7

20

07

_9

20

07

_11

20

08

_1

20

08

_3

20

08

_5

20

08

_7

20

08

_9

20

08

_11

20

09

_1

20

09

_3

20

09

_5

20

09

_7

20

09

_9

20

09

_11

20

10

_1

20

10

_3

20

10

_5

20

10

_7

20

10

_9

20

10

_11

20

11

_1

20

11

_3

20

11

_5

20

11

_7

20

11

_9

20

11

_11

20

12

_1

20

12

_3

20

12

_5

20

12

_7

20

12

_9

20

12

_11

20

13

_1

20

13

_3

20

13

_5

20

13

_7

20

13

_9

20

13

_11

20

14

_1

20

14

_3

20

14

_5

20

14

_7

20

14

_9

20

14

_11

20

15

_1

20

15

_3

Ind

ex (

10

0=0

6/2

00

9)

Proposed JDPower Index

CPI: New Vehicles JDP: YOY + Cycle


Research Nielsen Indexes

Data set for August 2005 – September 2010

2 million UPC codes

Scantrack coverage limitations

Grocery>$2 million; Drug Stores>$1 million; Mass Merchandisers

Excludes one major retailer and non-UPC items (some produce, deli, bakery, fresh meat, etc.)


Nielsen Indexes

18

0

20

40

60

80

100

120

140

FJ011 - Milk

CPI Nielsen


Nielsen Indexes

19

0

20

40

60

80

100

120

140

FR02 - Candy and chewing gum

CPI Nielsen


Nielsen Indexes – Current focus

Refine Nielsen indexes to :

Limit research to items that are well represented in the Scantrack data

Account for product downsizing

Account for UPC “churn”

Calculate a geomeans index (in addition to a Tornqvist index)


Nielsen indexes – Current focus

Preliminary results for 4 item strata

Work on additional 10-12 strata in FY16

95

100

105

110

115

200608

200609

200610

200611

200612

200701

200702

200703

200704

200705

200706

200707

200708

200709

200710

200711

200712

200801

200802

200803

200804

200805

200806

200807

200808

200809

200810

200811

200812

200901

200902

200903

200904

200905

200906

200907

200908

200909

200910

200911

200912

201001

201002

201003

201004

201005

201006

201007

201008

201009

CPI and Nielsen Indexes for FA02 – 0000 Cereal and Cereal Products

CPI TQ (price in t & (t-1); Churn & Dwnsz) Geo (No missing prices, Churn & Dwnsz)


Nielsen Indexes – Current focus

90.00

95.00

100.00

105.00

110.00

115.00

120.00

125.00

130.00

200608

200609

200610

200611

200612

200701

200702

200703

200704

200705

200706

200707

200708

200709

200710

200711

200712

200801

200802

200803

200804

200805

200806

200807

200808

200809

200810

200811

200812

200901

200902

200903

200904

200905

200906

200907

200908

200909

200910

200911

200912

201001

201002

201003

201004

201005

201006

201007

201008

201009

CPI and Nielsen Indexes for FA01 – 0000 Flour & Prepared Flour Mixes

CPI TQ (price in t and (t-1); churn & dwnsz Geo (no missing prices; churn & dwnsz)


Nielsen downsizing

Automate identification

Compare to CPI

$0

$1

$2

$3

$4

Mill

ions

Betty Crocker Fudge Brownie Mix1600019726 - 18.3OZ

1600044830 - 19.8OZ


Summary: Benefits vs. Challenges

Benefits:

Increasingly more available

Allows for evaluation & improvement

May reduce collection costs

Reduces respondent burden

Increased sample size

May increase data quality

Sometimes ability to get quantity data

Challenges:

Data quality issues –especially lack of descriptive info

Timeliness and reliability concerns – mitigation strategies

Cost and other considerations (new skill set, IT infrastructure, etc.)


What’s Next

Continue work on CorpX, JD Power, Nielsen

Project to modify CPI production to more readily accept future alternative data

Work with CE to investigate secondary sources for Rent Data

Explore new opportunities

Contact Information


David FriedmanAssociate Commissioner for Prices & Living

Conditionswww.bls.gov/bls/inflation.htm

[email protected]

Measuring Retail Trade with Administrative Data...2016/06/10 · data” Web scraping/ application...

Documents

Transcript of Measuring Retail Trade with Administrative Data...2016/06/10 · data” Web scraping/ application...