2007 CAS PREDICTIVE MODELING SEMINAR

PROJECT MANAGEMENT FOR PREDICTIVE MODELS

BETH FITZGERALD, ISO

2

Accomplishing Business Goals

• Project Management

• Implementation

• Future

3

Project Management

• Determine business processes that support strategic goals

– Underwriting decisions– Pricing decisions

• Develop project plan aligned with strategic goals

– Model Building– Technology Development– Implementation Phases

• Determine project needs

• Monitor actual vs. planned costs/milestones

4

Project Needs

• Team Skills

– Data management – Analytical/statistical– Technology– Business Knowledge

• Data

• Statistical Tools

• Computer Capacity

5

Prior to Modeling

•Formulate the Problem

•Evaluate Possible Data Sources

•Prepare the Data

•Explore the Data with Simple Modeling Techniques

6

25% 50% 75% 85%

What percent of a model building project is the data preparation and data

management?

7

Prepare the Data

• Do quality checks in level of detail needed for project

• Understand how to prepare individual variables for use in models

• Need to be practical about number of classification categories models can handle

• Need to decide on truncation and bucketing of variables that are continuous

• Create new variables

8

Data Management Issues

• Matching additional internal policy information to premium/loss data

– Different points in time– Tracking & balancing audited exposures

• Different summarization keys – handling of mid-term endorsements

• Address scrubbing

• Matching to external data for correct point in time

• Significance of missing values within variable

9

Modeling Procedures and Diagnostics

•Basic modeling training – GLM, Data Mining

•Decide on appropriate diagnostics

•Evaluate diagnostics

10

Modeling Process

BusinessKnowledge

Data Linking

Data Cleansing

Analyze Variables

Determine Predictive Variables

Evaluation

Data Gathering

Modeling

11

Business Questions

•What goals are you trying to achieve?

•What results do you expect to see?

•How will you know if the results are reasonable?

•How do you ensure sufficient knowledge transfer to business staff?

12

Model Performance

Relative Loss Ratio LiftOptimal Model

0.7

0.8

0.9

1

1.1

1.2

1.3

1 2 3 4 5 6 7 8 9 10

Decile of Worst to Best Risk

Loss

Rat

io R

elat

ivity

LR Relativity by Decile

13

Model Input/Output

• Model input considerations

– Access to data

– Robustness/quality of data

– Timeliness of refreshed data

• Design Model output for users

– Definition of output – expected loss ratio, pure

premium, loss ratio relativity?

– Provide support for output – reason codes

14

Business Implementation of Model

• Model usage determined by strategic goals

– Underwriting risk decision

– Pricing of risks

– Support of market growth

• Integration of Model into business workflow

decisions

– Consistency in underwriting/pricing decisions

– Compliance with regulations based on

implementation decisions

15

Implementation of Model

Workflows:

• Underwriting– New Business– Renewal business

• Rating– Pricing– Coverage Adjustment

16

Implementation of Model

• New Business decision options

– Write risk– Request additional info on risk– Decline risk– Adjust price/coverage

• Consider model output alone or along with other information available from application

• Model output needed within seconds for quick decision

17

Implementation of Model

• Renewal decision options

– Automatic renewal– Flag for non-renewal– Adjust coverage level for risk– Adjust pricing for risk

• Initial Year

– review all in-force policies on weekly or monthly basis

• Subsequent years

– establish schedule for reevaluation based on specific underwriting guidelines

18

Implementation of Model

Rating

•Model O/P represents relative loss ratio factor

•Determine rating selections

•Determine rating process– Modify application of IRPM plan– Implement new rating factors based on

Model– Tier risks into different insurers within

insurer group

19

Technology Development

• Incorporate business implementation decisions

• Decide on how Model will be accessible

electronically

– Web-based interface

– Integrated into existing workflow

– Batch processing

• Develop/Modify Systems

– Phase-in technology

• Model uses information from a third-party vendor

• Determine I/P and O/P criteria

20

Rollout Implementation of Model

•Prepare Announcement/Training

Material for Internal & External

Customers

•Coordinate Implementation Phases

•Monitor Feedback/Adjust

Implementation

•Monitor Results against Strategic

Goals

21

Future of Predictive Modeling

• More refined rating plans– Industry-sourced or internally developed

– Combination of internally-developed & industry-sourced risk component variables

• Ongoing updating and maintenance of Models – Refresh data

– New data sources/variables

– New tools/techniques

– React to new market environments