Overview of Simulation Models and a Simulation Model for NHIS Field Operations and Cost EstimatesBor-Chung Chen

Office of Railroad SafetyFederal Railroad Administration/USDOTApril 7, 2011

Staff Allocation ModelsRisk and SafetySafety DataRisk/Reliability DataSafety Data-Driven

Workload and ActivitiesInspector Activity Report DataDemand-DrivenRequired by Regulations/Law*

Example Model (1)NIP (National Inspection Plan) Staff Allocation ModelMinimize damages of railroad accidentsRegression analysis is used based on injuries/fatalities dataConstraints on specified percentage changes of current inspector position slots*

Example Model (2)Transportation Security Administrations Staff Allocation ModelThree Components:GRA Flight Data by GRA, Inc.: baggage volume, flight & passenger distribution, and load factorRegal Software by Regal Decision Systems: airport configuration and screening processSabre Software by Sabre Airline Solutions, Inc.: scheduling process to determine staff needed in waiting time of 10 minutes (or 5) or less.*

Example Model (3)Positive Train Control (PTC) Staff Allocation ModelFour PTC Components:Dispatch CenterCommunication SystemsLocomotive UnitsWayside UnitsAppropriateness/Effectiveness Index ScoreMaximize the total score of all inspector assignments to all inspector activities. (To allocate the skilled inspectors to the inspection activities in reaching the most effective performance)Constraints*

Simulation ModelingAn Operations Research MethodOptimizationSave Resources and/orImprove Data Quality*

Operations Research (OR) seeks the determination of the best (optimum) course of action of a decision problem under the restriction of limited resources*

An optimization model is a decision-making tool that recommends an answer (the goal to be optimized) based on analyses of information (constraints and decision variables). It consists of three components:The goal to be optimized,Constraints, andDecision variables*

Operations Research ModelsDeterministic ModelsLinear Programming ModelsInteger Programming ModelsNetwork Flow Programming ModelsNonlinear Programming ModelsStochastic ModelsInventory ModelsQueueing ModelsQueueing Networks and Decision ModelsSimulation Models*

Types of OR ModelsAnalytical Models:The objective and constraints of the models can be expressed quantitatively or mathematically as functions of the decision variables.

Simulation Models:The relationship between input and output of the models are not explicitly stated; the models break down the modeled system into basic or elemental modules that are then linked to one another by well-defined logical relationships.*

An M/M/c Queueing System*

Performance Measures of Queueing SystemsArrival rateDeparture rateServer utilizationExpected number of customers in queueExpected number of customers in systemExpected waiting time in queueExpected waiting time in system*

Total Cost of A Queueing System (Taha[2011])Cost ofWaitingTotalCostCost ofOperationOptimum Numberof Servers (Tellers)Number of Servers (Tellers)Cost Per Unit Time *

Queueing Systems vs. Field OperationsQueueing SystemsThe customers come to the serversThe system is small and simpleNo traveling time involvedField Operations (Personal Visits)The servers (interviewers) go to the customers (respondents)The system is very large and complicatedServer traveling time*

Inbound vs. Outbound Telephone Call CentersInbound800 Customer ServicesHelp DesksOutboundTelemarketingTelephone SurveysCharitiesPoliticiansSome Companies*

Outbound Telephone Dialing Systemas a Closed Queueing Network (Samuelson[1999])ServiceFacilityQueue orWaiting Linex x x x x xANADRNParty Does Not AnswerParty AnswersLines with PartiesWho Hang Up orGet Turned AwayWaitingTo Dial12c...SW*

Outbound Telephone Dialing SystemDecision VariablesAmount of time to anticipate service completionsObtaining the new party too early, resulting in an abandoned call and the need to start dialing againCost of waiting too long, resulting in unnecessary idle time for the representatives

Number of calls to attempt at onceTwo or more answer, we will have one or more abandoned callsNone answers, we will have idle representative time*

ObjectivesDevelop a valid method of predicting cost, response rates, and timing of new or continuing surveys for the field operations.

The simulation modeling will be followed by the optimization of the field operations if a simulation model is feasible and valid.*

Definition of Discrete-Event SimulationEvent Driven: Each occurrence of an event changes the state of the system

Using a model (implemented as a computer program), rather than experimenting with a real system

*

Steps of Simulation Study (Banks 1998)Model ConceptualizationData CollectionInput Data AnalysisModel TranslationVerification and ValidationExperimental DesignProduction Runs and Output Analysis

*

Model ConceptualizationProblem Formulation

Objectives and Project Plan

The modeling begins simply and the model grows until a model of appropriate complexity has been developed with the objectives in mind.*

Data CollectionA data set for each variable from a survey is collected.

Whenever possible, collect between 100 and 200 observations.

Collect a number of samples from different time periods, such as field operations (time of day and/or day of week)*

Input Data Analysis and ModelingAssessing IndependenceProbability PlotsEstimation of ParametersGoodness of Fit TestsEmpirical DistributionsSimulation Support SoftwareExpertFit (A. M. Law and Associates)Stat::Fit (Geer Mountain Software Corporation)

*

Model TranslationThe conceptual model constructed is coded into a computer-recognizable form, an operational model.General-Purpose SoftwareManufacturing-Oriented SoftwareBusiness Process ReengineeringSimulation-Based SchedulingField Operations? C++, FORTRAN?*

Random Number and Random Variate GenerationRandom (pseudorandom) numbers between 0 and 1 from the uniform distribution, U(0,1) or RN(0,1)Use Inverse Transform Method to obtain a random variable, X:otherwise*

Verification and ValidationVerification concerns if the operational model is performing properly.

Validation is the determination that the conceptual model is an accurate representation of the field operations (or the real system).*

Verification and Validation ProcessIt is an iterative process:Add new details to the modelRun the modelEvaluate the resultsThe results are not sufficiently accurateIdentify other details (operations/input data)Go to step 1 and the cycle starts anewAt some point, the model is determined to be close enough *

Experimental DesignFor each scenario that is to be simulated, decisions need to be made concerning the length of the simulation run, the number of runs (also called replications), the manner of initialization, and controllable decision variables as required.*

Production Runs and Output AnalysisProduction runs and their subsequent output analysis are used to estimate the performance measures (cost, timing, and response rates) for the scenario that are being simulated.Finite-Horizon SimulationsSteady-State Simulations*

Simulation Model of Simplified NHIS Field Operations (Prototype)Ten FRs, 1050 cases, 105 cases per FREach FR covers a PSU of 60 x 60 square milesFRs are given 17 days starting from a MondayAll FRs start to work at 3:00 PM each day2004 NHIS CHI data set for input modelingVisiting order: Traveling Salesman ProblemThe model: about 1900 lines of C++ code*

Field Operation InputsFrequency distribution of 28 outcomes

Interview length distributions by outcomes

Contact/No-Contact Bernoulli distribution

Contact time distributions

Uniform distributions for vehicle speed*

Software Development for Field Operations Simulation ModelingInput ModelingField Operations Simulation ModelOutput AnalysisCostsResponse RatesTimingField Operations Inputs*

Performance MeasuresLow Cost: Direct Labor Cost (Hours and Mileage)Average number of personal visits per case

High Response Rate

Short Timing: How long it takes each month (17 days)

It is called LHS*

Preliminary Results1000 independent replications with different seeds

Cost: $25,475Based on $10/hr and $0.35/mileAverage number of PV = 1.74

Response Rate: 86.04%

Timing: 17 days*

Response Rates of 2004 NHIS Q2*

RegionRR(%)RegionRR(%)Boston86.09Charlotte90.62New York76.40Atlanta91.72Philadelphia82.86Dallas87.23Detroit93.46Denver92.04Chicago91.21Los Angles87.32Kansas City93.48Seattle86.99National88.63

Design of Experiments:Controllable ParametersStarting time: 10:00 AM, 12:00 noon, and 3:00 PM

Number of FRs: 10 and 15Timing: 17 days vs. 11 daysArea: 3600 vs. 2401 square milesCases per FR: 105 vs. 70FR-Days: 170 vs. 165

*

Selected Frequency Distributions of Contact (C)/No-Contact (NC)*

%HoursSunMonTueWedThurFriSatAllCNC10:0012:0049.0250.9851.5448.4649.7550.2550.6749.3353.6246.3851.0948.9155.1744.8351.8848.12CNC12:0015:0052.9747.0350.6349.3751.1048.9051.2248.7850.3149.6951.9648.0454.2545.7551.6448.36CNC15:0020:0051.9548.0555.5044.5056.0543.9556.9443.0656.2643.7453.8646.1451.7748.2355.3244.68

The Six Parameter Settingsfor the Experiments*

S. T.FRsDaysAreaFR-DaysAdj. Days110:001017360017017.00212:001017360017017.00315:001017360017017.00410:001511240116511.33512:001511240116511.33615:001511240116511.33

The Estimates of the PMs of the Six Parameter SettingsAdjusted to 170 FR-Days*

Cost($)RR(%)AVsCost($)RR(%)AVsSaved(%)125,37586.191.7225,37586.191.72225,23886.861.7125,23886.861.71325,47586.041.7425,47586.041.74420,72282.231.6821,34984.721.7315.86520,57583.501.6621,19986.031.7116.00620,58983.881.6721,21386.421.7216.73

Federal Statistics in the FY 2010 BudgetSource: http://www.copafs.org/reports/federal_statistics_in_the_fy_2010_budget.aspx

*

(Total direct funding in millions)FY2008 ActualFY2009 EstimateFY2010 RequestCensus Bureau Current Programs$ 232.8$ 263.6$ 289.0Census Bureau Periodic Programs1,234.03,906.37,115.7Others1,217.71,330.51,431.0Total2,684.55,500.48,835.7

Cost Estimates of the Replication with Seed 169001*

SettingTotalTime (hours)Wages($)TotalDistance(miles)Mileage($)TotalCost($)31,349.3713,49435,01212,25425,74861,107.5211,07527,1019,48620,5616(adj)1,141.0811,41127,9229,77321,184

The Other Three Parameter Settingsfor the Experiments*

S. T.FRsDaysAreaFR-DaysAdj. Days110:001017360017017.00212:001017360017017.00315:001017360017017.00710:001517240125511.33812:001517240125511.33915:001517240125511.33

The Estimates of the PMs of the Other Three Parameter Settings*

Cost($)RR(%)AVsCost Saved(%)RR Gain(%)125,37586.191.72225,23886.861.71325,47586.041.74724,54589.931.783.273.74824,08589.961.754.573.10923,92689.981.756.083.94

Optimum number of FRs*

ConclusionsSimulation models can be used for optimizing field operations

Smaller PSU area is more cost effectiveLess time on the roads and more time knocking on the doorsNot at the expense of the response rateField operations can be completed sooner*

Microsimulation of NHISPhysical Impediments and At-Home Patterns of HouseholdsInterviewer StrategiesMultiple Visits of Completed InterviewsUnrelated Persons Living in the Same HouseClassification of InterviewersMultiple SurveysSample Designs*

What Next?Most Recent NHIS CHI Data

Classification of PSUs:Population DensitiesCar DensitiesTraffic Statistics

Development of A Simulation Language for Field Operations?

*

Simulation and Modeling TextbooksLaw and Kelton: Simulation Modeling and Analysis. 3rd edition, 2000, McGraw-HillJerry Banks, Editor: Handbook of Simulation. 1998, Wiley & SonsHamdy A. Taha: Operations Research: An Introduction. 9th edition, 2011, Prentice HallHillier and Lieberman: Introduction to Operations Research, 8th edition, 2005, McGraw-Hill*

Operations Research ModelsDeterministic ModelsLinear Programming ModelsInteger Programming ModelsNetwork Flow Programming ModelsNonlinear Programming ModelsStochastic ModelsInventory ModelsQueueing ModelsQueueing Networks and Decision ModelsSimulation ModelsField Operating Models?*

****

***************Each occurrence of an event changes the state of the system; for example, end of an interview will change the state of the system from n completed interviews to n+1 completes interviews.*These are the major steps in a typical simulation study. (Read from the slide) I will briefly go thru each one.**************************