STATISTICAL EVALUATION OF TRANSIT ONBOARD SURVEY RESULTS AND METHODS
description
Transcript of STATISTICAL EVALUATION OF TRANSIT ONBOARD SURVEY RESULTS AND METHODS
STATISTICAL EVALUATION OF STATISTICAL EVALUATION OF TRANSIT ONBOARD SURVEY TRANSIT ONBOARD SURVEY
RESULTS AND METHODSRESULTS AND METHODS
Kathleen Yu, Arash Mirzaei, Behruz Paschai, Hua Yang
North Central Texas Council of Governments (NCTCOG)
FTA Workshop on Travel Forecasting for New Starts 2March 2009
Analyzing the 2007 DART On-Board Survey
Dallas Area Rapid Transit (DART) Transit System Information
On-Board Survey, Spring 2007
Transit ModeWeekday
BoardingsNo. of Routes
Commuter Rail 9,000 1
Light Rail 62,000 2
Express Bus 6,700 10
Local Bus 141,000 98
Total DART System 218,700 111
May 2009 3TRB National Transportation Planning Applications Conference
DART System Map
May 2009 4TRB National Transportation Planning Applications Conference
DART Onboard Survey Schedule
May 2009 5TRB National Transportation Planning Applications Conference
December 2006 RFP posted.
February 2007 Proposals reviewed.
March 2007 Contract signed.
Pre-tests almost eliminated due to time.
May 2007 Data collection ended.
November 2007 Final product delivered
March 2008 NCTCOG clean-up completed.
Weekday Sampling Plan Sample size based on daily route boardings.
Desired: 95% confidence interval with +/- 5% error Random sampling of vehicles by route direction in four
time periods (AM, mid-day, PM, and evening)
For each sampled vehicle: Total adult (15+) boarding counts by stop All adults boarding bus asked to participate
May 2009 6TRB National Transportation Planning Applications Conference
Expansion Plan ExampleSample Vehicle Trip in Route = 45, Time Period = a.m., Direction = Northbound, Day = Weekday
BusStop 1
BusStop 2
BusStop 3
BusStop 4
# of Boardings 15 8 27 10
# of Completes 6 0 9 2
Stop Response Fact. 2.5 0 3.0 5.0
Calculated Boards 15 0 27 10
Route Trip Factor = 60/52 = 1.151.15 1.15 1.15 1.15
+ + + = 60
+ + + = 52
Trip Factor
Vehicle Trip Expansion Factor = Total Vehicle Trips per RTDD / Sampled Vehicle Trips per RTDD = 15/2 = 7.5
7.5 7.5 7.5 7.5Vehicle Trip Exp.
21.6 0 25.9 43.1RTDD Factor 90.6+ + + =
Total Calculated Boardings in all RTDs = 582.4 Observed Avg. Daily Boards = 971 971/582.4=1.7
1.7 1.7 1.7 1.7Route Factor
36.7 0 44.0 73.3Final Exp. Factor8
Quality Control Checks Evaluation of geo-coding
Origin and destination Boarding location (recorded) Alighting location (imputed)
Correction of conflicting answers Identification of inconsistent answers Routes in the paths Boarding and alighting locations
Assessment of the quality of final database
May 2009 9TRB National Transportation Planning Applications Conference
Evaluation of Geo-coding A reasonable qualitative question is: how
accurately were origin and destination locations coded in the final database?
Practically, we accepted some level of inaccuracy in locating the true locations and determined how often the coding has been done within that distance.
Instead of manually checking all 6,447 records, we randomly selected records to assert an accuracy statement based on statistical sampling.
May 2009 10TRB National Transportation Planning Applications Conference
Evaluation of Geo-coding – Results Origin point is within 0.75 miles of the user-
specified origin place and address. 71 of 74 random records checked met criteria. 95% confident that 90 to 100% of all points in database
meet criteria.
Destination point is within 0.75 miles of the user-specified destination place and address. 72 of 74 random records checked met criteria. 95% confident that 90 to 100% of all points in database
meet criteria.
May 2009 11TRB National Transportation Planning Applications Conference
Questions1. Home address2. Trip origin address and type3. Mode of access4. Transfer from5. Total number of transit vehicles
in the trip6. Transfer to7. Mode of egress8. Trip destination address and
type9. First and last rail lines and
stations boarded and alighted 10. Route sequence11. Main reason for taking the
route12. Perceived trip length in minutes
13. Frequency of use14. Weekend and/or weekday users15. Substitute mode of travel16. Type of payment for the transit17. Type of fare18. Number of registered vehicles19. Household size20. Number of adults (15+) in the
household21. Male or female22. Age group23. Work status24. Ethnicity25. Household income group
May 2009 12TRB National Transportation Planning Applications Conference
Identification of Inconsistent Answers 4 to 6 redundant questions about the path possibly
confused people
2,593 of 6,447 records found to have inconsistent path information.
May 2009 13TRB National Transportation Planning Applications Conference
Path Response Errors People described all possible routes they could
take for their trip and not the ones they are specifically taking on this trip.
People described a round trip and not a one-way trip, so a route/rail was repeated in the sequence.
People put down origin and destination for round trip, but described path for one-way trip – or vice versa.
People described their reverse trip rather than what they were taking. Additional 575 surveys were appended to survey review
list because the origin seemed closer to last route than first route in sequence.
May 2009 14TRB National Transportation Planning Applications Conference
Correction of Paths1. Create a map with the origin, destination and all routes
reported in the answers to the redundant questions.
2. Assume origin, destination, and surveyed “route” are correct.
3. Review routes in path sequence and the routes in the transfer questions to see if they contain a reasonable path.
4. If needed, use any routes listed on the survey to create a reasonable path for the OD using the surveyed “route.”
5. For walk mode of access and egress, confirm distance is less than or equal to 2.5 miles.
Of 3,168* surveys reviewed, 3,004 surveys were checked, modified (if necessary), and returned to the database, and 164 flagged for uncorrectable paths.
* Original list of 2,593 records plus the 575 surveys with possible reverse path sequence.
May 2009 15TRB National Transportation Planning Applications Conference
Examination of Path Questions
Path Transfer Questions Sequence Question
No Transfer 86% 98%
1 Transfer 73% 73%
2 Transfers
26%
59%
3 Transfers 24%
Overall 60% 72%
Percentage of Correct Answers to Path Questions
May 2009 16TRB National Transportation Planning Applications Conference
Confidence in Path Sequence Random selection of 74 records from the database
of 6,283 records. 71 determined to have a correct path. Assertion: 95% confident that 95% of the respondent-
identified sequences from origin to destination is feasible.
May 2009 17TRB National Transportation Planning Applications Conference
Potential Survey Improvements Rigorous sample expansion plan.
Detect and correct for non-response biases Ancillary count program
Pilot tests conducted in a timely manner. Test graphic design Clarify meaning of one-way trip
Reduction in number of questions. Elimination of redundant questions. Alighting location asked or captured. Transit wait time asked or captured. Clarification of expected quality in surveyor contract.
May 2009 18TRB National Transportation Planning Applications Conference
FTA Workshop on Travel Forecasting for New Starts 19March 2009
Planning for 2008 On-Board Surveys in Fort Worth and Denton
FWTA and DCTA Transit System Information
Transit ModeWeekday
BoardingsNo. of Routes
Express Bus 1,126 9
Local Bus 32,882 53
Total FWTA and DCTA 34,118 62
On-Board Survey, Fall 2008
FWTA = Fort Worth Transportation Authority
DCTA = Denton County Transportation Authority
May 2009 20TRB National Transportation Planning Applications Conference
FWTA and DCTA System Map
May 2009 21TRB National Transportation Planning Applications Conference
FWTA and DCTAOnboard Survey Schedule
May 2009 22TRB National Transportation Planning Applications Conference
June 2008 RFP posted.
August 2008 Proposals reviewed.
Contract signed.
October 2008 Pilot test/cognitive interviews conducted.
November 2008 Data collection ended.
January 2009 Draft geo-coded database delivered.
April 2009 Expanded database delivered.
June 2009 Final report scheduled.
Improvements Implemented
Clarification of expected quality in the survey contract Reduced number of questions
Unused questions in DART 2007 survey eliminated Redundant path questions in DART survey removed
Inclusion of new questions Alighting location asked or captured Transit wait time asked or captured
Clarification of meaning of one-way trip Use of graphic diagrams Advertisement campaigns Consistent use of term “one-way trip”
May 2009 23TRB National Transportation Planning Applications Conference
Improvements Implemented
Enhanced questionnaire design Modified text design Graphic design
Pilot tests of different instruments In-field Cognitive interview
Sample expansion plan Sampling expansion correction by station counts Non-response follow-up survey
May 2009 24TRB National Transportation Planning Applications Conference
Clarification of Expected Quality in Surveyor Contract Identify the sample frame, confidence interval, and error
Daily boardings per route, 95%, and +/- 5%.
Identify QC specification.The consultants should show they are 95% confident that 90 to 100% of location data and all other collected information in the survey instrument are coded correctly. This quality will be tested by random sampling of the final coded records. A record is considered incomplete if ….
Identify QC implementation plan.At the end of the data collection, the consultants should show that they are 95% confident that 90 to 100% of the collected surveys are useable.
A filled questionnaire is considered useable if the respondent properly answered all of the following questions: origin, destination, purposes, path questions that identify the routes that the users take, mode of access and egress, time of survey, and home address.
May 2009 25TRB National Transportation Planning Applications Conference
Unused Questions in DART 2007 Survey1. Home address2. Trip origin address and type3. Mode of access4. Transfer from5. Total number of transit
vehicles in the trip6. Transfer To7. Mode of egress8. Trip destination address and
type9. First and last rail lines and
stations boarded and alighted 10. Route sequence11. Main reason for taking the
route12. Perceived trip length in
minutes
13. Frequency of use14. Weekend and/or weekday users15. Substitute mode of travel16. Type of payment for the transit17. Type of fare18. Number of registered cars19. Household size20. Number of adults (15+) in the
household21. Male or female22. Age group23. Work status24. Ethnicity25. Household income group
May 2009 26TRB National Transportation Planning Applications Conference
Redundant Questions in DART 2007 Survey1. Home address2. Trip origin address and type3. Mode of access4. Transfer from5. Total number of transit
vehicles in the trip6. Transfer to7. Mode of egress8. Trip destination address and
type9. First and last rail lines and
stations boarded and alighted 10. Route sequence11. Main reason for taking the
route12. Perceived trip length in
minutes
13. Frequency of use14. Weekend and/or weekday users15. Substitute mode of travel16. Type of payment for the transit17. Type of fare18. Number of registered cars19. Household size20. Number of adults (15+) in the
household21. Male or female22. Age group23. Work status24. Ethnicity25. Household income group
May 2009 27TRB National Transportation Planning Applications Conference
Clarification of One-Way Trip Use of graphic diagrams. Advertisement campaigns:
Rail hangers in Fort Worth Posters in Denton
Consistent use of the term “one-way trip.” Consistent font throughout the questionnaire.
May 2009 28TRB National Transportation Planning Applications Conference
May 2009 29TRB National Transportation Planning Applications Conference
Use of Graphic Diagrams
Advertisement Campaign – FWTA
May 2009 30TRB National Transportation Planning Applications Conference
Advertisement Campaign – DCTA
May 2009 31TRB National Transportation Planning Applications Conference
Consistent Use of Term “One-Way Trip”
May 2009 32TRB National Transportation Planning Applications Conference
Questionnaire Design Modified text design:
Graphic design:
May 2009 33TRB National Transportation Planning Applications Conference
Modified Text Design
May 2009 34TRB National Transportation Planning Applications Conference
May 2009 35TRB National Transportation Planning Applications Conference
Graphic Design
Pilot Tests In-field: test on buses of two questionnaire designs
(NCTCOG staff). Cognitive interview (consultant staff)
People recruited and interviewed for one hour. Asked to fill out both questionnaire designs. Discussion on how they answered each question and
what they did like and did not like in each questionnaire.
May 2009 36TRB National Transportation Planning Applications Conference
In-Field Pilot Test Observations Shaky buses made writing hard. Early morning dark conditions in the bus should be
considered in design. Font sizes should correspond to all users (some
users may not use their glasses in the bus). The placement of the Advertisement poster on a
clear glass made it difficult to read. Informal interviews conducted with respondents to
learn more about riders’ view of the transit system.
By personally conducting the pilot test, NCTCOG staff got a realistic feeling of what surveyors and respondents experienced.
May 2009 37TRB National Transportation Planning Applications Conference
Pilot Test Results Comparative in-field test.
Modified text: 69% completion rate Graphic: 50% completion rate
Cognitive interviews. 28 interviewees Mixed response to both questionnaires.
May 2009 38TRB National Transportation Planning Applications Conference
Pilot Test Conclusions Modified text design was marginally more
successful than graphic design. Tests of graphic design were not comprehensive enough
to reach a solid conclusion. Final design incorporated a few elements of the graphic
design.
In-field pilot test was more useful. Cognitive interview method might be useful when
there are many unknowns about which to decide.
May 2009 39TRB National Transportation Planning Applications Conference
Final Questionnaire Design
Sample Expansion Plan Boarding counts by stop
APC counts not available. Manual counts for surveyed vehicles only.
Station counts by mode of access/egress Major park-and-ride and transfer stations.
Non-response follow-up survey Personal interviews of people not returning the
questionnaire. Measurement of bias in the filled surveys.
May 2009 41TRB National Transportation Planning Applications Conference
Non-Response Follow-up Personal Interview Form
NCTCOG (The T) On-Board Non-Response Survey
Assignment #: ____________ Trip #:______________ Route #:________________________
1) Reason for not taking/completing a survey: Never participate Too many questions Not interested/Don’t Care Conditions on bus not suitableNo time to complete it on this trip / trip too short Other (specify): _______________________________
2) How many minutes will you be traveling on THIS BUS for THIS TRIP? 5 or less 6-10 11-15 16-20 21-25 26-30 More than 30
3) Age 15 - 24 25 - 34 35 - 44 44 - 54 55-64 65+
4) Ethnicity White African American Hispanic Asian Native American Other (specify): ______________________
5) HH Income 2007 <10K 10K–14.9K 15K–24.9K 25K–34.9K 35K-49.9K 50K–75.9K 75K+
May 2009 42TRB National Transportation Planning Applications Conference
Non-Response Survey Results
Survey Question # Responded % Response
Reason for refusal 1,424 98.27%
Minutes traveled 903 62.32%
Age 1,332 91.93%
Ethnicity 1,366 94.27%
Income 500 34.51%
Response Rate to Questions
Total Participation: 1,449 surveys (Early estimate of 87.8% response rate)
May 2009 43TRB National Transportation Planning Applications Conference
Acknowledgment FTA staff: for providing ideas and help in analyzing
the results: Jim Ryan Ken Cervenka
NCTCOG Travel Model Development staff: for managing the project, analysis, and presentation: Arash Mirzaei Behruz Paschai Hua Yang
May 2009 44TRB National Transportation Planning Applications Conference