An Assessment of Repeatability
for Crown Measurements Taken
on Conifer Tree Species
James A. WestfallWilliam A. Bechtold
KaDonna C. Randolph
USDA Forest ServiceUSDA Forest ServiceForest Inventory and AnalysisForest Inventory and Analysis
FIA QA/QC Data CollectionFIA QA/QC Data Collection
• Hot Checks• Cold Checks• Blind Checks
• Independent Plot Remeasurement• Randomly Chosen Plots• Experienced Personnel• Target 3%
Conifer DataConifer DataFIA Tree Crown Indicator (Phase 3)
• Uncompacted Crown Ratio (nearest 1%)
• Light Exposure (6 categories)
• Crown Position (4 categories)
• Crown Vigor Class (saplings – 3 categories)
• Crown Density (nearest 5%)
• Dieback (nearest 5%)
• Foliage Transparency (nearest 5%)
AnalysisAnalysis• Data Matching (i.e., trees)
No one-to-one correspondence (independent remeasure)
Manually expensive (large # observations)
Automate most matches– 2-pass approach
– Weighted distance = f(dbh, horz. distance, azimuth)
– ‘Conservatism’ via a decision rule
MUST review unmatched trees and add legitimate matches into analysis data set
AnalysisAnalysisMQOs and Tolerances
• Tolerance• A range of acceptable variation
• Can be specific value or percentage
• Example: ± 0.1 in. for dbh
• Measurement Quality Objective (MQO)• The desired percentage of measurements that fall
within the tolerance range
• Example: 95% of the time
AnalysisAnalysisComputations
• Obtain differences between field and QA crews for matched observations
• Determine percentage of total observations where difference is within the tolerance range
• Compare with MQO to see if standard is met
• Optional: compute percentages across range of tolerance values
ResultsResults
Variable Tolerance MQO @1x @2x @3x @4x RecordsUncompacted CR ±10 % 90 81.9% 94.7% 98.1% 99.6% 1350
Crown Light Exposure ±1 class 85 85.2% 93.0% 96.0% 97.3% 1350Crown Position No Tolerance 85 77.3% 1350
Vigor Class No Tolerance 90 81.5% 259Crown Density ±10 % 90 68.6% 91.1% 98.6% 99.5% 1091Crown Dieback ±10 % 90 97.7% 99.5% 99.6% 99.8% 1091
Foliage Transparency ±10 % 90 95.5% 98.8% 99.5% 99.9% 1091
Percentage of data within tolerance
*
ResultsResultsRegion Variable Tolerance
% within tolerance
Mean difference Records
NRS Uncompacted CR ±10 % 81.8 0.4 395SRS Uncompacted CR ±10 % 75.0 2.1 176IW Uncompacted CR ±10 % 84.2 -0.3 727PNW Uncompacted CR ±10 % 73.1 0.1 52
NRS Crown Light Exposure ±1 class 85.1 0 395SRS Crown Light Exposure ±1 class 86.9 -0.1 176IW Crown Light Exposure ±1 class 84.2 -0.2 727PNW Crown Light Exposure ±1 class 94.2 0.1 52
NRS Crown Position No Tolerance 84.6 -0.1 395SRS Crown Position No Tolerance 93.2 0 176IW Crown Position No Tolerance 69.2 0.1 727PNW Crown Position No Tolerance 80.8 -0.1 52
NRS Vigor Class No Tolerance 85.7 0 189SRS Vigor Class No Tolerance 76.9 -0.1 26IW Vigor Class No Tolerance 65.9 0.2 44PNW Vigor Class No Tolerance . . _
ResultsResultsRegion Variable Tolerance
% within tolerance
Mean difference Records
NRS Crown Density ±10 % 64.6 -5.1 206SRS Crown Density ±10 % 82.0 1.1 150IW Crown Density ±10 % 67.3 0.4 683PNW Crown Density ±10 % 61.5 -5.8 52
NRS Crown Dieback ±10 % 98.1 0.4 206SRS Crown Dieback ±10 % 97.3 -0.6 150IW Crown Dieback ±10 % 97.7 0.3 683PNW Crown Dieback ±10 % 98.1 0.4 52
NRS Foliage Transparency ±10 % 95.6 0.7 206SRS Foliage Transparency ±10 % 88.7 -1.1 150IW Foliage Transparency ±10 % 96.6 -0.4 683PNW Foliage Transparency ±10 % 100.0 -2.6 52
Uncompacted CRUncompacted CR
QA crew Tolerance% within tolerance
Mean difference Records
5.0% ±10 % 100.0 0 110.0% ±10 % 50.0 -16.5 215.0% ±10 % 50.0 -24.3 420.0% ±10 % 60.0 -11.1 1025.0% ±10 % 76.9 -4.2 2630.0% ±10 % 71.9 -6.3 3235.0% ±10 % 94.4 1.2 5440.0% ±10 % 79.7 -1.6 7945.0% ±10 % 78.7 -1.6 8950.0% ±10 % 77.5 -2.9 8955.0% ±10 % 82.3 1.1 7960.0% ±10 % 70.8 2.9 6565.0% ±10 % 61.4 -1.9 7070.0% ±10 % 77.5 0.9 7175.0% ±10 % 64.3 1.2 7080.0% ±10 % 76.9 0.1 9185.0% ±10 % 80.3 2.6 7690.0% ±10 % 89.2 1.9 11195.0% ±10 % 90.6 2.7 117
100.0% ±10 % 97.7 1.4 214
Vigor ClassVigor Class
QA crew Tolerance MQO% within tolerance Records
1 No Tolerance 90 88.8 2152 No Tolerance 90 51.3 393 No Tolerance 90 0.0 5
Crown DensityCrown Density
QA crew Tolerance MQO% within tolerance
Mean difference Records
0.0% ±10 % 90 0.0 -50 15.0% ±10 % 90 66.7 -11.7 3
10.0% ±10 % 90 60.0 -10 515.0% ±10 % 90 44.4 -15.6 920.0% ±10 % 90 75.0 -8.8 2025.0% ±10 % 90 73.8 -8.3 4230.0% ±10 % 90 65.5 -5.7 8435.0% ±10 % 90 77.8 -2.7 17140.0% ±10 % 90 67.8 -3.7 19945.0% ±10 % 90 74.9 -1 17550.0% ±10 % 90 69.0 0 11655.0% ±10 % 90 71.3 1.8 8760.0% ±10 % 90 66.7 5.3 7565.0% ±10 % 90 51.1 7.4 4770.0% ±10 % 90 50.0 10.5 2875.0% ±10 % 90 40.0 15.7 1580.0% ±10 % 90 27.3 18.6 1185.0% ±10 % 90 0.0 23.3 3
Crown DiebackCrown Dieback
QA crew Tolerance MQO% within tolerance
Mean difference Records
0.0% ±10 % 90 99.2 -1.4 7135.0% ±10 % 90 98.2 1.7 27310.0% ±10 % 90 100.0 5.3 7615.0% ±10 % 90 69.2 7.7 1320.0% ±10 % 90 60.0 3 525.0% ±10 % 90 100.0 10 130.0% ±10 % 90 33.3 8.3 335.0% ±10 % 90 0.0 -15 1
**45.0% ±10 % 90 0.0 45 150.0% ±10 % 90 0.0 35 1
**65.0% ±10 % 90 0.0 55 170.0% ±10 % 90 100.0 5 1
**90.0% ±10 % 90 0.0 25 2
Foliage TransparencyFoliage Transparency
QA crew Tolerance MQO% within tolerance
Mean difference Records
0.0% ±10 % 90 25.0 -17.5 45.0% ±10 % 90 66.7 -10 310.0% ±10 % 90 98.6 -3.9 21415.0% ±10 % 90 97.6 -0.9 46520.0% ±10 % 90 96.6 1.7 26425.0% ±10 % 90 86.4 2.5 10330.0% ±10 % 90 84.4 1.9 3235.0% ±10 % 90 75.0 3.8 4
**50.0% ±10 % 90 0.0 35 1
**95.0% ±10 % 90 0.0 80 1
ConclusionConclusion
Crown Light Exposure, Crown Dieback, and Foliage Transparency measurements met the stated repeatability standard.
Uncompacted CR, Crown Position, Crown Vigor, and Crown Density measurements did not meet the repeatability standard.
ConclusionConclusion• With few exceptions, levels of repeatability
are similar across geographic regions.
• The poorest repeatability statistics were generally associated with relatively rarer crown characteristics.
• For some variables, improved training and/ or re-evaluation of the tolerance/MQO may be needed.
ConclusionConclusion
• Quality assurance data are important for:• Evaluating training effectiveness• Employee performance feedback• Evaluating measurement protocols• Identifying significant sources of error for
computed attributes, model projections, etc.
ConclusionConclusion• Further reading:
• Westfall, J.A., ed. 2009. FIA national assessment of data quality for forest health indicators. USDA For. Serv. Gen. Tech. Rep. NRS-53.
• Pollard, J.E., et al. 2006. FIA national data quality assessment report for 2000-2003. USDA For. Serv. Gen. Tech. Rep. RMRS-181.
Questions ??Questions ??
Top Related