www.elsevier.com/locate/jpedsurg

Journal of Pediatric Surgery (2008) 43, 783–787

Mortality prediction in congenital diaphragmatic herniaRobert Bairda,b, Ying C. MacNabc, Erik D. Skarsgarda,b,⁎the Canadian Pediatric Surgery Network

aDivision of Pediatric Surgery, British Columbia Children's Hospital, Vancouver, British Columbia, Canada V6H 3V4bDepartment of Surgery, University of British Columbia, Vancouver, British Columbia, Canada V5Z 4E3cDepartment of Health Care and Epidemiology, University of British Columbia, Vancouver, British Columbia,Canada V6H 3V4

Received 12 November 2007; accepted 3 December 2007

P

AT

0d

Key words:Congenitaldiaphragmatic hernia;

Survival;Risk-adjustment;Predictive equation

AbstractBackground: A validated risk stratification tool for congenital diaphragmatic hernia (CDH) is requiredfor accurate outcomes analyses. Existing mortality-predictive models include those of the CDH StudyGroup (CDHSG) based on birth weight and 5-minute Apgar score, the Canadian Neonatal Network(CNN) based on gestational age and admission score in Score for Neonatal Acute Physiology version II,and the Wilford Hall/Santa Rosa clinical prediction formula (WHSRPF) derived from blood gasmeasurements. The purpose of this study was to evaluate the calibration and discrimination of thesepredictive models using the Canadian Pediatric Surgical Network dataset.Methods: Neonatal risk variables and birth hospital survivorship were collected prospectively in 11perinatal centers, between May 2005 and October 2006. Actual vs predicted outcomes were analyzedfor each equation to measure the calibration and discrimination of each model.Results: Twenty (21.2%) of 94 infants with CDH died during birth hospitalization. The CDHSGmodel demonstrated superior discrimination (area under the receiver operator characteristic curve = 0.85;CNN = 0.79; WHSRPF = 0.63). Model calibration reflected by the Hosmer-Lemeshow P value was poorestwith the WHSRPF = 0.37 and comparable between CDHSG and CNN (0.48 and 0.46, respectively).Conclusion: Predictive outcome models are essential for risk-adjusted outcome analysis of CDH. The idealpredictive equation should prove robust across CDH datasets.© 2008 Elsevier Inc. All rights reserved.

Despite significant advances in prenatal diagnosis and birth. Although implementation of rational treatment strate-

neonatal intensive care, congenital diaphragmatic hernia(CDH), continues to be a vexing congenital malformationwith broadly variable cardiopulmonary disease severity at

Presented at the 39th Annual Meeting of the Canadian Association ofediatric Surgeons, August 23-26, 2007, St John's Newfoundland, Canada.⁎ Corresponding author. Division of Pediatric General Surgery, KO-123

CB, 448 Oak Street, Vancouver British Columbia, Canada V6H 3V4.el.: +1 604 875 3744; fax: +1 604 875 2721.E-mail address: eskarsgard@cw.bc.ca (E.D. Skarsgard).

022-3468/$ – see front matter © 2008 Elsevier Inc. All rights reserved.oi:10.1016/j.jpedsurg.2007.12.012

gies including preoperative stabilization, lung protectiveventilation, extracorporeal membrane oxygenation (ECMO),pulmonary vasodilator therapy, and delayed surgical treat-ment have resulted in reduced mortality trends in individualcenters, there continues to be great variability in overall CDHmortality with rates ranging between 20% and 60% [1-4].

An ongoing barrier to outcome research in CDH is thelack of a validated and widely accepted disease-severityadjustment tool. Validated illness-severity assessment of thenewborn with CDH should be done early in the postnatal

784 R. Baird et al.

course so that the outcome prediction provided is notsignificantly influenced by postnatal treatment. Earlyoutcome prediction enables anticipation of the likelihoodof the need for more aggressive treatment strategies (suchas ECMO) and informs physician-parent discussions on anindividual patient basis. From the perspective of studyingpopulations of CDH infants, such a tool allows compar-isons of risk-adjusted patients within and between institu-tions. Finally, accurate risk-adjusted outcome stratificationenables the rigorous evaluation of treatment strategies inprospective trials.

There have been at least 3 predictive equations derived forearly CDH mortality. These include the predictive equations ofthe CDH Study Group (CDHSG) [5], the Canadian NeonatalNetwork (CNN) [6], and the Wilford Hall/Santa Rosa group(WHSRPF) [7]. Each predictive equation was derived from aunique CDH patient cohort. The purpose of our study was tovalidate each of these outcome predictors using a single CDHdataset from the Canadian Pediatric Surgical Network(CAPSNET) database.

1. Materials and methods

The dataset was obtained from the CAPSNET, a multi-disciplinary group of 16 Canadian perinatal centers that collectprospective, disease-specific data on both CDH, and gastro-schisis (Appendix 1). A perinatal center is defined as onewith alevel III neonatal intensive care unit (NICU), pediatricanesthesia, and subspecialty surgery (at least general andneurosurgery) capabilities, and a geographically or function-ally adjoined maternal-fetal medicine/advanced prenataldiagnosis center. Pediatric cardiac surgery and ECMO areavailable in 9 and 4 CAPSNET centers, respectively. InCanada, all perinatal care for birth defects such as CDH isprovided exclusively through these provincial referral centers.

Eleven centers contributed data during the study period;the largest site contributed 25 patients whereas 3 sitescontributed 2 patients each.

1.1. Study population

Congenital diaphragmatic hernia cases for this study wereaccrued between May 1, 2005, and December 31, 2006.Cases were ascertained at prenatal diagnosis (if one wasmade) or after birth, and data were abstracted from diagnosisto death or discharge from a CAPSNET center. Infantstransferred from one CAPSNET center to another weretracked back to their initial admission.

1.2. Data collection

Notification of the prenatal diagnosis or birth of a case ofCDH was forwarded to on-site, trained research assistantswho abstracted data from maternal and infant charts using a

customized data entry program with built-in error checkingand a standard manual of operations and definitions. Datawere deidentifed and transmitted electronically to a cen-tralized repository where data were cleaned, stored, andthereafter managed by a study coordinator and a geographi-cally representative, multidisciplinary steering committeecomprised of pediatric surgeons, neonatologists, maternal-fetal medicine specialists, and an epidemiologist.

1.3. Predictive models

Three predictive models were evaluated. Each model wasderived from a separate CDH patient cohort, by testing(individually and in combination), those risk variablesdeemed to be predictive of mortality by multivariablelogistic regression analysis within that cohort:

(1) The CDHSG probability of survival equation = 1−1/(1+e−x), where −x = −5.0240 + 0.9165 (birth weight inkilograms) + 0.4512 (Apgar score at 5 minutes) [5].

(2) The CNN predictive equation that uses a combina-tion of 2 risk variables, the Score for Neonatal AcutePhysiology version II (SNAP-II), and gestational age (GA).The SNAP-II is a standardized index, validated in otherneonatal patient populations, which depicts illness severityby the magnitude of derangement in 6 physiologicparameters: mean blood pressure, lowest temperature, PO2(mm Hg)/FIO2 (%) ratio, lowest serum pH, presence ofseizure activity, and urine output (mL/kg per hour),expressed as an aggregate score [8].

(3) WHSRPF. This equation uses blood gas values(from a primarily postductal source), measured during thefirst 24 hours of life to calculate the equation: highestPaO2−highest PCO2, with a cutoff value of zero or greaterexpected to predict survival [7].

1.4. Data analysis

Perinatal characteristics, characteristics of operatedpatients, and outcomes were recorded directly from theCAPSNET database. For each predictive equation, modeledand actual outcomes were compared using the receiveroperator characteristic (ROC) curve technique of Hanley andMacNeil [9] to assess model discrimination. The area underthe ROC curve (AUC) depicts model discrimination—ameasure of its predictive performance. An AUC of 0.5represents a completely random association between themodeled and actual outcomes, whereas an AUC of 1.0represents perfect discrimination. The conformity betweenactual and predicted outcome is also depicted by modelcalibration or “goodness of fit,” where, using the Hosmer-Lemeshow (H-L) technique, a P value of .05 or highersuggests that there is no difference between modeled andactual outcomes. The higher the P value, the better the modelcalibration [10]. All analyses were performed using SPSS forWindows statistical software (SPSS, Chicago, Ill).

Table 2 Predictive performance and goodness-of-fit oflogistic regression models for predicting mortality amonginfants with CDH

Variable CNN andSNAP-II(alone)

CNN andSNAP-II + GA(combined)

CDHSG WHSRPF

AUC 0.79 0.78 0.85 0.63χ2 5.66 12.27 6.54 8.67H-L P .46 .14 .48 .37

785Mortality prediction in congenital diaphragmatic hernia

2. Results

2.1. Descriptive characteristics and outcomes ofCDH cohort

The total number of NICU admissions during the studyperiod from the 11 CAPSNET contributing sites was 10,094.One hundred thirteen CDH cases were entered into thedatabase, of which 105 were liveborn. Eleven patients hadopen files (meaning they remained alive in hospital), whereas94 achieved the outcome criteria of death or discharge fromthe hospital of birth, and these represent the study cohort. Ofthese, 62 (66.0%) were diagnosed in the prenatal period.Babies were born at a mean GA of 38 weeks and a mean birthweight of 3103 g. Thirty-four babies (40.4%) were eitherinborn or had a planned delivery at a functionally linked,specialized obstetrical center. Nine babies (10.5%) requiredECMO therapy, 4 of whom (44.4%) did not survive.Seventy-six babies (84.8%) survived to surgery, which wasperformed an average of 3.5 ± 2.8 days after birth. Sevenbabies underwent surgery on ECMO. The CDH defect wasleft-sided in 64 (76.2%) of 84 known cases, and prostheticpatch repairs were performed in 25 (30.4%) of 82 operatedbabies. Overall, 74 live-born neonates survived to discharge(78.7% of live births with closed files) with a mean length ofstay of 24.4 ± 18.2 days (Table 1).

2.2. Predictive performance and calibration ofeach model

In this comparative analysis, the CDHSG predictiveequation demonstrated the best discrimination with an AUCof 0.85. The CNN model had comparable discriminationusing SNAP-II alone (AUC = 0.79) or in combination withGA (AUC = 0.78) but was inferior to that of the CDHSG.The WHSRPF demonstrated the poorest discrimination withan AUC of only 0.63. From the perspective of modelcalibration, the CDHSG performed slightly better than theCNN (SNAP-II alone) with H-L P values of .48 and .46,

Table 1 Characteristics and selected outcomes in CDHpatients in CAPSNET Study cohort

Parameter Value (n [%])

Female sex 46 (48.9)Weight (g) 3081 ± 682GA (wk) 38 ± 2.2Cesarean delivery 26 (27.6)Prenatal diagnosis 62 (68.1)Left-sided hernia 47 (81.0)Inborn 34 (40.4)ECMO required 9 (9.6)Mesh repair 21 (33.9)Survival to discharge 74 (78.7)Length of stay (d) 24.4 ± 18.2

respectively. The WHSRPF H-L P value was .371, whereasthe CNN combined model (SNAP-II + GA) had the poorestcalibration with an H-L P value of .14 (Table 2).

3. Discussion

Congenital diaphragmatic hernia presents with a spectrumof disease severity that makes it difficult to accurately predictoutcome. Although some neonates are born virtually asympto-matic, others require maximal NICU treatment strategies,including high frequency oscillatory ventilation, inhaled nitricoxide, and ECMO before operative repair [11-16]. Predictingthe specific interventions a patient is likely to require as well astheir overall outcome based on early physiologic performanceremains an important goal in CDH research.

Several publications have attempted to describe anaccurate outcome predictor for neonates with CDH [5-7].Although each has proven reasonable in the context of itspublication cohort, they all have yet to demonstratereliability in subsequent investigations. The CDHSG scorewas derived from a large cohort (N1000) of CDH neonatesand allowed for risk stratification into terciles based on thebirth weight and 5-minute Apgar [5]. The CDHSG under-estimated actual survival in 2 other cohorts [1,3], however,suggesting that it might be improved upon.

We recently reported on the SNAP-II score as a predictorof mortality in infants with CDH [6]. The SNAP-II score, avalidated outcome predictor in non-CDH neonatal popula-tions, applies a weighted score to 6 physiologic variableswithin the first 12 hours of admission to the NICU [8]. In aCDH cohort of 88 CDH patients from the CNN database,multivariable logistic regression revealed that SNAP-II, incombination with GA, yielded a predictive model withcomparable discrimination and superior calibration (AUC =0.81; H-L P = .88) compared to the CDHSG predictiveequation (AUC = 0.83; H-L P = .06) [6].

Recently, Schultz et al [7] describe a simplifiedpostnatal predictor of outcome based on arterial bloodgas measurements obtained during the initial 24 hours oflife (but before ECMO or operative repair). The WHSRPF

was developed based on known respiratory pathophysiol-ogy in CDH [17,18]. It was investigated retrospectively ina local group of 88 patients and subsequently in a CDHSG

786 R. Baird et al.

cohort of 849 patients, both accrued during a 7-yearperiod. The WHSRPF demonstrated a discrimination(AUC) of 0.87 in the local group and 0.79 in theCDHSG data set, comparable with the CDHSG's ownequation (0.76) [7]. Goodness-of-fit was not reported.

In this study, we collected prospective, disease-specificdata from 11 Canadian institutions for an 18-month periodand investigated the predictive performance of these 3outcome models. Our results demonstrate that the CDHSGmodel had superior discrimination to all other models(AUC = 0.85), whereas the WHSRPF had the poorestdiscrimination with an AUC = 0.63. All models demon-strated reasonable goodness-of-fit, with the CDHSG modelagain outperforming the other 2 predictive models.

The reason for the superior predictive ability of theCDHSG is unclear. The CDHSG score is generated from 2descriptive data points (birth weight and 5-minute Apgar),the CNN score includes one descriptive data point (GA),and 6 physiologic variables (SNAP-II), whereas theWHSRPF is generated from 2 physiologic variables. TheWHSRPF has the advantage of being easily generated andcalculated, whereas the CDHSG score necessitates somemathematical manipulation of easily collected birth data.Both are therefore relatively easy to apply. Mostcomponents of the SNAP-II are also collected, althoughnot all components (lowest temperature, for example) arenecessarily recorded, nor is the SNAP-II score itselfroutinely calculated and recorded. The combined scoretherefore uses the greatest number of physiologic variableand theoretically may provide the best snapshot of neonatalphysiology of the 3 scores. Its performance, however, maynot justify its routine calculation.

It must also be conceded that given the evidencefavoring delayed surgical repair [19,20], delayed orsequential risk stratification may ultimately provide themost accurate prognostic information. Whereas all of theinvestigated models are described in the early NICU course(CDHSG at birth, SNAP b12 hours, WHSRPF b24 hours),investigating a particular predictive model at differenttime-points may enable more accurate outcome assessment.Finally, it is possible that some combination of existingpredictive models may offer further refinement of theaccuracy of outcome prediction.

Several predictive models exist for the postnatal evalua-tion of CDH infants. Given the importance of CDH riskadjustment to outcome analysis and informing individualfamily counseling, further investigation is warranted toidentify the optimal predictive equation.

Acknowledgments

This study was supported by grant FRN no. 69050 fromthe Canadian Institute of Health Research (Ontario, Canada).We thank all the local CAPSNET data abstractors for their

work in data collection and Jennifer Claydon for herassistance in compiling the data.

Appendix A

Participating CAPSNET Centers:Victoria General Hospital, Victoria, British ColumbiaB.C. Children's & Women's Health Centre, Vancouver,

British ColumbiaRoyal University Hospital, Saskatoon, SaskatchewanWinnipeg Health Sciences Centre, Winnipeg, ManitobaSt. Boniface Health Centre, Winnipeg, ManitobaHospital for Sick Children, Toronto, OntarioMt. Sinai Hospital, Toronto, OntarioMcMaster Children's Hospital, Hamilton, OntarioLondon Health Sciences Centre, London, OntarioKingston General Hospital, Kingston, OntarioChildren's Hospital of Eastern Ontario, Ottawa, OntarioMontreal Children's Hospital, Montreal, QuebecHôpital Ste-Justine, Montreal, QuebecCentre Hospitalier de L'Université Laval, Ste-Foy,

QuebecIWK Health Centre, Halifax, Nova ScotiaCharles Janeway Child Health Centre, St. John's,

Newfoundland

References

[1] Downard CD, Jaksic T, Garza JJ, et al. Analysis of an improvedsurvival rate for congenital diaphragmatic hernia. J Pediatr Surg2003;38:729-32.

[2] Stege G, Fenton A, Jaffray B. Nihilism in the 1990s: the truemortality of congenital diaphragmatic hernia. Pediatrics 2003;112:532-5.

[3] Javid PJ, Jaksic T, Skarsgard ED, et al. Survival in congenitaldiaphragmatic hernia: the experience of the Canadian NeonatalNetwork. J Pediatr Surg 2004;39:657-60.

[4] The Ontario Congenital Anomalies Study Group. Apparent truthabout congenital diaphragmatic hernia: a population based databaseis needed to establish benchmarking for clinical outcomes for CDH.J Pediatr Surg 2004;39:661-5.

[5] The Congenital Diaphragmatic Study Group. Estimating diseaseseverity of congenital diaphragmatic hernia in the first 5 minutes oflife. J Pediatr Surg 2001;36:141-5.

[6] Skarsgard ED, MacNab YC, the Canadian Neonatal Network, et al.SNAP-II predicts mortality among infants with congenital diaphrag-matic hernia. J Perinatol 2005;25:315-9.

[7] Schultz CM, DiGeronimo RJ, Yoder BA. Congenital diaphragmatichernia: a simplified postnatal predictor of outcome. J Pediatr Surg2007;42:510-6.

[8] Richardson DK, Corcoran JD, Escobar GJ, et al. SNAP-II andSNAPPE-II: simplified newborn illness severity and mortality scores.J Pediatr 2001;138:92-100.

[9] Hanley JA, McNeil BJ. The meaning and use of the area under a receiveroperating characteristic (ROC) curve. Radiology 1982;143:29-36.

[10] Lemeshow S, Hosmer Jr DW. A review of goodness of fit statistics foruse in the development of logistic regression models. Am J Epidemiol1982;115:92-106.

787Mortality prediction in congenital diaphragmatic hernia

[11] Caciari A, Ruggeri G, Mondenti M, et al. High-frequency oscillatoryventilation versus conventional mechanical ventilation in congenitaldiaphragmatic hernia. Eur J Pediatr Surg 2001;11:3-7.

[12] Miguet D, Claris O, Lapillonne A, et al. Preoperative stabilization usinghigh-frequency oscillatory ventilation in the management of congenitaldiaphragmatic hernia. Crit Care Med 1994;22(9 Suppl):S77-82.

[13] Okuyama H, Kubota A, Oue T, et al. Inhaled nitric oxide with earlysurgery improves the outcome of antenatally diagnosed congenitaldiaphragmatic hernia. J Pediatr Surg 2002;37:1188-90.

[14] Kinsella JP, Parker TA, Ivy DD, et al. Noninvasive delivery of inhalednitric oxide therapy for late pulmonary hypertension in newborn infantswith congenital diaphragmatic hernia. J Pediatr 2003;142:397-401.

[15] Atkinson JB, Kitagawa H. Extracorporeal membrane oxygenation andthe management of congenital diaphragmatic hernia. Pediatr Surg Int1993;8:200-3.

[16] Flamant C, Lorino E, Nolent P, et al. Newborn infants supported byextracorporeal membrane oxygenation: survival and clinical outcome.Arch Pediatr 2007;14:354-61.

[17] Bohn DJ, James I, Filler RM, et al. The relationship between PaCo2and ventilation parameters predicting survival in congenital diaphrag-matic hernia. J Pediatr Surg 1984;16:666-71.

[18] Germain JF, Farnouz C, Pinquier D, et al. Can blood gas values predictpulmonary hypoplasia in antenatally diagnosed congenital diaphrag-matic hernia. J Pediatr Surg 1996;31:1634-9.

[19] Langer JC, Filler RM, Bohn DJ, et al. Timing of surgery for congenitaldiaphragmatic hernia: is emergency operation necessary. J Pediatr Surg1988;23:731-4.

[20] West KW, Bengston K, Rescorla FJ, et al. Delayed surgical repair andECMO improves survival in congenital diaphragmatic hernia. AnnSurg 1992;216:454-62.