Am J Gastroenterol 2012; 107:612–619Thomas L. Bollen et al.

A COMPARATIVE EVALUATION OF RADIOLOGIC AND CLINICAL SCORING SYSTEMS IN THE EARLY PREDICTION OF SEVERITY IN ACUTE PANCREATITIS

INTRODUCTION AP can vary from a mild self-limited disease in approximately

80 – 90 % of patients, to a clinically severe form in 10 – 20 % with local and systemic complications.

The identification of patients with clinically severe AP is important for several reasons: first, these patients may benefit from transfer to an intermediate or intensive care unit, where they can receive aggressive fluid resuscitation and be closely monitored for the development of organ failure.

Second, these patients may benefit from targeted therapy, i.e., enteral feeding, endoscopic sphincterotomy, or antibiotics.

Finally, severity stratification is important when reporting and evaluating the results of clinical trials in AP.

Besides assessment of relevant clinical and biochemical parameters in AP, in many centers, it is standard practice to obtain a computed tomography (CT) scan on admission (i.e., within 24 h of hospitalization), not only for diagnostic purposes but also for assessing the severity of disease.

The severity of AP by CT imaging can be evaluated using unenhanced or contrast-enhanced CT studies. Unenhanced CT scoring systems evaluate the extent of pancreatic and peripancreatic inflammatory changes (Balthazar grade and ‘‘ pancreatic size index ’’ or PSI) or evaluate both peripancreatic inflammatory changes and extrapancreatic complications ( ‘‘ mesenteric oedema and peritoneal fluid ’’ or MOP score, ‘‘extrapancreatic ’’ or EP score, and the more recently developed ‘ ‘ extrapancreatic inflammation on CT ’ ’ or EPIC score).

In addition, there are two CT scoring systems that require the use of intravenous contrast agents to determine the presence and extent of pancreatic parenchymal necrosis. The ‘ ‘ CT severity index ’ ’ or CTSI is a numerical scoring system combining the quantification of extrapancreatic inflammation with the extent of pancreatic necrosis.

Mortele et al. proposed a ‘‘ modified CTSI ’’ or MCTSI, which, in addition to the CTSI, assigns points for extrapancreatic complications (vascular, gastrointestinal and extrapancreatic parenchymal complications as well as the presence of pleural effusion and / or ascites).

Although many studies have demonstrated a correlation between morphologic severity according to CT scoring systems and clinical disease severity, only five utilized data from “ early ” CT scans, defined as those obtained within 24 h of admission to the hospital.

No comparison of the accuracy of the existing CT scoring systems was performed and no comparative assessment was done between the CT scoring systems and clinical scoring systems such as the (APACHE)-II and the recently developed and validated Bedside Index for Severity in AP (BISAP).

Th e aim of this study is to compare the accuracy of seven different existing CT scoring systems in predicting the severity of AP on the first day of admission.

The secondary aim is to assess whether these CT scoring systems are superior to two commonly employed clinical scoring systems.

METHODS The demographic, clinical, and laboratory data of all

consecutive patients with a primary diagnosis of AP admitted or transferred to our institution during a 2.5-year period was prospectively collected for this study.

AP was defined as two or more of the following: characteristic abdominal pain; serum amylase and / or lipase levels three or more times the upper limit of normal (i.e., >210 and 180 U / l, respectively); and / or an imaging study (CT or magnetic resonance imaging) demonstrating changes consistent with AP.

The day of admission was defined as the first 24 h of hospitalization in our institution or in the referring hospital.

For all episodes, appropriate clinical data were recorded prospectively by two authors (V.K.S. and K.R.), who were unaware of the CT scores, to permit calculation of the APACHE-II, BISAP, and Charlson Comorbidity Index scores. The decision to obtain a CT scan was based on the clinical discretion of the evaluating physician.

Two radiologists retrospectively reviewed all CT studies and were unaware of patient outcomes.

The following parameters were collected for each episode of AP:

In-hospital mortality, length of hospital stay, admission to and length of intensive care unit stay, presence and duration of organ failure (transient; ≤ 48 h and persistent; >48 h), pancreatic infection (infection of pancreatic and / or peripancreatic necrosis), and need for intervention (endoscopic, percutaneous drainage, and / or surgical necrosectomy).

Clinically severe AP was defined as one or more of the following: mortality, persistent organ failure and / or the presence of local pancreatic complications that require intervention (endoscopic or radiologic drainage or surgical necrosectomy).

This definition is in accordance with the most updated revised Atlanta classification.

The principle distinction between the new and former definition of clinical severity is that the mere presence of pancreatic parenchymal necrosis, peripancreatic collections, or organ failure is not regarded as clinically severe disease, unless organ failure exceeds 48 h in duration or complications of pancreatic necrosis or peripancreatic collections occur, which require active intervention.

STATISTICAL ANALYSIS

Descriptive statistics were used for baseline characteristics, outcomes, and CT parameters.

The diagnostic accuracy of each scoring system for mortality and clinical severity were assessed using the area under the receiver operating characteristic curve (AUC) with standard error and 95 % confi dence intervals (CIs).

To rule out potential bias introduced by the inclusion of transferred patients, receiver operating characteristic curve (ROC) analysis was also performed in the non-transferred group. All statistical analysis was performed using SAS v.9.1 (SAS, Cary, NC), SPSS v15.0 (SPSS, Chicago, IL), and MedCalc v.10.4.3.0 (MedCalc, Mariakerke, Belgium).

RESULTS

The 159 episodes of AP in which an early CT scan was performed constitute our study cohort.

The additional 187 episodes in which no ( n = 139) or delayed ( n = 48) CT imaging was performed, were excluded from the study.

In 131 episodes, a contrast-enhanced CT was performed permitting the assessment of all seven CT scoring systems; Balthazar grade, CTSI, MCTSI, EP score, EPIC score, MOP score, and PSI.

In 28 episodes, only an unenhanced CT scan was performed, allowing the assessment of only fi ve of the seven CT scoring systems.

The median age of patients was 54 years (range 21 – 91) with 84 men and 66 women.

Etiologies of AP included gallstones in 4 8 (30 % ) episodes, miscellaneous (e.g., hypertriglyceridemia, hereditary, and post-endoscopic retrograde cholangiopancreatography) in 38 (24 % ) episodes, alcohol in 34 (21 % ) episodes, idiopathic in 27 (17 % ) episodes, and drug-induced in 12 (8 % ) episodes.

One hundred and thirty episodes (82 % ) were labeled as mild AP and 29 episodes (18 % ) as clinically severe AP.

In 16 episodes, the lack of enhancement of pancreatic parenchyma was noted indicating acute necrotizing pancreatitis (four of whom were categorized as having clinically mild disease).

On follow-up imaging, 13 more episodes were identified, in whom necrotizing pancreatitis was detected (three of them had an unenhanced CT on admission). The lack of enhancement on follow-up imaging was not used for data analysis.

In all episodes, early CT did not reveal an alternative diagnosis or local complication that changed clinical management.

COMPARISON OF SCORING INDICES IN PREDICTING CLINICAL SEVERITY

On the basis of highest sensitivity and specificity values generated from the ROC curves, the following cutoffs were selected for predicting clinically severe disease:

CTSI ≥ 4, MCTSI ≥ 6, Balthazar grade ≥ 5, EPIC score ≥ 3, EP score ≥ 3, MOP score ≥ 2, PSI ≥ 1, APACHE-II ≥ 10, and BISAP ≥ 3.

AUC comparison of computed tomography (CT) and clinical scoring systems for predicting clinical severity.

Although the CTSI demonstrated the highest accuracy for predicting clinically severe AP among the 131 cases who underwent contrast-enhanced CT (AUC 0.88; 95 % CI: 0.82 – 0.93), no statistically significant pairwise differences were observed between the CTSI and the other CT scoring systems, except for PSI ( P = 0.014).

Balthazar grade demonstrated the highest accuracy for clinically severe AP for all 159 episodes (AUC 0.79; 95 % CI: 0.72 – 0.85). However, no statistically significant differences were observed between the Balthazar grade and the other CT scoring systems.

Also, no statistically significant differences were found between the CT and clinical scoring systems with highest AUC for clinical severity.

No significant changes in results were observed when excluding the transferred patients.

COMPARISON OF SCORING INDICES IN PREDICTING MORTALITY

On the basis of highest sensitivity and specificity values generated from the ROC curves, the following cutoffs were selected for predicting mortality: CTSI ≥ 4, MCTSI ≥ 4, Balthazar grade ≥ 5, EPIC score ≥ 3, EP score ≥ 3, MOP score ≥ 1, PSI ≥ 1, APACHE-II ≥ 17, and BISAP ≥ 3.

Among the CT indices, Balthazar grade had the highest AUC for mortality in both groups (AUC 0.81; 95 % CI: 0.74 – 0.88 and AUC 0.79; 95 % CI: 0.72 – 0.85, respectively).

In the study cohort, the APACHE-II score performed best among all studied indices in both groups of patients (AUC 0.91; 95 % CI: 0.85 – 0.95 and AUC 0.91; 95 % CI: 0.86 – 0.95, respectively).

COMPARISON OF SCORING INDICES IN PREDICTING MORTALITY

No statistically significant differences were observed between Balthazar grade and the other CT scoring systems and between Balthazar grade and the clinical scoring system with the highest accuracy for predicting mortality (APACHE-II).

When using a fixed cutoff value for APACHE-II for predicting clinically severe disease and mortality (i.e., the universally accepted value of 8 or more) there was an increase of the sensitivity and negative predictive value and concomitant decrease in specificity and positive predictive value compared with the optimal cutoff s derived from the ROC curves.

The AUC value of APACHE-II for both cutoff values was the same and, again, no significant changes were seen between CT scoring system and APACHE-II.

DISCUSSION

This study did not detect any significant differences between the studied CT scoring systems.

There was no advantage of performing a CT on admission as an independent predictor over the more easily obtainable clinical scoring systems in terms of accuracy in predicting clinically severe AP and mortality

There are several potential explanations for the observed mode rate accuracy for clinical severity and mortality using CT scoring systems.

First, the anatomic extent of pancreatic inflammation and the size and volume of peripancreatic fluid collections are not included in any of the studied scoring systems; both peripancreatic fat stranding and fluid collections can range from discrete to extensive in magnitude, but are accorded equal points in the studied scoring systems.

Second, some patients initially predicted to have mild AP may, nonetheless, progress to clinically severe AP over the initial 48 h of hospitalization along with worsening morphologic changes on imaging.

A substantial number of patients with pancreatic necrosis established on admission CT (25 % in the present study and 38 % in a study by Casas et al.) did not develop clinically severe disease. It remains poorly understood why differentclinical courses are observed in patients with significant pancreatic parenchymal necrosis.

Scoring systems work best at the extremes of the spectrum (i.e., high negative or positive predictive value in patients with very low or high scores), whereas the performance of these scoring systems is only moderate in intermediate scores.

In five previous studies, all showed a moderate positive correlation between CT score and mortality and morbidity.

A comparison of our results with those of previously published studies is difficult for several more reasons. First, no clear definition of clinical severity of disease has been consistently applied. Second, different threshold values, as opposed to ROC curve analysis, were used to predict clinical severity. It is generally acknowledged that the overall performance of a test and comparison of prognostic scoring systems is best performed using ROC curve analysis, which was only one study. Third, only one of these studies performed comparative analysis between the CT scoring systems

and no studies compared the radiologic systems with clinical scoring systems.

Although this study highlights the prognostic accuracy of CT, we feel that a more judicious use of CT in AP is warranted .

This study showed that early CT did not reveal any other diagnosis, did not reveal any local pancreatic complication, and underestimated the presence of parenchymal necrosis in a substantial number of patients.

Hence, we recommend that CT studies should be reserved only for those patients with predicted severe AP by clinical assessment, for those who fail to improve clinically with conservative management or those in whom the diagnosis is unclear or a severe complication is suspected (such as bleeding, bowel ischemia, or perforation).

LIMITATIONS

First, not all patients admitted or transferred to our hospital underwent a CT on the day of admission. Instead, CT was performed based on the discretion of treating physician (primarily for severity assessment) and, therefore, our methodology reflects, in some respects, current clinical practice.

Second, there were a relatively small number of severe cases. However, this study was the largest so far to compare the use of different CT and clinical prognostic scoring systems on the day of admission. In addition, the prevalence of mild and severe cases in our study (82 % and 18 % , respectively) is similar to the literature ( 1 ).

CONCLUSION

Our study did not detect significant differences between any of the seven studied CT scoring systems in predicting mortality and clinical severity of AP.

Moreover, CT scoring systems were not superior to the studied clinical scoring systems.

There appears to be no advantage of performing a CT on admission for prognostic purposes compared with the simpler and more easily obtained clinical scoring systems and, therefore, obtaining a CT for assessment of severity on the day of admission is not recommended.

Thank you for your attention