For peer review only
Neutrophil Gelatinase-Associated Lipocalin in the Prediction of Acute Kidney Injury among Adult Intensive Care Unit
Patients: Determination of Optimal Thresholds in Plasma and Urine
Journal: BMJ Open
Manuscript ID bmjopen-2017-016028
Article Type: Research
Date Submitted by the Author: 23-Jan-2017
Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute
<b>Primary Subject Heading</b>:
Renal medicine
Secondary Subject Heading: Diagnostics
Keywords: neutrophil gelatinase associated lipocalin, acute kidney injury, risk prediction, sensitivity, specificity
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on July 11, 2020 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2017-016028 on 10 July 2017. D
ownloaded from
For peer review only
1
Neutrophil Gelatinase-Associated Lipocalin in the Prediction of Acute Kidney Injury
among Adult Intensive Care Unit Patients: Determination of Optimal Thresholds in
Plasma and Urine
Kristen M. Tecson, PhD, Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, A. Osama Gaber,
MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC, Linda
W. Moore, MS, Peter A. McCullough, MD, MPH
Author Affiliations: Baylor Heart and Vascular Institute, Dallas, Texas (Tecson, McCullough);
Baylor Scott & White Research Institute, Dallas, Texas (Tecson); Texas A&M College of
Medicine Health Science Center, Dallas, Texas (Tecson, McCullough); BioPorto Diagnostics
A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of Medicine, Bronx,
NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore); Baystate Medical Center,
Springfield, MA (Germain); Baylor University Medical Center, Dallas, Texas (McCullough); Baylor Jack
and Jane Hamilton Heart and Vascular Hospital, Dallas, Texas (McCullough);
Short Title: Blood and Urine NGAL in Risk Prediction of AKI
Keywords: neutrophil gelatinase associated lipocalin, siderocalin-2, acute kidney injury, risk
prediction, sensitivity, specificity
Word Count: 3175
Corresponding Author:
Peter A. McCullough, MD, MPH
621 N Hall Street, Suite H-030
Dallas, TX 75226
Page 1 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
2
Abstract
Objectives: To determine the optimal threshold of blood and urine Neutrophil gelatinase-
associated lipocalin (NGAL) to predict moderate to severe AKI and persistent moderate to severe
AKI lasting at least 48 consecutive hours, as defined by an adjudication panel.
Methods: A multicenter prospective observational study enrolled intensive care unit patients and
recorded daily ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine
NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict stage
2/3 AKI (defined by Kidney Disease International Global Organization). We performed the same
analysis using the NGAL value at the start of persistent stage 2/3 AKI.
Results: Of 245 subjects, 33 (13.5%) developed stage 2/3 AKI and 25 (10.2%) developed
persistent stage 2/3 AKI. Predicting stage 2/3 AKI revealed the optimal NGAL cutoffs in EDTA
plasma (142.0 ng/ml), heparin plasma (148.3 ng/ml), and urine (78.0 ng/ml) and yielded the
following decision statistics: sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive
predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7%, diagnostic accuracy
(DA)=73.8% (EDTA plasma); SN=72.7%, SP=73.8%, PPV=30.4%, NPV=94.5%, DA=73.7%
(heparin plasma); SN=69.7%, SP=76.8%, PPV=32.9%, NPV=94%, DA=75.8% (urine). The
optimal NGAL cutoffs to predict persistent stage 2/3 AKI were similar: 148.3 ng/ml (EDTA
plasma), 169.6 ng/ml (heparin plasma), and 79.0 ng/ml (urine) yielding: SN=84.0%, SP=73.5%,
PPV=26.6%, NPV=97.6, DA=74.6% (EDTA plasma); SN=84%, SP=76.1%, PPV=26.8%,
NPV=96.5%, DA=76.1% (heparin plasma); SN=75%, SP=75.8%, PPV=26.1, NPV=96.4%,
DA=75.7% (urine).
Page 2 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
3
Conclusion: Blood and urine NGAL predicted stage 2/3 AKI, as well as persistent 2/3 AKI in
the ICU with acceptable decision statistics using a single cut point in each type of specimen.
Abstract word count: 261
Strengths and limitations of this study:
• This study utilized biomarker data from both plasma and urine samples.
• Acute kidney injury diagnosis was adjudicated by an expert panel.
• We used an unbiased, data-driven approach to identify a single cut point for each of plasma and
urinary NGAL samples to predict stage 2 or 3 acute kidney injury and persistent stage 2 or 3 acute
kidney injury.
• This small prospective cohort study had limited clinical follow-up.
Page 3 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
4
Introduction
Serum creatinine and urine output are used to diagnose acute kidney injury (AKI) as
defined by the 2012 Kidney Disease International Global Organization (KDIGO) guidelines
(Supplemental Table 1); however, creatinine is an imperfect marker as it peaks anywhere from 2-
7 days following an insult to the kidneys.1 Additionally, urine output may lead to an inaccurate
diagnosis of AKI in the intensive care unit (ICU), as it is commonly manipulated in the ICU
through use of intravenous fluids and diuretics. Neutrophil gelatinase-associated lipocalin
(NGAL) has been shown to peak following tubular injury earlier than changes in creatinine and
urine output, making it a desirable biomarker for predicting the development of AKI.2 Previous
studies have shown the utility of this biomarker but have suffered from heterogenous AKI
thresholds and lack of validation of parenchymal renal involvement in the injury. In those
studies, AKI may have been due to a transient hemodynamically related reduction in renal
filtration due poor forward perfusion or impaired plasma refill, which represents a distinct AKI
pathophsiology.3, 4
Furthermore, the optimal body fluid for NGAL assay performance (blood or
urine) is unknown, as is the optimal cut point for the detection of moderate to severe AKI
(KDIGO stage 2 or 3). We set out to use a commercially available NGAL assay in a multicenter,
blinded study to assess its performance in two types of plasma samples and in one urine sample
for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by clinicians
blinded to the NGAL values.
Methods
Study Details
This study prospectively enrolled consecutive adult patients admitted to an ICU or critical
care setting after obtaining informed consent between March 5, 2014 and April 15, 2015 from 4
Page 4 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
5
participating hospitals in Boston and Springfield, Massachusetts, Bronx, New York, and
Houston, Texas. Each study site received Institutional Review Board approval prior to enrolling
patients. Those with history of nephrectomy, renal transplantation and/or renal replacement
therapy initiated before admission were not eligible to participate. In addition to the standard
clinical care and laboratory testing, one urine and two plasma samples were drawn and frozen per
day in the ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was
assayed by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range
of 25-3000 ng/ml. 5
At mean NGAL concentrations of 97, 116, and 112 ng/ml in
ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine, respectively, the
coefficients of variation were 3.1%, 1.8%, and 2.8%, respectively. If patients were discharged
early from the ICU during the study period, serum creatinine levels were collected manually from
the hospital data system for 48 hours provided the patient was still hospitalized. Reference
standard AKI was determined daily, up to 8 days, by a three-physician adjudication panel using
the KDIGO® guidelines. AKI was confirmed by the adjudication panel and disagreements were
settled by conference calls and discussion of the cases. Adjudicators were blinded to the NGAL
values and physician notes concerning the renal diagnosis.6
Patient Involvement
Patient involvement began at the time of informed consent; patients and caregivers were not involved
in the design of, recruitment for, or conduct of this study. The development of the outcome measures was
not informed by patients’ priorities, experiences, or preferences. The participants’ contributions to this
study are deeply appreciated and the results of the study will be publicly available to the patients via this
publication.
Page 5 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
6
Analyses
NGAL values were highly skewed in all three sample types so we utilized the natural-log
transformed distributions (Figure 1). We performed logistic regression using the natural log-
transformed baseline NGAL [i.e. ln(baseline NGAL)] value as a predictor for the outcome of
moderate/severe (stage 2/3) AKI at any point during observation. Additionally, we defined
persistent moderate/severe (stage 2/3) AKI as 2 or more days of consecutive stage 2/3 AKI and
we constructed another logistic regression model for the outcome of persistent 2/3 AKI using a
wandering ln(baseline NGAL) value as the predictor variable. The wandering ln(baseline NGAL)
value was taken on the morning of the first day of a 48 hour persistent AKI period. If AKI did
not occur, the wandering ln(baseline NGAL) value was taken to be the ln(baseline NGAL). We
found the optimal NGAL value for both outcomes by using the criterion of smallest distance to
the “perfect point” on the receiver operating characteristic (ROC) curve, (0,1). Baseline NGAL
values < 5 ng/ml were adjusted using a regression model, which affected approximately 2% of
the data. We assessed differences in baseline characteristics between those who had at least 1 day
of stage 2/3 AKI at any time during observation and those who did not via Wilcoxon Rank Sum
and Chi-square tests; we did the same for those who had persistent stage 2/3 AKI and those who
did not have persistent stage 2/3 AKI at any point during observation. Continuous variables are
reported as medians [quarter 1, quarter 3]. Categorical variables are reported as frequencies and
proportions.
Results
A total of 252 patients were screened for the study, 245 of whom were enrolled and
included in analyses. There were 33 (13.5%) subjects who developed stage 2/3 AKI as
determined by the adjudicators. With this event rate, we had 95% power to detect an area under
Page 6 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
7
the curve (AUC) as low as 0.69. Age and gender did not differ significantly between those with
or without stage 2/3 AKI; however, those with stage 2/3 AKI had more comorbidities (sepsis:
18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%, p=0.04; chronic kidney disease: 33.3% v.
10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2 [1, 3] days, p=0.001), and higher baseline
NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v. 98 [70, 154] ng/ml, p<0.001; heparin
plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p < 0.001; urine: 190 [61, 1133] ng/ml v.
31 [15, 67], p < 0.001) than those without AKI (Table 1). Using the ln(baseline EDTA plasma
NGAL) as the independent variable in a logistic regression model for the outcome of stage 2/3
AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87, p<0.0001 (Figure 2). Optimizing the
ROC curve with respect to the distance from (0,1) resulted in a cut point of 142.0 ng/ml, which
yielded a sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive predictive value (PPV) of
31.3%, negative predictive value (NPV) of 95.7% and diagnostic accuracy (DA) of 73.8%.
Similar results are shown for heparin plasma and urine NGAL in Figures 2b and 2c.
Twenty-five (10.2%) subjects had persistent stage 2/3 AKI. Demographics did not differ
significantly between those who did and those who did not have persistent stage 2/3 AKI;
however, those with persistent stage 2/3 AKI had a higher rate of CKD (36.0% v. 10.9%, p <
0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and higher wandering baseline
NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156] ng/ml, p <0.001; heparin:
355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine: 231.1[84.3, 1203] ng/ml v. 32 [15,
75] ng/ml, p<0.001) (Table 2). We utilized the wandering ln(baseline EDTA plasma NGAL) in a
logistic regression model to predict persistent stage 2/3 AKI, which yielded an AUC = 0.85, 95%
CI 0.77-0.94, p<0.001. The optimal cut point for NGAL based on the distance to (0,1) was 148.3
ng/ml (Figure 3). This value yielded SN= 84.0%, SP=73.5%, PPV= 26.6%, NPV= 97.6%,
Page 7 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
8
DA=74.6%. Similar cut points and results are shown for heparin plasma and urine NGAL in
Figures 3b and 3c. Odds ratios from the regression models are shown in Table 3.
Discussion
We found that a single measured threshold for NGAL, either in blood or urine, was
effective in risk prediction for the development of moderate to severe AKI and persistent AKI, as
defined by AKI lasting 48 consecutive hours or longer. NGAL measured on a daily basis
performed well in the prognostication of persistent AKI. Our results were internally consistent
and add to the literature as they yielded unbiased cut points from prospectively collected samples
and adjudicated AKI outcomes. We found that urine and plasma yielded different absolute
NGAL concentrations, however, within each sample type the cut point for the prediction of stage
2/3 AKI or persistent AKI was similar, making daily measurements and utilization of a single
threshold feasible, provided there is consistency in sample source and handling procedures. Our
results are concordant with a prior study that observed slightly better discriminatory power of
plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7
Thus, we believe for ease of
use, plasma NGAL is more likely to be incorporated into routine care than urinary NGAL.
Our findings build upon the work of others using a variety of NGAL assays
demonstrating the concept that 25 kilodalton protein is rapidly and reliably upregulated after
ischemia/reperfusion injury to the kidney.8,9, 4
In human clinical studies, urine NGAL measured
in the emergency department from patients with a variety of presentations was associated with
the development of end-stage renal disease requiring dialysis in the hospital.10
A 2010-2011
study in western Australia, n=102, demonstrated the effectiveness of using NGAL to predict
inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage kidney
disease classification) in a tertiary care setting. In the Australian study, the Biosite Triage®
Page 8 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
9
NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a single
optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-
0.84.11
While the threshold was similar to that found in our EDTA and heparin samples, this
study used all stages of AKI as the endpoint and did not have blinded adjudication by a
committee. Nevertheless, these data are in line with our conclusion that NGAL measured in
blood performs well as a prognostic aid for AKI. A meta-analysis of 1,478 septic patients
demonstrated that NGAL was associated with the development of AKI, need for renal
replacement therapy, and mortality.12
Numerous studies have demonstrated that NGAL rises
after cardiac surgery and is congruent with AKI.13,14
Additionally, NGAL concentrations
increase after contrast-induced AKI; however, it is to a much lesser extent than cardiac surgery
or critical illness.15
Our study is the first among these to demonstrate that NGAL can be used to
risk predict stage 2/3 AKI and its persistence in a cohort of ICU patients. We further
demonstrated a single cutoff value performs well in terms of decision statistics for the
outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.
There have been large prospective studies examining novel biomarkers other than NGAL
for the prediction of stage 2/3 AKI among the critically ill. Kashani and colleagues (n=728)
measured the concentrations of urinary tissue inhibitor of metalloproteinases (TIMP)-2 and
insulin-like growth factor binding protein 7 (IGFBP7), and combined the 2 measures into 1
predictor, referred to as [TIMP-2]*[IGFBP7], NephroCheck® (Astute Medical, San Diego, CA)
to identify imminent stage 2/3 AKI (i.e. in the next 12-hours) which occurred in 14% of study
participants. Baseline urinary [TIMP-2]*[IGFBP7] yielded an AUC = 0.80, 97% CI 0.76-0.83;
however, the full decision statistics of an optimal cut point were not disclosed. 16
A second study
by Bihorac and colleagues (n=420), evaluated urinary [TIMP-2]*[IGFBP7] for the same
Page 9 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
10
endpoint. In this study, 71 (17.4%) patients had immenent stage 2/3 AKI and the model had an
AUC = 0.82, 95% CI 0.76-0.88. However, [TIMP-2]*[IGFBP7] did not appear amenable to a
single threshold value. Two cut points were presented; one yielding high SN and low SP and the
other yielding low SN and high SP. At the cutoff of 0.3 (ng/ml)2/1000 the SN = 92, 95% CI, 85–
98%, but the SP= 46, 95% CI 41–52%. At a 7-fold higher cut point of 2.0 (ng/ml)2/1000 the SN
= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.17
In comparison, an NGAL EDTA plasma
level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI 73.7-
83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower
bounds of the 95% CIs for sensitivity and specificity were well above 60 and 40%, respectively.
Accordingly, NGAL appears to have considerable advantages over [TIMP-2]*[IGFBP7] in
forecasting AKI over a longer time window, at a single prognostic threshold, and avoids low
sensitivity or specificity. Additionally, we demonstrated NGAL performed well when measured
daily using similar cut points for the prediction of persistent stage 2/3 AKI. Daily use in risk
prediction has not been demonstrated in any study with [TIMP2]*[IGFBP7].
Limitations
Our study has all the limitations of small prospective cohort studies of biomarkers
including limited clinical follow-up and lack of precise follow-up information, including accurate
urine output. We did not consider stage I AKI in the definition of the primary endpoint because
prior studies suggested that many in this category could have transient hemodynamic elevations
of creatinine without parenchymal AKI. Thus, our event rate is lower than other studies which
included all KDIGO stages of AKI. Finally, we did not have information on long term outcomes
such as need for renal replacement therapy or mortality which would have been useful in
Page 10 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
11
integrating NGAL with serum creatinine and urine output in an in-depth assessment of patient
outcomes.18
Conclusion
NGAL concentrations of 142.0, 149.5, and 78.0 ng/ml in EDTA plasma, heparin plasma,
and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients.
Similarly, in daily prospective samples, NGAL concentration thresholds at 148.3, 169.6, and 79.2
ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were optimal in predicting
persistent stage 2/3 AKI in the same cohort of ICU patients. These results suggest that
prospective validation of NGAL in one or more sample types in a larger study is warranted.
Page 11 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
12
Funding
The study was funded by BioPorto Diagnostics A/S.
Disclosures
Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees of BioPorto
Diagnostics A/S.
Acknowledgements
The authors would like to thank the study participants who graciously shared their time and materials for the
purposes of this research.
Transparency
The lead author affirms that this manuscript is an honest, accurate, and transparent account of the study
being reported; that no important aspects of the study have been omitted; and that any discrepancies from
the study as planned (and, if relevant, registered) have been explained.
Copyright
“I, Peter A. McCullough, The Corresponding Author of this article contained within the original manuscript
which includes any diagrams & photographs within and any related or stand alone film submitted (the
Contribution”) has the right to grant on behalf of all authors and does grant on behalf of all authors, a licence
to the BMJ Publishing Group Ltd and its licencees, to permit this Contribution (if accepted) to be published
in the BMJ and any other BMJ Group products and to exploit all subsidiary rights, as set out in our licence
set out at: http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-
access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the Contribution.
Contributorship
PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study design and
helped with implementation. KMT conducted the statistical analysis. KMT and PAM drafted the article. All
authors contributed to the manuscript’s refinement and approved the final version.
Data: The data from this study will not be made available.
Page 12 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
13
Figure 1. Mean and standard error of natural log-transformed NGAL by day and acute kidney
injury status (a. EDTA plasma, b. heparin plasma, c. urine)
Figure 2. Receiver operating characteristic curves for natural log-transformed baseline NGAL as
a predictor of stage 2/3 acute kidney injury with the optimal NGAL threshold identified (a.
EDTA plasma, b. heparin plasma, c. urine). Decision statistics are presented for the optimal cut
point (bold) as well as at cut points approximately 20 ng/ml below and above the optimal value.
LR+ indicates positive likelihood ratio and LR- indicated negative likelihood ratio.
Figure 3. Receiver operating characteristic curves for wandering natural log-transformed
baseline NGAL as a predictor of persistent stage 2/3 acute kidney injury with the optimal NGAL
threshold identified (a. EDTA plasma, b. heparin plasma, c. urine). Decision statistics are
presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/ml below
and above the optimal value. LR+ indicates positive likelihood ratio and LR- indicated negative
likelihood ratio.
Page 13 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
14
Table 1. Patient characteristics by occurrence of moderate/severe acute kidney injury (AKI)
Variable
Stage 2 or 3
AKI (n=33)
Stage 1 or No AKI
(n = 212) P-value
Age (years) 68 [56, 74] 63 [54, 73] 0.50
Gender (male) 22 (66.7%) 135 (63.7%) 0.74
Race/Ethnicity 0.86
African American 3 (9.1%) 29 (13.7%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 4 (12.1%) 28 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 26 (78.8%) 151 (71.2%)
Hypertension 26 (78.8%) 142 (67.0%) 0.17
Congestive Heart Failure 6 (18.2%) 38 (17.9%) 0.97
Diabetes Mellitus 0.04
Type I 0 (0.0%) 2 (0.9%)
Type II 16 (48.5%) 57 (26.9%)
None 17 (51.5%) 153 (72.2%)
Urinary Tract Infection 2 (6.1%) 6 (2.8%) 0.33
Sepsis 6 (18.2%) 12 (5.7%) 0.01
Chronic Kidney Disease History <0.001
None 22 (66.7%) 190 (89.6%)
Stage 1 1 (3.0%) 4 (1.9%)
Stage 2 1 (3.0%) 3 (1.4%)
Stage 3A 8 (24.2%) 5 (2.4%)
Stage 3B 0 (0.0%) 8 (3.8%)
Stage 4 1 (3.0%) 2 (0.9%)
Baseline EDTA plasma NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001
Baseline heparin plasma NGAL (ng/ml) 288 [150, 669] 100 [68, 160] <0.001
Baseline urine NGAL (ng/ml) 190 [61, 1133] 31 [15, 67] <0.001
Days in intensive care unit 4 [2, 8] 2 [1,3] 0.001
Page 14 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
15
Table 2. Sample characteristics for subjects with and without persistent stage 2/3 acute kidney
injury
Variable Persistent stage 2/3
AKI (n=25)
No persistent
stage 2/3 AKI
(n=220)
p-value
Age (years) 68 [60, 73] 63 [54, 73] 0.31
Gender (male) 16 (64.0%) 141 (64.1%) 0.99
Race/Ethnicity
0.89
African American 2 (8.0%) 30 (13.6%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 3 (12.0%) 29 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 20 (80.0%) 157 (71.4%)
Hypertension 19 (76.0%) 149 (67.7%) 0.40
Congestive Heart Failure 4 (16.0%) 40 (18.2%) 0.79
Diabetes Mellitus
0.40
Type I 0 (0.0%) 2 (0.9%)
Type II 10 (40.0%) 63 (28.6%)
None 15 (60.0%) 155 (70.5%)
Urinary Tract Infection 2 (8.0%) 6 (2.73%) 0.19
Sepsis 4 (16.0%) 14 (6.4%) 0.10
Chronic Kidney Disease History
<0.001
None 16 (64.0%) 196 (89.1%)
Stage 1 1 (4.0%) 4 (1.8%)
Stage 2 1 (4.0%) 3 (1.4%)
Stage 3A 6 (24.0%) 7 (3.2%)
Stage 3B 0 (00%) 8 (3.6%)
Stage 4 1 (94.0%) 2 (0.9%)
Wandering baseline EDTA plasma NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001
Wandering baseline heparin plasma NGAL (ng/ml) 355 [178, 716] 101 [68, 162] <0.001
Wandering baseline urine NGAL (ng/ml) 231.1 [84.3, 1203] 32 [15, 75] <0.001
Days in intensive care unit 6 [2, 8] 2 [1,3] <0.001
Page 15 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
16
Table 3. Odds ratios for stage 2/3 acute kidney injury and persistent stage 2/3 acute kidney injury
using EDTA, heparin, and urine NGAL values as univariate predictors (odds are associated with
an approximate 2.73-fold increase in NGAL value)
Outcome NGAL Measure P-Value Odds Ratio (95% CI)
Stage 2/3 acute
kidney injury
EDTA plasma <0.001 2.8 (1.8, 4.4)
Heparin plasma <0.001 3.2 (2.0, 4.9)
Urine <0.001 2.2 (1.7, 2.9)
Persistent stage 2/3
acute kidney injury
EDTA plasma <0.001 3.8 (2.3, 6.3)
Heparin plasma <0.001 3.9 (2.3, 6.3)
Urine <0.001 2.2 (1.6, 3.0)
Page 16 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
17
Supplemental Table 1. Kidney Disease International Global Organization 2012 definitions of
acute kidney injury stages
Stage Serum Creatinine Urine output
1 1.5-1.9 times baseline OR ≥ 0.3 mg/dl
(≥26.5 µmol/l) increase
< 0.5 ml/hg/h for 6-12 hours
2 2.0-2.9 times baseline < 0.5 ml/kg/h ≥ 12 hours
3 3.0 times baseline OR increase in serum
creatinine to ≥ 4.0 mg/dl (≥ 353.6 µmol/l)
OR initiation of renal replacement
therapy OR in patients < 18 years,
decrease in eGFR to < 35 ml/min per
1.73 m2
< 0.3 ml/kg/h ≥ 24 hours OR Anuria for
≥ 12 hours
Page 17 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
18
References
1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1).
2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta
RL, Ronco C. Diagnosis of acute kidney injury using functional and injury biomarkers:
workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference.
Contrib Nephrol. 2013;182:13-29. doi: 10.1159/000349963. PubMed PMID: 23689653.
3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang
JT, Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute
Kidney Injury in Coronary Care Units. Medicine (Baltimore). 2015
Oct;94(40):e1703. doi: 10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed
Central PMCID: PMC4616771.
4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster
C, Deng R, Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J,
D'Agati V, Erdjument-Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N,
Tempst P, Oliver JA, Guarnieri P, Barasch J. Unique Transcriptional Programs
Identify Subtypes of AKI. J Am Soc Nephrol. 2016 Dec 27. pii: ASN.2016090974.
doi: 10.1681/ASN.2016090974. [Epub ahead of print] PubMed PMID: 28028135.
5 The NGAL Test
TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013
6 U.S. Department of Health and Human Services, Food and Drug Administration, Center for
Devices and Radiological Health. 2007 March 13. Guidance for Industry and FDA Staff:
Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
Washington, DC: Author.
7 Schley G, Köberle C, Manuilova E, Rutz S, Forster C, Weyand M, et al. (2015) Comparison of
Plasma and Urine Biomarker Performance in Acute Kidney Injury. PLoS ONE 10(12):
e0145042. doi:10.1371/journal.pone.0145042
8 Clerico A, Galli C, Fortunato A, Ronco C. Neutrophil gelatinase-associated lipocalin (NGAL)
as biomarker of acute kidney injury: a review of the laboratory characteristics and clinical
evidences. Clin Chem Lab Med. 2012 Feb
15;50(9):1505-17. doi: 10.1515/cclm-2011-0814. Review. PubMed PMID: 22962216.
9 Devarajan P. Neutrophil gelatinase-associated lipocalin: a promising biomarker for human
acute kidney injury. Biomarkers in medicine. 2010;4(2):265-280. doi:10.2217/bmm.10.12.
10 Nickolas TL, O'Rourke MJ, Yang J, Sise ME, Canetta PA, Barasch N, Buchen C, Khan F,
Mori K, Giglio J, Devarajan P, Barasch J. Sensitivity and specificity of a single emergency
department measurement of urinary neutrophil gelatinase-associated lipocalin for diagnosing
Page 18 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
19
acute kidney injury. Ann Intern Med. 2008 Jun 3;148(11):810-9. PubMed PMID: 18519927;
PubMed Central PMCID: PMC2909852.
11
Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin (NGAL)
predicts renal injury in acute decompensated cardiac failure: a prospective observational study. BMC
Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.
12
Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,
Yin YH. Diagnosis and prognosis of neutrophil gelatinase-associated lipocalin for acute kidney
injury with sepsis: a systematic review and meta-analysis. Crit Care. 2016 Feb 16;20:41. doi:
10.1186/s13054-016-1212-x. PubMed PMID: 26880194; PubMed Central PMCID:
PMC4754917.
13
Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,
McCullough PA. Urine catalytic iron and neutrophil gelatinase-associated lipocalin as
companion early markers of acute kidney injury after cardiac surgery: a prospective pilot study.
Cardiorenal Med. 2013 Apr;3(1):7-16. doi:
10.1159/000346815. PubMed PMID: 23946721; PubMed Central PMCID: PMC3743453.
14 Haase-Fielitz A, Haase M, Devarajan P. Neutrophil gelatinase-associated lipocalin as a
biomarker of acute kidney injury: a critical evaluation of current status. Annals of clinical
biochemistry. 2014;51(0 3):335-351. doi:10.1177/0004563214521795.
15 Filiopoulos V, Biblaki D, Vlassopoulos D. Neutrophil gelatinase-associated lipocalin (NGAL):
a promising biomarker of contrast-induced nephropathy after computed tomography. Ren Fail.
2014 Jul;36(6):979-86. doi: 10.3109/0886022X.2014.900429. Review. PubMed PMID:
24673459.
16 Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A,
Birkhahn R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson
KJ, Haase M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis
M, Kim P, Koyner JL, Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney
S, Ostermann M, Rimmelé T, Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL,
Vinsonneau C, Wagner L, Walker MG, Wilkerson RG, Zacharowski K, Kellum JA.
Discovery and validation of cell cycle arrest biomarkers in human acute kidney injury.
Crit Care. 2013 Feb 6;17(1):R25. doi: 10.1186/cc12503. PubMed PMID: 23388612;
PubMed Central PMCID: PMC4057242.
17 Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald
R, Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL,
Krell K, Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH,
Sellman R, Shi J, Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J,
Wunderink R, Zimmerman J, Kellum JA. Validation of cell-cycle arrest biomarkers for
acute kidney injury using clinical adjudication. Am J Respir Crit Care Med.
Page 19 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
20
2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-0077OC. PubMed PMID: 24559465.
18 McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner
JL, Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C,
Okusa MD, Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M,
Murray PT, Cruz D, Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C.
Implementation of novel biomarkers in the diagnosis, prognosis, and management of
acute kidney injury: executive summary from the tenth consensus conference of the
Acute Dialysis Quality Initiative (ADQI). Contrib Nephrol. 2013;182:5-12. doi:
10.1159/000349962. PubMed PMID: 23689652; PubMed Central PMCID: PMC3856225.
Page 20 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1a
338x190mm (300 x 300 DPI)
Page 21 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1b
338x190mm (300 x 300 DPI)
Page 22 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1c
338x190mm (300 x 300 DPI)
Page 23 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2a
338x190mm (300 x 300 DPI)
Page 24 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2b
338x190mm (300 x 300 DPI)
Page 25 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2c
338x190mm (300 x 300 DPI)
Page 26 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3a
338x190mm (300 x 300 DPI)
Page 27 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3b
338x190mm (300 x 300 DPI)
Page 28 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3c
338x190mm (300 x 300 DPI)
Page 29 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for the Prediction of Acute
Kidney Injury among Critically Ill Adults: Retrospective Determination and Clinical Validation of a Prospective
Multicenter Study
Journal: BMJ Open
Manuscript ID bmjopen-2017-016028.R1
Article Type: Research
Date Submitted by the Author: 16-Apr-2017
Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital
Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute
<b>Primary Subject Heading</b>:
Renal medicine
Secondary Subject Heading: Diagnostics
Keywords: neutrophil gelatinase associated lipocalin, acute kidney injury, risk prediction, sensitivity, specificity
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on July 11, 2020 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2017-016028 on 10 July 2017. D
ownloaded from
For peer review only
1
Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for
the Prediction of Acute Kidney Injury among Critically Ill Adults: Retrospective
Determination and Clinical Validation of a Prospective Multicenter Study
Kristen M. Tecson, PhD, Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, A. Osama Gaber,
MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC,
Linda W. Moore, MS, Peter A. McCullough, MD, MPH
Author Affiliations: Baylor Heart and Vascular Institute, Dallas, Texas (Tecson, McCullough);
Baylor Scott & White Research Institute, Dallas, Texas (Tecson); Texas A&M College of
Medicine Health Science Center, Dallas, Texas (Tecson, McCullough); BioPorto Diagnostics
A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of
Medicine, Bronx, NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore);
Baystate Medical Center, Springfield, MA (Germain); Baylor University Medical Center,
Dallas, Texas (McCullough); Baylor Jack and Jane Hamilton Heart and Vascular Hospital,
Dallas, Texas (McCullough)
Short Title: Blood and Urine NGAL in Risk Prediction of AKI
Keywords: neutrophil gelatinase associated lipocalin, siderocalin-2, acute kidney injury, risk
prediction, sensitivity, specificity
Word Count: 3661
Corresponding Author:
Peter A. McCullough, MD, MPH
621 N Hall Street, Suite H-030
Dallas, TX 75226
Page 1 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
2
Abstract
Objectives: To determine the optimal threshold of blood and urine Neutrophil gelatinase-
associated lipocalin (NGAL) to predict moderate to severe AKI and persistent moderate to severe
AKI lasting at least 48 consecutive hours, as defined by an adjudication panel.
Methods: A multicenter prospective observational study enrolled intensive care unit patients and
recorded daily ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine
NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict
stage 2/3 AKI (defined by Kidney Disease International Global Organization). We performed the
same analysis using the NGAL value at the start of persistent stage 2/3 AKI.
Results: Of 245 subjects, 33 (13.5%) developed stage 2/3 AKI and 25 (10.2%) developed
persistent stage 2/3 AKI. Predicting stage 2/3 AKI revealed the optimal NGAL cutoffs in EDTA
plasma (142.0 ng/ml), heparin plasma (148.3 ng/ml), and urine (78.0 ng/ml) and yielded the
following decision statistics: sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive
predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7%, diagnostic accuracy
(DA)=73.8% (EDTA plasma); SN=72.7%, SP=73.8%, PPV=30.4%, NPV=94.5%, DA=73.7%
(heparin plasma); SN=69.7%, SP=76.8%, PPV=32.9%, NPV=94%, DA=75.8% (urine). The
optimal NGAL cutoffs to predict persistent stage 2/3 AKI were similar: 148.3 ng/ml (EDTA
plasma), 169.6 ng/ml (heparin plasma), and 79.0 ng/ml (urine) yielding: SN=84.0%, SP=73.5%,
PPV=26.6%, NPV=97.6, DA=74.6% (EDTA plasma); SN=84%, SP=76.1%, PPV=26.8%,
NPV=96.5%, DA=76.1% (heparin plasma); SN=75%, SP=75.8%, PPV=26.1, NPV=96.4%,
DA=75.7% (urine).
Page 2 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
3
Conclusion: Blood and urine NGAL predicted stage 2/3 AKI, as well as persistent 2/3 AKI in
the ICU with acceptable decision statistics using a single cut point in each type of specimen.
Abstract word count: 261
Strengths and limitations of this study:
• This study utilized biomarker data from both plasma and urine samples.
• Acute kidney injury diagnosis was adjudicated by an expert panel.
• We used an unbiased, data-driven approach to identify a single cut point for each of
plasma and urinary NGAL samples to predict stage 2 or 3 acute kidney injury and
persistent stage 2 or 3 acute kidney injury.
• This small prospective cohort study had limited clinical follow-up.
Page 3 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
4
Introduction
Serum creatinine and urine output are used to diagnose acute kidney injury (AKI) as
defined by the 2012 Kidney Disease International Global Organization (KDIGO) guidelines
(Supplemental Table 1); however, creatinine is an imperfect marker as it peaks anywhere from 2-
7 days following an insult to the kidneys.1 Additionally, urine output may lead to an inaccurate
diagnosis of AKI in the intensive care unit (ICU), as it is commonly manipulated in the ICU
through use of intravenous fluids and diuretics. Neutrophil gelatinase-associated lipocalin
(NGAL) has been shown to peak following tubular injury earlier than changes in creatinine and
urine output, making it a desirable biomarker for predicting the development of AKI.2 Previous
studies have shown the utility of this biomarker but have suffered from heterogeneous AKI
thresholds and lack of validation of parenchymal renal involvement in the injury. In those
studies, AKI may have been due to a transient hemodynamically related reduction in renal
filtration due poor forward perfusion or impaired plasma refill, which represents a distinct AKI
pathophysiology.3, 4
Furthermore, the optimal body fluid for NGAL assay performance (blood or
urine) is unknown, as is the optimal cut point for the detection of moderate to severe AKI
(KDIGO stage 2 or 3). We set out to use a commercially available NGAL assay in a multicenter,
blinded study to assess its performance in two types of plasma samples and in one urine sample
for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by
clinicians blinded to the NGAL values.
Methods
Study Details
This study prospectively enrolled consecutive adult patients admitted to an ICU or critical
care setting after obtaining informed consent between March 5, 2014 and April 15, 2015 from 4
Page 4 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
5
participating hospitals in Boston and Springfield, Massachusetts, Bronx, New York, and
Houston, Texas; the protocol may be found at the following:
http://www.bioporto.com/Products/Featured-areas/NGAL.aspx. Each study site received
Institutional Review Board (IRB) approval prior to enrolling patients (IRBs utilized for this study
are as follows: Baystate Health IRB #1; Houston Methodist Research Institute (HMRI) IRB 1;
Pratners Human Research Committee; Biomedical Research Alliance of New York (BRANY)
IRB). Those with history of nephrectomy, renal transplantation and/or renal replacement therapy
initiated before admission were not eligible to participate. In addition to the standard clinical care
and laboratory testing, one urine and two plasma samples were drawn and frozen per day in the
ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was assayed
by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range of 25-
3000 ng/ml. 5
At mean NGAL concentrations of 97, 116, and 112 ng/ml in
ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine, respectively, the
coefficients of variation were 3.1%, 1.8%, and 2.8%, respectively. If patients were discharged
early from the ICU during the study period, serum creatinine levels were collected manually
from the hospital data system for 48 hours provided the patient was still hospitalized. Reference
standard AKI was determined daily, up to 8 days, by a three-physician adjudication panel using
the KDIGO® guidelines. AKI was confirmed by the adjudication panel and disagreements were
settled by conference calls and discussion of the cases. Adjudicators were blinded to the NGAL
values and physician notes concerning the renal diagnosis.6
Patient Involvement
Patient involvement began at the time of informed consent; patients and caregivers were
not involved in the design of, recruitment for, or conduct of this study. The development of the
Page 5 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
6
outcome measures was not informed by patients’ priorities, experiences, or preferences. The
participants’ contributions to this study are deeply appreciated and the results of the study will
be publicly available to the patients via this publication.
Analyses
This manuscript was constructed via retrospective analyses of prospectively collected
data and was developed under STARD guidelines. NGAL values were highly skewed in all three
sample types so we utilized the natural-log transformed distributions (Figure 1). We performed
logistic regression using the natural log-transformed baseline NGAL [i.e. ln(baseline NGAL)]
value as a predictor for the outcome of moderate/severe (stage 2/3) AKI at any point during
observation. Additionally, we defined persistent moderate/severe (stage 2/3) AKI as 2 or more
days of consecutive stage 2/3 AKI and we constructed another logistic regression model for the
outcome of persistent 2/3 AKI using a wandering ln(baseline NGAL) value as the predictor
variable. The wandering ln(baseline NGAL) value was taken on the morning of the first day of a
48 hour persistent AKI period. If AKI did not occur, the wandering ln(baseline NGAL) value
was taken to be the ln(baseline NGAL). We found the optimal NGAL value for both outcomes
by using the criterion of smallest distance to the “perfect point” on the receiver operating
characteristic (ROC) curve, (0,1). Baseline NGAL values < 5 ng/ml were adjusted using a
regression model, which affected approximately 2% of the data. We assessed differences in
baseline characteristics between those who had at least 1 day of stage 2/3 AKI at any time during
observation and those who did not via Wilcoxon Rank Sum and Chi-square tests; we did the
same for those who had persistent stage 2/3 AKI and those who did not have persistent stage 2/3
AKI at any point during observation. Continuous variables are reported as medians [quarter 1,
quarter 3]. Categorical variables are reported as frequencies and proportions.
Page 6 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
7
Results
A total of 252 patients were screened for the study; 2 were screen failures, and 5 did not
have sufficient information to be included in analyses. Hence, 245 patients were enrolled and
included in analyses. Only 2 patients in this study had cardiac surgery during the observational
period. No adverse events occurred during the observation period. There were 33 (13.5%)
subjects who developed stage 2/3 AKI as determined by the adjudicators. With this event rate,
we had 95% power to detect an area under the curve (AUC) as low as 0.69. Age and gender did
not differ significantly between those with or without stage 2/3 AKI; however, those with stage
2/3 AKI had more comorbidities (sepsis: 18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%,
p=0.04; chronic kidney disease: 33.3% v. 10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2
[1, 3] days, p=0.001), and higher baseline NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v.
98 [70, 154] ng/ml, p<0.001; heparin plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p <
0.001; urine: 190 [61, 1133] ng/ml v. 31 [15, 67], p < 0.001) than those without AKI (Table 1).
Using the ln(baseline EDTA plasma NGAL) as the independent variable in a logistic regression
model for the outcome of stage 2/3 AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87,
p<0.0001 (Figure 2). Optimizing the ROC curve with respect to the distance from (0,1) resulted
in a cut point of 142.0 ng/ml, which yielded a sensitivity (SN) of 78.8%, specificity (SP) of
73.0%, positive predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7% and
diagnostic accuracy (DA) of 73.8%. Similar results are shown for heparin plasma and urine
NGAL in Figures 2b and 2c.
Twenty-five (10.2%) subjects had persistent stage 2/3 AKI. Demographics did not differ
significantly between those who did and those who did not have persistent stage 2/3 AKI;
however, those with persistent stage 2/3 AKI had a higher rate of chronic kidney disease (CKD)
Page 7 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
8
(36.0% v. 10.9%, p < 0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and
higher wandering baseline NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156]
ng/ml, p <0.001; heparin: 355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine:
231.1[84.3, 1203] ng/ml v. 32 [15, 75] ng/ml, p<0.001) (Table 2). We utilized the wandering
ln(baseline EDTA plasma NGAL) in a logistic regression model to predict persistent stage 2/3
AKI, which yielded an AUC = 0.85, 95% CI 0.77-0.94, p<0.001. The optimal cut point for
NGAL based on the distance to (0,1) was 148.3 ng/ml (Figure 3). This value yielded SN= 84.0%,
SP=73.5%, PPV= 26.6%, NPV= 97.6%, DA=74.6%. Similar cut points and results are shown for
heparin plasma and urine NGAL in Figures 3b and 3c. Odds ratios from the regression models
are shown in Table 3.
Discussion
We found that a single measured threshold for NGAL, either in blood or urine, was
effective in risk prediction for the development of moderate to severe AKI and persistent AKI, as
defined by AKI lasting 48 consecutive hours or longer. NGAL measured on a daily basis
performed well in the prognostication of persistent AKI. Our results were internally consistent
and add to the literature as they yielded unbiased cut points from prospectively collected samples
and adjudicated AKI outcomes. We found that urine and plasma yielded different absolute
NGAL concentrations, however, within each sample type the cut point for the prediction of stage
2/3 AKI or persistent AKI was similar, making daily measurements and utilization of a single
threshold feasible, provided there is consistency in sample source and handling procedures. Our
results are concordant with a prior study that observed slightly better discriminatory power of
plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7
Thus, we believe for ease of
use, plasma NGAL is more likely to be incorporated into routine care than urinary NGAL.
Page 8 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
9
Our findings build upon the work of others using a variety of NGAL assays
demonstrating the concept that 25 kilodalton protein is rapidly and reliably upregulated after
ischemia/reperfusion injury to the kidney.8,9, 4
The work by Nikolas et al. revealed that NGAL
even has the ability to distinguish AKI from non-progressive CKD, a feat that creatinine has
failed to accomplish.10
Further, the urine NGAL measured in the emergency department was
associated with the development of end-stage renal disease requiring dialysis in the hospital. A
2010-2011 study in western Australia, n=102, demonstrated the effectiveness of using NGAL to
predict inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage
kidney disease classification) in a tertiary care setting. In the Australian study, the Biosite
Triage® NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a
single optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-
0.84.11
While the threshold was similar to that found in our EDTA and heparin samples, this
study used all stages of AKI as the endpoint and did not have blinded adjudication by a
committee. Nevertheless, these data are in line with our conclusion that NGAL measured in
blood performs well as a prognostic aid for AKI. A meta-analysis of 1,478 septic patients
demonstrated that NGAL was associated with the development of AKI, need for renal
replacement therapy, and mortality.12
Marino et al.’s study of 101 emergency department
patients with sepsis confirmed that NGAL was related to (the severity of) AKI; however, NGAL
concentrations were also elevated in the absence of AKI in these patients.13
Numerous studies
have demonstrated that NGAL rises after cardiac surgery and is congruent with AKI.14,15
Additionally, NGAL concentrations increase after contrast-induced AKI; however, it is to a
much lesser extent than from cardiac surgery or critical illness.16
An extensive literature review
and meta-analysis by Haase et al. revealed a wide range of optimal reported NGAL cutoffs for
Page 9 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
10
AKI prediction (100 – 270 ng/ml).17
The optimal plasma cutoffs found in our study to detect
stage 2/3 AKI (EDTA: 142.0 ng/ml, heparin:149.5 ng/ml) were within the range reported in the
meta-analysis. Haase suggested cutoffs between 150-170 ng/ml for adults, which is slightly
higher than our findings, and may be attributable to the high rate of postoperative patients used
in the meta-analysis, as well as the varied definitions of AKI. Our study is the first among these
to demonstrate that NGAL can be used to risk predict stage 2/3 AKI and its persistence in a
cohort of ICU patients. We further demonstrated a single cutoff value performs well in terms of
decision statistics for the outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.
There have been large prospective studies examining novel biomarkers other than NGAL
for the prediction of stage 2/3 AKI among the critically ill. Kashani and colleagues (n=728)
measured the concentrations of urinary tissue inhibitor of metalloproteinases (TIMP)-2 and
insulin-like growth factor binding protein 7 (IGFBP7), and combined the 2 measures into 1
predictor, referred to as [TIMP-2]*[IGFBP7], NephroCheck® (Astute Medical, San Diego, CA)
to identify imminent stage 2/3 AKI (i.e. in the next 12-hours) which occurred in 14% of study
participants. Baseline urinary [TIMP-2]*[IGFBP7] yielded an AUC = 0.80, 97% CI 0.76-0.83;
however, the full decision statistics of an optimal cut point were not disclosed. 18
A second study
by Bihorac and colleagues (n=420), evaluated urinary [TIMP-2]*[IGFBP7] for the same
endpoint. In this study, 71 (17.4%) patients had imminent stage 2/3 AKI and the model had an
AUC = 0.82, 95% CI 0.76-0.88. However, [TIMP-2]*[IGFBP7] did not appear amenable to a
single threshold value. Two cut points were presented; one yielding high SN and low SP and the
other yielding low SN and high SP. At the cutoff of 0.3 (ng/ml)2/1000 the SN = 92, 95% CI, 85–
98%, but the SP= 46, 95% CI 41–52%. At a 7-fold higher cut point of 2.0 (ng/ml)2/1000 the SN
= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.19
In comparison, an NGAL EDTA
Page 10 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
11
plasma level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI
73.7-83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower
bounds of the 95% CIs for sensitivity and specificity were well above 60 and 40%, respectively.
Accordingly, NGAL appears to have considerable advantages over [TIMP-2]*[IGFBP7] in
forecasting AKI over a longer time window, at a single prognostic threshold, and avoids low
sensitivity or specificity. Additionally, we demonstrated NGAL performed well when measured
daily using similar cut points for the prediction of persistent stage 2/3 AKI. Daily use in risk
prediction has not been demonstrated in any study with [TIMP2]*[IGFBP7].
Limitations
Our study has all the limitations of small prospective cohort studies of biomarkers
including limited clinical follow-up and lack of precise follow-up information, including accurate
urine output. We did not consider stage I AKI in the definition of the primary endpoint because
prior studies suggested that many in this category could have transient hemodynamic elevations
of creatinine without parenchymal AKI. Thus, our event rate is lower than other studies which
included all KDIGO stages of AKI. Our study sample was non-homogeneous in that it included
patients with sepsis, a diagnosis which has been shown to increase NGAL levels even for those
without AKI.13
Further, due to the small sample size, we could not analyze sepsis or CKD
subjects separately, rendering it impossible to determine whether or not different cutoffs were
needed. To examine confounders’ relations with (persistent) stage 2/3 AKI, we built multivariate
models; however, neither sepsis, nor CKD, nor DM were statistically significant when
considered with NGAL simultaneously. Consequently, the adjusted odds ratio estimates changed
only slightly, and the corresponding p-values were unchanged. Perhaps this is due, in part, to the
fact that NGAL can distinguish AKI from non-progressive CKD. Additionally, the small sample
Page 11 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
12
size did not permit the use of training and/or validation sets, which prohibited further
investigation of the models. However, all AUCs were at least 0.76, which indicates good
predictive ability. Finally, we did not have information on long term outcomes such as need for
renal replacement therapy or mortality which would have been useful in integrating NGAL with
serum creatinine and urine output in an in-depth assessment of patient outcomes.20
Conclusion
NGAL concentrations of 142.0, 149.5, and 78.0 ng/ml in EDTA plasma, heparin plasma,
and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients
participating in a prospective observational study. Similarly, NGAL concentration thresholds of
148.3, 169.6, and 79.2 ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were
optimal in predicting persistent stage 2/3 AKI in the same cohort of ICU patients. A larger
prospective study to provide external validation of these cut points is warranted.
Page 12 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
13
Funding
The study was funded by BioPorto Diagnostics A/S.
Disclosures
Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees
of BioPorto Diagnostics A/S. The remaining authors have nothing to disclose.
Acknowledgements
The authors would like to thank the study participants who graciously shared their time and
materials for the purposes of this research.
Transparency
The lead author affirms that this manuscript is an honest, accurate, and transparent account of the
study being reported; that no important aspects of the study have been omitted; and that any
discrepancies from the study as planned (and, if relevant, registered) have been explained.
Copyright
“I, Peter A. McCullough, The Corresponding Author of this article contained within the original
manuscript which includes any diagrams & photographs within and any related or stand alone
film submitted (the Contribution”) has the right to grant on behalf of all authors and does grant
on behalf of all authors, a licence to the BMJ Publishing Group Ltd and its licencees, to permit
this Contribution (if accepted) to be published in the BMJ and any other BMJ Group products
and to exploit all subsidiary rights, as set out in our licence set out at:
http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-
access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the
Contribution.
Contributorship
PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study
design and helped with implementation. KMT conducted the statistical analysis. KMT and PAM
drafted the article. All authors contributed to the manuscript’s refinement and approved the final
version.
Data: The data from this study will not be made available.
Page 13 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
14
Figure 1. Mean and standard error of natural log-transformed NGAL by day and acute kidney
injury status (a. EDTA plasma, b. heparin plasma, c. urine)
Figure 2. Receiver operating characteristic curves for natural log-transformed baseline NGAL as
a predictor of stage 2/3 acute kidney injury with the optimal NGAL threshold identified (a.
EDTA plasma, b. heparin plasma, c. urine). Decision statistics are presented for the optimal cut
point (bold) as well as at cut points approximately 20 ng/ml below and above the optimal value.
LR+ indicates positive likelihood ratio and LR- indicated negative likelihood ratio.
Figure 3. Receiver operating characteristic curves for wandering natural log-transformed
baseline NGAL as a predictor of persistent stage 2/3 acute kidney injury with the optimal NGAL
threshold identified (a. EDTA plasma, b. heparin plasma, c. urine). Decision statistics are
presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/ml below
and above the optimal value. LR+ indicates positive likelihood ratio and LR- indicated negative
likelihood ratio.
Page 14 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
15
Table 1. Patient characteristics by occurrence of moderate/severe acute kidney injury (AKI)
Variable Stage 2 or 3 AKI
(n=33)
Stage 1 or No AKI
(n = 212) P-value
Age (years) 68 [56, 74] 63 [54, 73] 0.5
Gender (male) 22 (66.7%) 135 (63.7%) 0.74
Race/Ethnicity 0.86
African American 3 (9.1%) 29 (13.7%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 4 (12.1%) 28 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 26 (78.8%) 151 (71.2%)
Hypertension 26 (78.8%) 142 (67.0%) 0.17
Congestive Heart Failure 6 (18.2%) 38 (17.9%) 0.97
Diabetes Mellitus 0.04
Type I 0 (0.0%) 2 (0.9%)
Type II 16 (48.5%) 57 (26.9%)
None 17 (51.5%) 153 (72.2%)
Urinary Tract Infection 2 (6.1%) 6 (2.8%) 0.33
Sepsis 6 (18.2%) 12 (5.7%) 0.01
Chronic Kidney Disease History <0.001
None 22 (66.7%) 190 (89.6%)
Stage 1 1 (3.0%) 4 (1.9%)
Stage 2 1 (3.0%) 3 (1.4%)
Stage 3A 8 (24.2%) 5 (2.4%)
Stage 3B 0 (0.0%) 8 (3.8%)
Stage 4 1 (3.0%) 2 (0.9%)
In-hospital renal replacement therapy 4 (12.1%) 2 (0.9%) 0.003
Nephrotoxin use at baseline 18 (54.6%) 88 (41.5%) 0.19
Baseline EDTA NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001
Baseline heparin plasma NGAL (ng/ml) 288 [150, 669] 100 [68, 160] <0.001
Baseline urine NGAL (ng/ml) 190 [61, 1133] 31 [15, 67] <0.001
Days in intensive care unit 4 [2, 8] 2 [1,3] 0.001
Table 2. Sample characteristics for subjects with and without persistent stage 2/3 acute kidney
Page 15 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
16
injury
Variable Persistent AKI
(n=25)
No persistent
AKI (n=220) P-value
Age (years) 68 [60, 73] 63 [54, 73] 0.31
Gender (male) 16 (64.0%) 141 (64.1%) 0.99
Race/Ethnicity 0.89
African American 2 (8.0%) 30 (13.6%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 3 (12.0%) 29 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 20 (80.0%) 157 (71.4%)
Hypertension 19 (76.0%) 149 (67.7%) 0.4
Congestive Heart Failure 4 (16.0%) 40 (18.2%) 0.79
Diabetes Mellitus 0.4
Type I 0 (0.0%) 2 (0.9%)
Type II 10 (40.0%) 63 (28.6%)
None 15 (60.0%) 155 (70.5%)
Urinary Tract Infection 2 (8.0%) 6 (2.73%) 0.19
Sepsis 4 (16.0%) 14 (6.4%) 0.1
Chronic Kidney Disease History <0.001
None 16 (64.0%) 196 (89.1%)
Stage 1 1 (4.0%) 4 (1.8%)
Stage 2 1 (4.0%) 3 (1.4%)
Stage 3A 6 (24.0%) 7 (3.2%)
Stage 3B 0 (00%) 8 (3.6%)
Stage 4 1 (94.0%) 2 (0.9%)
In-hospital renal replacement therapy 4 (16.0%) 2 (0.9%) 0.001
Use of nephrotoxins 14 (56.05) 92 (41.8%) 0.18
Wandering baseline EDTA NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001
Wandering baseline heparin plasma NGAL
(ng/ml) 355 [178, 716] 101 [68, 162] <0.001
Wandering baseline urine NGAL (ng/ml) 231.1 [84.3, 1203] 32 [15, 75] <0.001
Days in intensive care unit 6 [2, 8] 2 [1,3] <0.001
Page 16 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
17
Table 3. Odds ratios for stage 2/3 acute kidney injury and persistent stage 2/3 acute kidney injury
using EDTA, heparin, and urine NGAL values as predictors (odds are associated with an
approximate 2.73-fold increase in NGAL value)
Outcome NGAL Measure P-Value Odds Ratio (95% CI)
Stage 2/3 acute
kidney injury
EDTA <0.001 2.8 (1.8, 4.4)
Heparin <0.001 3.2 (2.0, 4.9)
Urine <0.001 2.2 (1.7, 2.9)
Persistent stage 2/3
acute kidney injury
EDTA <0.001 3.8 (2.3, 6.3)
Heparin <0.001 3.9 (2.3, 6.3)
Urine <0.001 2.2 (1.6, 3.0)
Page 17 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
18
Supplemental Table 1. Kidney Disease International Global Organization 2012 definitions of
acute kidney injury stages
Stage Serum Creatinine Urine output
1 1.5-1.9 times baseline OR ≥ 0.3 mg/dl
(≥26.5 µmol/l) increase
< 0.5 ml/hg/h for 6-12 hours
2 2.0-2.9 times baseline < 0.5 ml/kg/h ≥ 12 hours
3 3.0 times baseline OR increase in serum
creatinine to ≥ 4.0 mg/dl (≥ 353.6 µmol/l)
OR initiation of renal replacement
therapy OR in patients < 18 years,
decrease in eGFR to < 35 ml/min per
1.73 m2
< 0.3 ml/kg/h ≥ 24 hours OR Anuria for
≥ 12 hours
Page 18 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
19
References
1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1).
2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta
RL, Ronco C. Diagnosis of acute kidney injury using functional and injury biomarkers:
workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference.
Contrib Nephrol. 2013;182:13-29. doi: 10.1159/000349963. PubMed PMID: 23689653.
3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang JT,
Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute Kidney
Injury in Coronary Care Units. Medicine (Baltimore). 2015 Oct;94(40):e1703. doi:
10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed Central PMCID:
PMC4616771.
4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster C, Deng R,
Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J, D'Agati V, Erdjument-
Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N, Tempst P, Oliver JA, Guarnieri
P, Barasch J. Unique Transcriptional Programs Identify Subtypes of AKI. J Am Soc Nephrol.
2016 Dec 27. pii: ASN.2016090974. doi: 10.1681/ASN.2016090974. [Epub ahead of print]
PubMed PMID: 28028135.
5 The NGAL Test
TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013.
6 U.S. Department of Health and Human Services, Food and Drug Administration, Center for
Devices and Radiological Health. 2007 March 13. Guidance for Industry and FDA Staff:
Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
Washington, DC: Author.
7 Schley G, Köberle C, Manuilova E, Rutz S, Forster C, Weyand M, et al. (2015) Comparison of
Plasma and Urine Biomarker Performance in Acute Kidney Injury. PLoS ONE 10(12):
e0145042. doi:10.1371/journal.pone.0145042
8 Clerico A, Galli C, Fortunato A, Ronco C. Neutrophil gelatinase-associated lipocalin (NGAL)
as biomarker of acute kidney injury: a review of the laboratory characteristics and clinical
evidences. Clin Chem Lab Med. 2012 Feb
15;50(9):1505-17. doi: 10.1515/cclm-2011-0814. Review. PubMed PMID: 22962216.
9 Devarajan P. Neutrophil gelatinase-associated lipocalin: a promising biomarker for human
acute kidney injury. Biomarkers in medicine. 2010;4(2):265-280. doi:10.2217/bmm.10.12.
10 Nickolas TL, O'Rourke MJ, Yang J, Sise ME, Canetta PA, Barasch N, Buchen C, Khan F,
Mori K, Giglio J, Devarajan P, Barasch J. Sensitivity and specificity of a single emergency
department measurement of urinary neutrophil gelatinase-associated lipocalin for diagnosing
Page 19 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
20
acute kidney injury. Ann Intern Med. 2008 Jun 3;148(11):810-9. PubMed PMID: 18519927;
PubMed Central PMCID: PMC2909852.
11
Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin
(NGAL) predicts renal injury in acute decompensated cardiac failure: a prospective
observational study. BMC Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.
12
Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,
Yin YH. Diagnosis and prognosis of neutrophil gelatinase-associated lipocalin for acute kidney
injury with sepsis: a systematic review and meta-analysis. Crit Care. 2016 Feb 16;20:41. doi:
10.1186/s13054-016-1212-x. PubMed PMID: 26880194; PubMed Central PMCID:
PMC4754917.
13
Marino R, Struck J, Hartmann O, Maisel AS, Rehfeldt M, Magrini L, Melander O, Bergmann
A, Di Somma S. Diagnostic and short-term prognostic utility of plasma pro-enkephalin (pro-
ENK) for acute kidney injury in patients admitted with sepsis in the emergency department. J
Nephrol. 2015 Dec;28(6):717-24. doi:10.1007/s40620-014-0163-z. Epub 2014 Dec 9. PubMed
PMID: 25486879.
14
Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,
McCullough PA. Urine catalytic iron and neutrophil gelatinase-associated lipocalin as
companion early markers of acute kidney injury after cardiac surgery: a prospective pilot study.
Cardiorenal Med. 2013 Apr;3(1):7-16. doi: 10.1159/000346815. PubMed PMID: 23946721;
PubMed Central PMCID: PMC3743453.
15 Haase-Fielitz A, Haase M, Devarajan P. Neutrophil gelatinase-associated lipocalin as a
biomarker of acute kidney injury: a critical evaluation of current status. Annals of clinical
biochemistry. 2014;51(0 3):335-351. doi:10.1177/0004563214521795.
16 Filiopoulos V, Biblaki D, Vlassopoulos D. Neutrophil gelatinase-associated lipocalin (NGAL):
a promising biomarker of contrast-induced nephropathy after computed tomography. Ren Fail.
2014 Jul;36(6):979-86. doi: 10.3109/0886022X.2014.900429. Review. PubMed PMID:
24673459.
17 Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A; NGAL Meta-analysis
Investigator Group. Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in
diagnosis and prognosis in acute kidney injury: a systematic review and meta-analysis. Am J
Kidney Dis. 2009 Dec;54(6):1012-24. doi:10.1053/j.ajkd.2009.07.020. Epub 2009 Oct 21.
Review. PubMed PMID: 19850388.
18
Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A, Birkhahn
R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson KJ, Haase
M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis M, Kim P, Koyner JL,
Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney S, Ostermann M, Rimmelé T,
Page 20 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
21
Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL, Vinsonneau C, Wagner L, Walker MG,
Wilkerson RG, Zacharowski K, Kellum JA. Discovery and validation of cell cycle arrest
biomarkers in human acute kidney injury. Crit Care. 2013 Feb 6;17(1):R25. doi:
10.1186/cc12503. PubMed PMID: 23388612; PubMed Central PMCID: PMC4057242.
19
Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald R,
Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL, Krell K,
Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH, Sellman R, Shi J,
Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J, Wunderink R, Zimmerman J,
Kellum JA. Validation of cell-cycle arrest biomarkers for acute kidney injury using clinical
adjudication. Am J Respir Crit Care Med. 2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-
0077OC. PubMed PMID: 24559465.
20
McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner JL,
Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C, Okusa MD,
Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M, Murray PT, Cruz D,
Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C. Implementation of novel
biomarkers in the diagnosis, prognosis, and management of acute kidney injury: executive
summary from the tenth consensus conference of the Acute Dialysis Quality Initiative (ADQI).
Contrib Nephrol. 2013;182:5-12. doi:10.1159/000349962. PubMed PMID: 23689652; PubMed
Central PMCID: PMC3856225.
Page 21 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1a
338x190mm (300 x 300 DPI)
Page 22 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1b
338x190mm (300 x 300 DPI)
Page 23 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1c
338x190mm (300 x 300 DPI)
Page 24 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2a
338x190mm (300 x 300 DPI)
Page 25 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2b
338x190mm (300 x 300 DPI)
Page 26 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2c
338x190mm (300 x 300 DPI)
Page 27 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3a
338x190mm (300 x 300 DPI)
Page 28 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3b
338x190mm (300 x 300 DPI)
Page 29 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3c
338x190mm (300 x 300 DPI)
Page 30 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Section & Topic No Item Reported on page #
TITLE OR ABSTRACT
1 Identification as a study of diagnostic accuracy using at least one measure of accuracy
(such as sensitivity, specificity, predictive values, or AUC)
1, 2
ABSTRACT
2 Structured summary of study design, methods, results, and conclusions
(for specific guidance, see STARD for Abstracts)
2-3
INTRODUCTION
3 Scientific and clinical background, including the intended use and clinical role of the index test 4
4 Study objectives and hypotheses 4
METHODS
Study design 5 Whether data collection was planned before the index test and reference standard
were performed (prospective study) or after (retrospective study)
4, 6
Participants 6 Eligibility criteria 4-5
7 On what basis potentially eligible participants were identified
(such as symptoms, results from previous tests, inclusion in registry)
4-5
8 Where and when potentially eligible participants were identified (setting, location and dates) 4-5
9 Whether participants formed a consecutive, random or convenience series 4-5
Test methods 10a Index test, in sufficient detail to allow replication 5
10b Reference standard, in sufficient detail to allow replication 4, 5, Sup. Table 1
11 Rationale for choosing the reference standard (if alternatives exist) 4, 5, Sup. Table 1
12a Definition of and rationale for test positivity cut-offs or result categories
of the index test, distinguishing pre-specified from exploratory
6
12b Definition of and rationale for test positivity cut-offs or result categories
of the reference standard, distinguishing pre-specified from exploratory
4, 5, 6
13a Whether clinical information and reference standard results were available
to the performers/readers of the index test
4, 5
13b Whether clinical information and index test results were available
to the assessors of the reference standard
4, 5
Analysis 14 Methods for estimating or comparing measures of diagnostic accuracy 7-8, Figures
15 How indeterminate index test or reference standard results were handled 7
16 How missing data on the index test and reference standard were handled 7
17 Any analyses of variability in diagnostic accuracy, distinguishing pre-specified from exploratory
18 Intended sample size and how it was determined 7
RESULTS
Participants 19 Flow of participants, using a diagram 7 – in text
20 Baseline demographic and clinical characteristics of participants 7, Table 1
21a Distribution of severity of disease in those with the target condition Table 1
21b Distribution of alternative diagnoses in those without the target condition Table 1
22 Time interval and any clinical interventions between index test and reference standard 5
Test results 23 Cross tabulation of the index test results (or their distribution)
by the results of the reference standard
Table 1
24 Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) 7-8, Figures
25 Any adverse events from performing the index test or the reference standard 7
DISCUSSION
26 Study limitations, including sources of potential bias, statistical uncertainty, and generalisability 11
27 Implications for practice, including the intended use and clinical role of the index test 8, 11-12
OTHER
28 Registration number and name of registry 13
29 Where the full study protocol can be accessed 5
30 Sources of funding and other support; role of funders 13
Page 31 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
STARD 2015
AIM
STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the
completeness and transparency of reporting of diagnostic accuracy studies. Authors can use the list to write informative
study reports. Editors and peer-reviewers can use it to evaluate whether the information has been included in manuscripts
submitted for publication.
EXPLANATION
A diagnostic accuracy study evaluates the ability of one or more medical tests to correctly classify study participants as
having a target condition. This can be a disease, a disease stage, response or benefit from therapy, or an event or condition
in the future. A medical test can be an imaging procedure, a laboratory test, elements from history and physical examination,
a combination of these, or any other method for collecting information about the current health status of a patient.
The test whose accuracy is evaluated is called index test. A study can evaluate the accuracy of one or more index tests.
Evaluating the ability of a medical test to correctly classify patients is typically done by comparing the distribution of the
index test results with those of the reference standard. The reference standard is the best available method for establishing
the presence or absence of the target condition. An accuracy study can rely on one or more reference standards.
If test results are categorized as either positive or negative, the cross tabulation of the index test results against those of the
reference standard can be used to estimate the sensitivity of the index test (the proportion of participants with the target
condition who have a positive index test), and its specificity (the proportion without the target condition who have a negative
index test). From this cross tabulation (sometimes referred to as the contingency or “2x2” table), several other accuracy
statistics can be estimated, such as the positive and negative predictive values of the test. Confidence intervals around
estimates of accuracy can then be calculated to quantify the statistical precision of the measurements.
If the index test results can take more than two values, categorization of test results as positive or negative requires a test
positivity cut-off. When multiple such cut-offs can be defined, authors can report a receiver operating characteristic (ROC)
curve which graphically represents the combination of sensitivity and specificity for each possible test positivity cut-off. The
area under the ROC curve informs in a single numerical value about the overall diagnostic accuracy of the index test.
The intended use of a medical test can be diagnosis, screening, staging, monitoring, surveillance, prediction or prognosis. The
clinical role of a test explains its position relative to existing tests in the clinical pathway. A replacement test, for example,
replaces an existing test. A triage test is used before an existing test; an add-on test is used after an existing test.
Besides diagnostic accuracy, several other outcomes and statistics may be relevant in the evaluation of medical tests. Medical
tests can also be used to classify patients for purposes other than diagnosis, such as staging or prognosis. The STARD list was
not explicitly developed for these other outcomes, statistics, and study types, although most STARD items would still apply.
DEVELOPMENT
This STARD list was released in 2015. The 30 items were identified by an international expert group of methodologists,
researchers, and editors. The guiding principle in the development of STARD was to select items that, when reported, would
help readers to judge the potential for bias in the study, to appraise the applicability of the study findings and the validity of
conclusions and recommendations. The list represents an update of the first version, which was published in 2003.
More information can be found on http://www.equator-network.org/reporting-guidelines/stard.
Page 32 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for the Prediction of Acute
Kidney Injury among Critically Ill Adults: Retrospective Determination and Clinical Validation of a Prospective
Multicenter Study
Journal: BMJ Open
Manuscript ID bmjopen-2017-016028.R2
Article Type: Research
Date Submitted by the Author: 09-May-2017
Complete List of Authors: Tecson, Kristen; Baylor Heart and Vascular Institute, ; Texas A&M College of Medicine Health Science Center, Ehrhardtsen, Elisabeth; BioPorto Diagnostics A/S Erikson, Peter; BioPorto Diagnostics A/S Gaber, A; Houston Methodist Hospital
Germain, Michael; Baystate Medical Center Golestaneh, Ladan; Yeshiva University Albert Einstein College of Medicine Lavoria, Maria de los Angeles; BioPorto Diagnostics A/S Moore, Linda; Houston Methodist Hospital McCullough, Peter; Baylor Heart and Vascular Institute
<b>Primary Subject Heading</b>:
Renal medicine
Secondary Subject Heading: Diagnostics
Keywords: neutrophil gelatinase associated lipocalin, acute kidney injury, risk prediction, sensitivity, specificity
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on July 11, 2020 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2017-016028 on 10 July 2017. D
ownloaded from
For peer review only
1
Optimal Cut Points of Plasma and Urine Neutrophil Gelatinase-Associated Lipocalin for
the Prediction of Acute Kidney Injury among Critically Ill Adults: Retrospective
Determination and Clinical Validation of a Prospective Multicenter Study
Kristen M. Tecson, PhD, Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, A. Osama Gaber,
MD, Michael Germain, MD, Ladan Golestaneh, MD, Maria de los Angeles Lavoria, MsC, Linda
W. Moore, MS, Peter A. McCullough, MD, MPH
Author Affiliations: Baylor Heart and Vascular Institute, Dallas, Texas (Tecson, McCullough);
Baylor Scott & White Research Institute, Dallas, Texas (Tecson); Texas A&M College of
Medicine Health Science Center, Dallas, Texas (Tecson, McCullough); BioPorto Diagnostics
A/S, Copenhagen, Denmark (Erhardtsen, Eriksen, Lavoria); Albert Einstein College of
Medicine, Bronx, NY (Golestaneh); Houston Methodist Hospital, Houston, TX (Gaber, Moore);
Baystate Medical Center, Springfield, MA (Germain); Baylor University Medical Center,
Dallas, Texas (McCullough); Baylor Jack and Jane Hamilton Heart and Vascular Hospital,
Dallas, Texas (McCullough)
Short Title: Blood and Urine NGAL in Risk Prediction of AKI
Keywords: neutrophil gelatinase associated lipocalin, siderocalin-2, acute kidney injury, risk
prediction, sensitivity, specificity
Word Count: 3671
Corresponding Author:
Peter A. McCullough, MD, MPH
621 N Hall Street, Suite H-030
Dallas, TX 75226
Page 1 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
2
Abstract
Objectives: To determine the optimal threshold of blood and urine Neutrophil gelatinase-
associated lipocalin (NGAL) to predict moderate to severe AKI and persistent moderate to severe
AKI lasting at least 48 consecutive hours, as defined by an adjudication panel.
Methods: A multicenter prospective observational study enrolled intensive care unit patients and
recorded daily ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine
NGAL. We utilized natural log-transformed NGAL in a logistic regression model to predict
stage 2/3 AKI (defined by Kidney Disease International Global Organization). We performed the
same analysis using the NGAL value at the start of persistent stage 2/3 AKI.
Results: Of 245 subjects, 33 (13.5%) developed stage 2/3 AKI and 25 (10.2%) developed
persistent stage 2/3 AKI. Predicting stage 2/3 AKI revealed the optimal NGAL cutoffs in EDTA
plasma (142.0 ng/ml), heparin plasma (148.3 ng/ml), and urine (78.0 ng/ml) and yielded the
following decision statistics: sensitivity (SN) of 78.8%, specificity (SP) of 73.0%, positive
predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7%, diagnostic accuracy
(DA)=73.8% (EDTA plasma); SN=72.7%, SP=73.8%, PPV=30.4%, NPV=94.5%, DA=73.7%
(heparin plasma); SN=69.7%, SP=76.8%, PPV=32.9%, NPV=94%, DA=75.8% (urine). The
optimal NGAL cutoffs to predict persistent stage 2/3 AKI were similar: 148.3 ng/ml (EDTA
plasma), 169.6 ng/ml (heparin plasma), and 79.0 ng/ml (urine) yielding: SN=84.0%, SP=73.5%,
PPV=26.6%, NPV=97.6, DA=74.6% (EDTA plasma); SN=84%, SP=76.1%, PPV=26.8%,
NPV=96.5%, DA=76.1% (heparin plasma); SN=75%, SP=75.8%, PPV=26.1, NPV=96.4%,
DA=75.7% (urine).
Page 2 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
3
Conclusion: Blood and urine NGAL predicted stage 2/3 AKI, as well as persistent 2/3 AKI in
the ICU with acceptable decision statistics using a single cut point in each type of specimen.
Abstract word count: 261
Strengths and limitations of this study:
• This study utilized biomarker data from both plasma and urine samples.
• Acute kidney injury diagnosis was adjudicated by an expert panel.
• We used an unbiased, data-driven approach to identify a single cut point for each of
plasma and urinary NGAL samples to predict stage 2 or 3 acute kidney injury and
persistent stage 2 or 3 acute kidney injury.
• This small prospective cohort study had limited clinical follow-up.
Page 3 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
4
Introduction
Serum creatinine and urine output are used to diagnose acute kidney injury (AKI) as
defined by the 2012 Kidney Disease International Global Organization (KDIGO) guidelines
(Supplemental Table 1); however, creatinine is an imperfect marker as it peaks anywhere from 2-
7 days following an insult to the kidneys.1 Additionally, urine output may lead to an inaccurate
diagnosis of AKI in the intensive care unit (ICU), as it is commonly manipulated in the ICU
through use of intravenous fluids and diuretics. Neutrophil gelatinase-associated lipocalin
(NGAL) has been shown to peak following tubular injury earlier than changes in creatinine and
urine output, making it a desirable biomarker for predicting the development of AKI.2 Previous
studies have shown the utility of this biomarker but have suffered from heterogeneous AKI
thresholds and lack of validation of parenchymal renal involvement in the injury. In those
studies, AKI may have been due to a transient hemodynamically related reduction in renal
filtration due poor forward perfusion or impaired plasma refill, which represents a distinct AKI
pathophysiology.3, 4
Furthermore, the optimal body fluid for NGAL assay performance (blood or
urine) is unknown, as is the optimal cut point for the detection of moderate to severe AKI
(KDIGO stage 2 or 3). We set out to use a commercially available NGAL assay in a multicenter,
blinded study to assess its performance in two types of plasma samples and in one urine sample
for the prediction of moderate to severe AKI, as defined by KDIGO and adjudicated by
clinicians blinded to the NGAL values.
Methods
Study Details
This study prospectively enrolled consecutive adult patients admitted to an ICU or critical
care setting after obtaining informed consent between March 5, 2014 and April 15, 2015 from 4
Page 4 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
5
participating hospitals in Boston and Springfield, Massachusetts, Bronx, New York, and
Houston, Texas; the protocol may be found at the following:
http://www.bioporto.com/Products/Featured-areas/NGAL.aspx. Each study site received
Institutional Review Board (IRB) approval prior to enrolling patients (IRBs utilized for this study
are as follows: Baystate Health IRB #1; Houston Methodist Research Institute (HMRI) IRB 1;
Pratners Human Research Committee; Biomedical Research Alliance of New York (BRANY)
IRB). Those with history of nephrectomy, renal transplantation and/or renal replacement therapy
initiated before admission were not eligible to participate. In addition to the standard clinical care
and laboratory testing, one urine and two plasma samples were drawn and frozen per day in the
ICU, up to 8 days. The samples were shipped to a central laboratory where NGAL was assayed
by BioPorto Diagnostics A/S, Copenhagen, Denmark. This test has a measurable range of 25-
3000 ng/ml. 5 At mean NGAL concentrations of 97, 116, and 112 ng/ml in
ethylenediaminetetraacetic acid (EDTA) plasma, heparin plasma, and urine, respectively, the
coefficients of variation were 3.1%, 1.8%, and 2.8%, respectively. If patients were discharged
early from the ICU during the study period, serum creatinine levels were collected manually
from the hospital data system for 48 hours provided the patient was still hospitalized. Reference
standard AKI was determined daily, up to 8 days, by a three-physician adjudication panel using
the KDIGO® guidelines. AKI was confirmed by the adjudication panel and disagreements were
settled by conference calls and discussion of the cases. Adjudicators were blinded to the NGAL
values and physician notes concerning the renal diagnosis.6
Patient Involvement
Patient involvement began at the time of informed consent; patients and caregivers were
not involved in the design of, recruitment for, or conduct of this study. The development of the
Page 5 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
6
outcome measures was not informed by patients’ priorities, experiences, or preferences. The
participants’ contributions to this study are deeply appreciated and the results of the study will
be publicly available to the patients via this publication.
Analyses
This manuscript was constructed via retrospective analyses of prospectively collected
data and was developed under STARD guidelines; ethics committee approval was obtained prior
to conducting these analyses. NGAL values were highly skewed in all three sample types so we
utilized the natural-log transformed distributions (Figure 1). We performed logistic regression
using the natural log-transformed baseline NGAL [i.e. ln(baseline NGAL)] value as a predictor
for the outcome of moderate/severe (stage 2/3) AKI at any point during observation.
Additionally, we defined persistent moderate/severe (stage 2/3) AKI as 2 or more days of
consecutive stage 2/3 AKI and we constructed another logistic regression model for the outcome
of persistent 2/3 AKI using a wandering ln(baseline NGAL) value as the predictor variable. The
wandering ln(baseline NGAL) value was taken on the morning of the first day of a 48 hour
persistent AKI period. If AKI did not occur, the wandering ln(baseline NGAL) value was taken
to be the ln(baseline NGAL). We found the optimal NGAL value for both outcomes by using the
criterion of smallest distance to the “perfect point” on the receiver operating characteristic (ROC)
curve, (0,1). Baseline NGAL values < 5 ng/ml were adjusted using a regression model, which
affected approximately 2% of the data. We assessed differences in baseline characteristics
between those who had at least 1 day of stage 2/3 AKI at any time during observation and those
who did not via Wilcoxon Rank Sum and Chi-square tests; we did the same for those who had
persistent stage 2/3 AKI and those who did not have persistent stage 2/3 AKI at any point during
Page 6 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
7
observation. Continuous variables are reported as medians [quarter 1, quarter 3]. Categorical
variables are reported as frequencies and proportions.
Results
A total of 252 patients were screened for the study; 2 were screen failures, and 5 did not
have sufficient information to be included in analyses. Hence, 245 patients were enrolled and
included in analyses. Only 2 patients in this study had cardiac surgery during the observational
period. No adverse events occurred during the observation period. There were 33 (13.5%)
subjects who developed stage 2/3 AKI as determined by the adjudicators. With this event rate,
we had 95% power to detect an area under the curve (AUC) as low as 0.69. Age and gender did
not differ significantly between those with or without stage 2/3 AKI; however, those with stage
2/3 AKI had more comorbidities (sepsis: 18.2% v. 5.7%, p=0.01; diabetes 48.5% v. 27.8%,
p=0.04; chronic kidney disease: 33.3% v. 10.4%, p<0.001), longer ICU stays (4 [2, 8] days v. 2
[1, 3] days, p=0.001), and higher baseline NGAL levels (EDTA plasma: 276 [155, 517] ng/ml v.
98 [70, 154] ng/ml, p<0.001; heparin plasma: 288 [150, 669] ng/ml v. 100 [68, 160] ng/ml, p <
0.001; urine: 190 [61, 1133] ng/ml v. 31 [15, 67], p < 0.001) than those without AKI (Table 1).
Using the ln(baseline EDTA plasma NGAL) as the independent variable in a logistic regression
model for the outcome of stage 2/3 AKI at any time yielded an AUC = 0.76, 95% CI 0.64-0.87,
p<0.0001 (Figure 2). Optimizing the ROC curve with respect to the distance from (0,1) resulted
in a cut point of 142.0 ng/ml, which yielded a sensitivity (SN) of 78.8%, specificity (SP) of
73.0%, positive predictive value (PPV) of 31.3%, negative predictive value (NPV) of 95.7% and
diagnostic accuracy (DA) of 73.8%. Similar results are shown for heparin plasma and urine
NGAL in Figures 2b and 2c.
Page 7 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
8
Twenty-five (10.2%) subjects had persistent stage 2/3 AKI. Demographics did not differ
significantly between those who did and those who did not have persistent stage 2/3 AKI;
however, those with persistent stage 2/3 AKI had a higher rate of chronic kidney disease (CKD)
(36.0% v. 10.9%, p < 0.001), longer ICU stays (6 [2,8] days v. 2 [1, 3] days, p <0.001), and
higher wandering baseline NGAL values (EDTA plasma: 287 [188, 517] ng/ml v. 98 [70, 156]
ng/ml, p <0.001; heparin: 355 [178, 716] ng/ml v. 101 [68, 162] ng/ml, p<0.001; urine:
231.1[84.3, 1203] ng/ml v. 32 [15, 75] ng/ml, p<0.001) (Table 2). We utilized the wandering
ln(baseline EDTA plasma NGAL) in a logistic regression model to predict persistent stage 2/3
AKI, which yielded an AUC = 0.85, 95% CI 0.77-0.94, p<0.001. The optimal cut point for
NGAL based on the distance to (0,1) was 148.3 ng/ml (Figure 3). This value yielded SN= 84.0%,
SP=73.5%, PPV= 26.6%, NPV= 97.6%, DA=74.6%. Similar cut points and results are shown for
heparin plasma and urine NGAL in Figures 3b and 3c. Odds ratios from the regression models
are shown in Table 3.
Discussion
We found that a single measured threshold for NGAL, either in blood or urine, was
effective in risk prediction for the development of moderate to severe AKI and persistent AKI, as
defined by AKI lasting 48 consecutive hours or longer. NGAL measured on a daily basis
performed well in the prognostication of persistent AKI. Our results were internally consistent
and add to the literature as they yielded unbiased cut points from prospectively collected samples
and adjudicated AKI outcomes. We found that urine and plasma yielded different absolute
NGAL concentrations, however, within each sample type the cut point for the prediction of stage
2/3 AKI or persistent AKI was similar, making daily measurements and utilization of a single
threshold feasible, provided there is consistency in sample source and handling procedures. Our
Page 8 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
9
results are concordant with a prior study that observed slightly better discriminatory power of
plasma NGAL than urinary NGAL, with an AUC as high as 0.84. 7 Thus, we believe for ease of
use, plasma NGAL is more likely to be incorporated into routine care than urinary NGAL.
Our findings build upon the work of others using a variety of NGAL assays
demonstrating the concept that 25 kilodalton protein is rapidly and reliably upregulated after
ischemia/reperfusion injury to the kidney.8,9, 4
The work by Nikolas et al. revealed that NGAL
even has the ability to distinguish AKI from non-progressive CKD, a feat that creatinine has
failed to accomplish.10 Further, the urine NGAL measured in the emergency department was
associated with the development of end-stage renal disease requiring dialysis in the hospital. A
2010-2011 study in western Australia, n=102, demonstrated the effectiveness of using NGAL to
predict inpatient AKI (based on risk, injury, failure, loss of kidney function, and end-stage
kidney disease classification) in a tertiary care setting. In the Australian study, the Biosite
Triage® NGAL in whole blood (Alere, Inverness Medical, Australia Inc) was found to have a
single optimal cutoff of 89 ng/ml had SN = 68%, SP = 70%, and AUC = 0.71, 95% CI 0.58-
0.84.11While the threshold was similar to that found in our EDTA and heparin samples, this
study used all stages of AKI as the endpoint and did not have blinded adjudication by a
committee. Nevertheless, these data are in line with our conclusion that NGAL measured in
blood performs well as a prognostic aid for AKI. A meta-analysis of 1,478 septic patients
demonstrated that NGAL was associated with the development of AKI, need for renal
replacement therapy, and mortality.12 Marino et al.’s study of 101 emergency department
patients with sepsis confirmed that NGAL was related to (the severity of) AKI; however, NGAL
concentrations were also elevated in the absence of AKI in these patients.13 Numerous studies
have demonstrated that NGAL rises after cardiac surgery and is congruent with AKI.14,15
Page 9 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
10
Additionally, NGAL concentrations increase after contrast-induced AKI; however, it is to a
much lesser extent than from cardiac surgery or critical illness.16 An extensive literature review
and meta-analysis by Haase et al. revealed a wide range of optimal reported NGAL cutoffs for
AKI prediction (100 – 270 ng/ml).17 The optimal plasma cutoffs found in our study to detect
stage 2/3 AKI (EDTA: 142.0 ng/ml, heparin:149.5 ng/ml) were within the range reported in the
meta-analysis. Haase suggested cutoffs between 150-170 ng/ml for adults, which is slightly
higher than our findings, and may be attributable to the high rate of postoperative patients used
in the meta-analysis, as well as the varied definitions of AKI. Our study is the first among these
to demonstrate that NGAL can be used to risk predict stage 2/3 AKI and its persistence in a
cohort of ICU patients. We further demonstrated a single cutoff value performs well in terms of
decision statistics for the outcomes of stage 2/3 AKI and persistent stage 2/3 AKI.
There have been large prospective studies examining novel biomarkers other than NGAL
for the prediction of stage 2/3 AKI among the critically ill. Kashani and colleagues (n=728)
measured the concentrations of urinary tissue inhibitor of metalloproteinases (TIMP)-2 and
insulin-like growth factor binding protein 7 (IGFBP7), and combined the 2 measures into 1
predictor, referred to as [TIMP-2]*[IGFBP7], NephroCheck® (Astute Medical, San Diego, CA)
to identify imminent stage 2/3 AKI (i.e. in the next 12-hours) which occurred in 14% of study
participants. Baseline urinary [TIMP-2]*[IGFBP7] yielded an AUC = 0.80, 97% CI 0.76-0.83;
however, the full decision statistics of an optimal cut point were not disclosed. 18 A second study
by Bihorac and colleagues (n=420), evaluated urinary [TIMP-2]*[IGFBP7] for the same
endpoint. In this study, 71 (17.4%) patients had imminent stage 2/3 AKI and the model had an
AUC = 0.82, 95% CI 0.76-0.88. However, [TIMP-2]*[IGFBP7] did not appear amenable to a
single threshold value. Two cut points were presented; one yielding high SN and low SP and the
Page 10 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
11
other yielding low SN and high SP. At the cutoff of 0.3 (ng/ml)2/1000 the SN = 92, 95% CI, 85–
98%, but the SP= 46, 95% CI 41–52%. At a 7-fold higher cut point of 2.0 (ng/ml)2/1000 the SN
= 37, 95% CI, 26–47% and SP= 95, 95% CI, 93–97%.19 In comparison, an NGAL EDTA
plasma level of 142.0 ng/ml in our study predicted stage 2/3 AKI and yielded SN=78.8, 95% CI
73.7-83.9% and SP=73.0, 95% CI 67.4-78.6%. Thus, for NGAL at a single cut point, the lower
bounds of the 95% CIs for sensitivity and specificity were well above 60 and 40%, respectively.
Accordingly, NGAL appears to have considerable advantages over [TIMP-2]*[IGFBP7] in
forecasting AKI over a longer time window, at a single prognostic threshold, and avoids low
sensitivity or specificity. Additionally, we demonstrated NGAL performed well when measured
daily using similar cut points for the prediction of persistent stage 2/3 AKI. Daily use in risk
prediction has not been demonstrated in any study with [TIMP2]*[IGFBP7].
Limitations
Our study has all the limitations of small prospective cohort studies of biomarkers
including limited clinical follow-up and lack of precise follow-up information, including accurate
urine output. We did not consider stage I AKI in the definition of the primary endpoint because
prior studies suggested that many in this category could have transient hemodynamic elevations
of creatinine without parenchymal AKI. Thus, our event rate is lower than other studies which
included all KDIGO stages of AKI. Our study sample was non-homogeneous in that it included
patients with sepsis, a diagnosis which has been shown to increase NGAL levels even for those
without AKI.13 Further, due to the small sample size, we could not analyze sepsis or CKD
subjects separately, rendering it impossible to determine whether or not different cutoffs were
needed. To examine confounders’ relations with (persistent) stage 2/3 AKI, we built multivariate
models; however, neither sepsis, nor CKD, nor DM were statistically significant when
Page 11 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
12
considered with NGAL simultaneously. Consequently, the adjusted odds ratio estimates changed
only slightly, and the corresponding p-values were unchanged. Perhaps this is due, in part, to the
fact that NGAL can distinguish AKI from non-progressive CKD. Additionally, the small sample
size did not permit the use of training and/or validation sets, which prohibited further
investigation of the models. However, all AUCs were at least 0.76, which indicates good
predictive ability. Finally, we did not have information on long term outcomes such as need for
renal replacement therapy or mortality which would have been useful in integrating NGAL with
serum creatinine and urine output in an in-depth assessment of patient outcomes.20
Conclusion
NGAL concentrations of 142.0, 149.5, and 78.0 ng/ml in EDTA plasma, heparin plasma,
and urine, respectively, were found to be optimal in predicting stage 2/3 AKI in ICU patients
participating in a prospective observational study. Similarly, NGAL concentration thresholds of
148.3, 169.6, and 79.2 ng/ml in EDTA plasma, heparin plasma, and urine, respectively, were
optimal in predicting persistent stage 2/3 AKI in the same cohort of ICU patients. A larger
prospective study to provide external validation of these cut points is warranted.
Page 12 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
13
Funding
The study was funded by BioPorto Diagnostics A/S.
Disclosures
Elisabeth Erhardtsen, DVM, Peter Mørch Eriksen, Maria de los Angeles Lavoria are employees
of BioPorto Diagnostics A/S. The remaining authors have nothing to disclose.
Acknowledgements
The authors would like to thank the study participants who graciously shared their time and
materials for the purposes of this research.
Transparency
The lead author affirms that this manuscript is an honest, accurate, and transparent account of the
study being reported; that no important aspects of the study have been omitted; and that any
discrepancies from the study as planned (and, if relevant, registered) have been explained.
Copyright
“I, Peter A. McCullough, The Corresponding Author of this article contained within the original
manuscript which includes any diagrams & photographs within and any related or stand alone
film submitted (the Contribution”) has the right to grant on behalf of all authors and does grant
on behalf of all authors, a licence to the BMJ Publishing Group Ltd and its licencees, to permit
this Contribution (if accepted) to be published in the BMJ and any other BMJ Group products
and to exploit all subsidiary rights, as set out in our licence set out at:
http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open-
access-and-permission-reuse.” I am one author signing on behalf of all co-owners of the
Contribution.
Contributorship
PAM conceived the study. PME funded the study. AOG, MG, LG, and LWM initiated the study
design and helped with implementation. KMT conducted the statistical analysis. KMT and PAM
drafted the article. All authors contributed to the manuscript’s refinement and approved the final
version.
Data: The data from this study will not be made available.
Page 13 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
14
Figure 1. Mean and standard error of natural log-transformed NGAL by day and acute kidney
injury status (a. EDTA plasma, b. heparin plasma, c. urine)
Figure 2. Receiver operating characteristic curves for natural log-transformed baseline NGAL as
a predictor of stage 2/3 acute kidney injury with the optimal NGAL threshold identified (a.
EDTA plasma, b. heparin plasma, c. urine). Decision statistics are presented for the optimal cut
point (bold) as well as at cut points approximately 20 ng/ml below and above the optimal value.
LR+ indicates positive likelihood ratio and LR- indicated negative likelihood ratio.
Figure 3. Receiver operating characteristic curves for wandering natural log-transformed
baseline NGAL as a predictor of persistent stage 2/3 acute kidney injury with the optimal NGAL
threshold identified (a. EDTA plasma, b. heparin plasma, c. urine). Decision statistics are
presented for the optimal cut point (bold) as well as at cut points approximately 20 ng/ml below
and above the optimal value. LR+ indicates positive likelihood ratio and LR- indicated negative
likelihood ratio.
Page 14 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
15
Table 1. Patient characteristics by occurrence of moderate/severe acute kidney injury (AKI)
Variable Stage 2 or 3 AKI
(n=33)
Stage 1 or No AKI
(n = 212) P-value
Age (years) 68 [56, 74] 63 [54, 73] 0.5
Gender (male) 22 (66.7%) 135 (63.7%) 0.74
Race/Ethnicity 0.86
African American 3 (9.1%) 29 (13.7%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 4 (12.1%) 28 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 26 (78.8%) 151 (71.2%)
Hypertension 26 (78.8%) 142 (67.0%) 0.17
Congestive Heart Failure 6 (18.2%) 38 (17.9%) 0.97
Diabetes Mellitus 0.04
Type I 0 (0.0%) 2 (0.9%)
Type II 16 (48.5%) 57 (26.9%)
None 17 (51.5%) 153 (72.2%)
Urinary Tract Infection 2 (6.1%) 6 (2.8%) 0.33
Sepsis 6 (18.2%) 12 (5.7%) 0.01
Chronic Kidney Disease History <0.001
None 22 (66.7%) 190 (89.6%)
Stage 1 1 (3.0%) 4 (1.9%)
Stage 2 1 (3.0%) 3 (1.4%)
Stage 3A 8 (24.2%) 5 (2.4%)
Stage 3B 0 (0.0%) 8 (3.8%)
Stage 4 1 (3.0%) 2 (0.9%)
In-hospital renal replacement therapy 4 (12.1%) 2 (0.9%) 0.003
Nephrotoxin use at baseline 18 (54.6%) 88 (41.5%) 0.19
Baseline EDTA NGAL (ng/ml) 276 [155, 517] 98 [70, 154] <0.001
Baseline heparin plasma NGAL (ng/ml) 288 [150, 669] 100 [68, 160] <0.001
Baseline urine NGAL (ng/ml) 190 [61, 1133] 31 [15, 67] <0.001
Days in intensive care unit 4 [2, 8] 2 [1,3] 0.001
Table 2. Sample characteristics for subjects with and without persistent stage 2/3 acute kidney
Page 15 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
16
injury
Variable Persistent AKI
(n=25)
No persistent
AKI (n=220) P-value
Age (years) 68 [60, 73] 63 [54, 73] 0.31
Gender (male) 16 (64.0%) 141 (64.1%) 0.99
Race/Ethnicity 0.89
African American 2 (8.0%) 30 (13.6%)
Asian 0 (0.0%) 2 (0.9%)
Hispanic or Latino 3 (12.0%) 29 (13.2%)
Mixed 0 (0.0%) 2 (0.9%)
White 20 (80.0%) 157 (71.4%)
Hypertension 19 (76.0%) 149 (67.7%) 0.4
Congestive Heart Failure 4 (16.0%) 40 (18.2%) 0.79
Diabetes Mellitus 0.4
Type I 0 (0.0%) 2 (0.9%)
Type II 10 (40.0%) 63 (28.6%)
None 15 (60.0%) 155 (70.5%)
Urinary Tract Infection 2 (8.0%) 6 (2.73%) 0.19
Sepsis 4 (16.0%) 14 (6.4%) 0.1
Chronic Kidney Disease History <0.001
None 16 (64.0%) 196 (89.1%)
Stage 1 1 (4.0%) 4 (1.8%)
Stage 2 1 (4.0%) 3 (1.4%)
Stage 3A 6 (24.0%) 7 (3.2%)
Stage 3B 0 (00%) 8 (3.6%)
Stage 4 1 (94.0%) 2 (0.9%)
In-hospital renal replacement therapy 4 (16.0%) 2 (0.9%) 0.001
Use of nephrotoxins 14 (56.05) 92 (41.8%) 0.18
Wandering baseline EDTA NGAL (ng/ml) 287 [188,517] 98 [70,156] <0.001
Wandering baseline heparin plasma NGAL
(ng/ml) 355 [178, 716] 101 [68, 162] <0.001
Wandering baseline urine NGAL (ng/ml) 231.1 [84.3, 1203] 32 [15, 75] <0.001
Days in intensive care unit 6 [2, 8] 2 [1,3] <0.001
Page 16 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
17
Table 3. Odds ratios for stage 2/3 acute kidney injury and persistent stage 2/3 acute kidney injury
using EDTA, heparin, and urine NGAL values as predictors (odds are associated with an
approximate 2.73-fold increase in NGAL value)
Outcome NGAL Measure P-Value Odds Ratio (95% CI)
Stage 2/3 acute
kidney injury
EDTA <0.001 2.8 (1.8, 4.4)
Heparin <0.001 3.2 (2.0, 4.9)
Urine <0.001 2.2 (1.7, 2.9)
Persistent stage 2/3
acute kidney injury
EDTA <0.001 3.8 (2.3, 6.3)
Heparin <0.001 3.9 (2.3, 6.3)
Urine <0.001 2.2 (1.6, 3.0)
Page 17 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
18
References
1 KDIGO Clinical Practice Guideline for Acute Kidney Injury. Kidney Int. 2012; 2(suppl 1). 2 McCullough PA, Shaw AD, Haase M, Bouchard J, Waikar SS, Siew ED, Murray PT, Mehta
RL, Ronco C. Diagnosis of acute kidney injury using functional and injury biomarkers:
workgroup statements from the tenth Acute Dialysis Quality Initiative Consensus Conference.
Contrib Nephrol. 2013;182:13-29. doi: 10.1159/000349963. PubMed PMID: 23689653.
3 Chang CH, Yang CH, Yang HY, Chen TH, Lin CY, Chang SW, Chen YT, Hung CC, Fang JT,
Yang CW, Chen YC. Urinary Biomarkers Improve the Diagnosis of Intrinsic Acute Kidney
Injury in Coronary Care Units. Medicine (Baltimore). 2015 Oct;94(40):e1703. doi:
10.1097/MD.0000000000001703. PubMed PMID: 26448023; PubMed Central PMCID:
PMC4616771.
4 Xu K, Rosenstiel P, Paragas N, Hinze C, Gao X, Huai Shen T, Werth M, Forster C, Deng R,
Bruck E, Boles RW, Tornato A, Gopal T, Jones M, Konig J, Stauber J, D'Agati V, Erdjument-
Bromage H, Saggi S, Wagener G, Schmidt-Ott KM, Tatonetti N, Tempst P, Oliver JA, Guarnieri
P, Barasch J. Unique Transcriptional Programs Identify Subtypes of AKI. J Am Soc Nephrol.
2016 Dec 27. pii: ASN.2016090974. doi: 10.1681/ASN.2016090974. [Epub ahead of print]
PubMed PMID: 28028135.
5 The NGAL Test
TM [package insert]. Gentofte, Denmark. BioPorto Diagnostics A/S, 2013.
6 U.S. Department of Health and Human Services, Food and Drug Administration, Center for
Devices and Radiological Health. 2007 March 13. Guidance for Industry and FDA Staff:
Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests.
Washington, DC: Author.
7 Schley G, Köberle C, Manuilova E, Rutz S, Forster C, Weyand M, et al. (2015) Comparison of
Plasma and Urine Biomarker Performance in Acute Kidney Injury. PLoS ONE 10(12):
e0145042. doi:10.1371/journal.pone.0145042
8 Clerico A, Galli C, Fortunato A, Ronco C. Neutrophil gelatinase-associated lipocalin (NGAL)
as biomarker of acute kidney injury: a review of the laboratory characteristics and clinical
evidences. Clin Chem Lab Med. 2012 Feb
15;50(9):1505-17. doi: 10.1515/cclm-2011-0814. Review. PubMed PMID: 22962216.
9 Devarajan P. Neutrophil gelatinase-associated lipocalin: a promising biomarker for human
acute kidney injury. Biomarkers in medicine. 2010;4(2):265-280. doi:10.2217/bmm.10.12.
10 Nickolas TL, O'Rourke MJ, Yang J, Sise ME, Canetta PA, Barasch N, Buchen C, Khan F,
Mori K, Giglio J, Devarajan P, Barasch J. Sensitivity and specificity of a single emergency
department measurement of urinary neutrophil gelatinase-associated lipocalin for diagnosing
Page 18 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
19
acute kidney injury. Ann Intern Med. 2008 Jun 3;148(11):810-9. PubMed PMID: 18519927;
PubMed Central PMCID: PMC2909852.
11 Macdonald S, Arendts G, Nagree Y, Xu X-F. Neutrophil Gelatinase-Associated Lipocalin
(NGAL) predicts renal injury in acute decompensated cardiac failure: a prospective
observational study. BMC Cardiovascular Disorders. 2012;12:8. doi:10.1186/1471-2261-12-8.
12 Zhang A, Cai Y, Wang PF, Qu JN, Luo ZC, Chen XD, Huang B, Liu Y, Huang WQ, Wu J,
Yin YH. Diagnosis and prognosis of neutrophil gelatinase-associated lipocalin for acute kidney
injury with sepsis: a systematic review and meta-analysis. Crit Care. 2016 Feb 16;20:41. doi:
10.1186/s13054-016-1212-x. PubMed PMID: 26880194; PubMed Central PMCID:
PMC4754917.
13 Marino R, Struck J, Hartmann O, Maisel AS, Rehfeldt M, Magrini L, Melander O, Bergmann
A, Di Somma S. Diagnostic and short-term prognostic utility of plasma pro-enkephalin (pro-
ENK) for acute kidney injury in patients admitted with sepsis in the emergency department. J
Nephrol. 2015 Dec;28(6):717-24. doi:10.1007/s40620-014-0163-z. Epub 2014 Dec 9. PubMed
PMID: 25486879.
14 Akrawinthawong K, Shaw MK, Kachner J, Apostolov EO, Basnakian AG, Shah S, Tilak J,
McCullough PA. Urine catalytic iron and neutrophil gelatinase-associated lipocalin as
companion early markers of acute kidney injury after cardiac surgery: a prospective pilot study.
Cardiorenal Med. 2013 Apr;3(1):7-16. doi: 10.1159/000346815. PubMed PMID: 23946721;
PubMed Central PMCID: PMC3743453.
15 Haase-Fielitz A, Haase M, Devarajan P. Neutrophil gelatinase-associated lipocalin as a
biomarker of acute kidney injury: a critical evaluation of current status. Annals of clinical
biochemistry. 2014;51(0 3):335-351. doi:10.1177/0004563214521795.
16 Filiopoulos V, Biblaki D, Vlassopoulos D. Neutrophil gelatinase-associated lipocalin (NGAL):
a promising biomarker of contrast-induced nephropathy after computed tomography. Ren Fail.
2014 Jul;36(6):979-86. doi: 10.3109/0886022X.2014.900429. Review. PubMed PMID:
24673459.
17 Haase M, Bellomo R, Devarajan P, Schlattmann P, Haase-Fielitz A; NGAL Meta-analysis
Investigator Group. Accuracy of neutrophil gelatinase-associated lipocalin (NGAL) in
diagnosis and prognosis in acute kidney injury: a systematic review and meta-analysis. Am J
Kidney Dis. 2009 Dec;54(6):1012-24. doi:10.1053/j.ajkd.2009.07.020. Epub 2009 Oct 21.
Review. PubMed PMID: 19850388.
18 Kashani K, Al-Khafaji A, Ardiles T, Artigas A, Bagshaw SM, Bell M, Bihorac A, Birkhahn
R, Cely CM, Chawla LS, Davison DL, Feldkamp T, Forni LG, Gong MN, Gunnerson KJ, Haase
M, Hackett J, Honore PM, Hoste EA, Joannes-Boyau O, Joannidis M, Kim P, Koyner JL,
Laskowitz DT, Lissauer ME, Marx G, McCullough PA, Mullaney S, Ostermann M, Rimmelé T,
Page 19 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
20
Shapiro NI, Shaw AD, Shi J, Sprague AM, Vincent JL, Vinsonneau C, Wagner L, Walker MG,
Wilkerson RG, Zacharowski K, Kellum JA. Discovery and validation of cell cycle arrest
biomarkers in human acute kidney injury. Crit Care. 2013 Feb 6;17(1):R25. doi:
10.1186/cc12503. PubMed PMID: 23388612; PubMed Central PMCID: PMC4057242.
19 Bihorac A, Chawla LS, Shaw AD, Al-Khafaji A, Davison DL, Demuth GE, Fitzgerald R,
Gong MN, Graham DD, Gunnerson K, Heung M, Jortani S, Kleerup E, Koyner JL, Krell K,
Letourneau J, Lissauer M, Miner J, Nguyen HB, Ortega LM, Self WH, Sellman R, Shi J,
Straseski J, Szalados JE, Wilber ST, Walker MG, Wilson J, Wunderink R, Zimmerman J,
Kellum JA. Validation of cell-cycle arrest biomarkers for acute kidney injury using clinical
adjudication. Am J Respir Crit Care Med. 2014 Apr 15;189(8):932-9. doi: 10.1164/rccm.201401-
0077OC. PubMed PMID: 24559465.
20 McCullough PA, Bouchard J, Waikar SS, Siew ED, Endre ZH, Goldstein SL, Koyner JL,
Macedo E, Doi K, Di Somma S, Lewington A, Thadhani R, Chakravarthi R, Ice C, Okusa MD,
Duranteau J, Doran P, Yang L, Jaber BL, Meehan S, Kellum JA, Haase M, Murray PT, Cruz D,
Maisel A, Bagshaw SM, Chawla LS, Mehta RL, Shaw AD, Ronco C. Implementation of novel
biomarkers in the diagnosis, prognosis, and management of acute kidney injury: executive
summary from the tenth consensus conference of the Acute Dialysis Quality Initiative (ADQI).
Contrib Nephrol. 2013;182:5-12. doi:10.1159/000349962. PubMed PMID: 23689652; PubMed
Central PMCID: PMC3856225.
Page 20 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1a
338x190mm (300 x 300 DPI)
Page 21 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1b
338x190mm (300 x 300 DPI)
Page 22 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 1c
338x190mm (300 x 300 DPI)
Page 23 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2a
338x190mm (300 x 300 DPI)
Page 24 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2b
338x190mm (300 x 300 DPI)
Page 25 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 2c
338x190mm (300 x 300 DPI)
Page 26 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3a
338x190mm (300 x 300 DPI)
Page 27 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3b
338x190mm (300 x 300 DPI)
Page 28 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Figure 3c
338x190mm (300 x 300 DPI)
Page 29 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Supplemental Table 1. Kidney Disease International Global Organization 2012 definitions of
acute kidney injury stages
Stage Serum Creatinine Urine output
1 1.5-1.9 times baseline OR ≥ 0.3 mg/dl
(≥26.5 µmol/l) increase
< 0.5 ml/hg/h for 6-12 hours
2 2.0-2.9 times baseline < 0.5 ml/kg/h ≥ 12 hours
3 3.0 times baseline OR increase in serum
creatinine to ≥ 4.0 mg/dl (≥ 353.6 µmol/l)
OR initiation of renal replacement
therapy OR in patients < 18 years,
decrease in eGFR to < 35 ml/min per
1.73 m2
< 0.3 ml/kg/h ≥ 24 hours OR Anuria for
≥ 12 hours
Page 30 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
Section & Topic No Item Reported on page #
TITLE OR ABSTRACT
1 Identification as a study of diagnostic accuracy using at least one measure of accuracy
(such as sensitivity, specificity, predictive values, or AUC)
1, 2
ABSTRACT
2 Structured summary of study design, methods, results, and conclusions
(for specific guidance, see STARD for Abstracts)
2-3
INTRODUCTION
3 Scientific and clinical background, including the intended use and clinical role of the index test 4
4 Study objectives and hypotheses 4
METHODS
Study design 5 Whether data collection was planned before the index test and reference standard
were performed (prospective study) or after (retrospective study)
4, 6
Participants 6 Eligibility criteria 4-5
7 On what basis potentially eligible participants were identified
(such as symptoms, results from previous tests, inclusion in registry)
4-5
8 Where and when potentially eligible participants were identified (setting, location and dates) 4-5
9 Whether participants formed a consecutive, random or convenience series 4-5
Test methods 10a Index test, in sufficient detail to allow replication 5
10b Reference standard, in sufficient detail to allow replication 4, 5, Sup. Table 1
11 Rationale for choosing the reference standard (if alternatives exist) 4, 5, Sup. Table 1
12a Definition of and rationale for test positivity cut-offs or result categories
of the index test, distinguishing pre-specified from exploratory
6
12b Definition of and rationale for test positivity cut-offs or result categories
of the reference standard, distinguishing pre-specified from exploratory
4, 5, 6
13a Whether clinical information and reference standard results were available
to the performers/readers of the index test
4, 5
13b Whether clinical information and index test results were available
to the assessors of the reference standard
4, 5
Analysis 14 Methods for estimating or comparing measures of diagnostic accuracy 7-8, Figures
15 How indeterminate index test or reference standard results were handled 7
16 How missing data on the index test and reference standard were handled 7
17 Any analyses of variability in diagnostic accuracy, distinguishing pre-specified from exploratory
18 Intended sample size and how it was determined 7
RESULTS
Participants 19 Flow of participants, using a diagram 7 – in text
20 Baseline demographic and clinical characteristics of participants 7, Table 1
21a Distribution of severity of disease in those with the target condition Table 1
21b Distribution of alternative diagnoses in those without the target condition Table 1
22 Time interval and any clinical interventions between index test and reference standard 5
Test results 23 Cross tabulation of the index test results (or their distribution)
by the results of the reference standard
Table 1
24 Estimates of diagnostic accuracy and their precision (such as 95% confidence intervals) 7-8, Figures
25 Any adverse events from performing the index test or the reference standard 7
DISCUSSION
26 Study limitations, including sources of potential bias, statistical uncertainty, and generalisability 11
27 Implications for practice, including the intended use and clinical role of the index test 8, 11-12
OTHER
28 Registration number and name of registry 13
29 Where the full study protocol can be accessed 5
30 Sources of funding and other support; role of funders 13
Page 31 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
For peer review only
STARD 2015
AIM
STARD stands for “Standards for Reporting Diagnostic accuracy studies”. This list of items was developed to contribute to the
completeness and transparency of reporting of diagnostic accuracy studies. Authors can use the list to write informative
study reports. Editors and peer-reviewers can use it to evaluate whether the information has been included in manuscripts
submitted for publication.
EXPLANATION
A diagnostic accuracy study evaluates the ability of one or more medical tests to correctly classify study participants as
having a target condition. This can be a disease, a disease stage, response or benefit from therapy, or an event or condition
in the future. A medical test can be an imaging procedure, a laboratory test, elements from history and physical examination,
a combination of these, or any other method for collecting information about the current health status of a patient.
The test whose accuracy is evaluated is called index test. A study can evaluate the accuracy of one or more index tests.
Evaluating the ability of a medical test to correctly classify patients is typically done by comparing the distribution of the
index test results with those of the reference standard. The reference standard is the best available method for establishing
the presence or absence of the target condition. An accuracy study can rely on one or more reference standards.
If test results are categorized as either positive or negative, the cross tabulation of the index test results against those of the
reference standard can be used to estimate the sensitivity of the index test (the proportion of participants with the target
condition who have a positive index test), and its specificity (the proportion without the target condition who have a negative
index test). From this cross tabulation (sometimes referred to as the contingency or “2x2” table), several other accuracy
statistics can be estimated, such as the positive and negative predictive values of the test. Confidence intervals around
estimates of accuracy can then be calculated to quantify the statistical precision of the measurements.
If the index test results can take more than two values, categorization of test results as positive or negative requires a test
positivity cut-off. When multiple such cut-offs can be defined, authors can report a receiver operating characteristic (ROC)
curve which graphically represents the combination of sensitivity and specificity for each possible test positivity cut-off. The
area under the ROC curve informs in a single numerical value about the overall diagnostic accuracy of the index test.
The intended use of a medical test can be diagnosis, screening, staging, monitoring, surveillance, prediction or prognosis. The
clinical role of a test explains its position relative to existing tests in the clinical pathway. A replacement test, for example,
replaces an existing test. A triage test is used before an existing test; an add-on test is used after an existing test.
Besides diagnostic accuracy, several other outcomes and statistics may be relevant in the evaluation of medical tests. Medical
tests can also be used to classify patients for purposes other than diagnosis, such as staging or prognosis. The STARD list was
not explicitly developed for these other outcomes, statistics, and study types, although most STARD items would still apply.
DEVELOPMENT
This STARD list was released in 2015. The 30 items were identified by an international expert group of methodologists,
researchers, and editors. The guiding principle in the development of STARD was to select items that, when reported, would
help readers to judge the potential for bias in the study, to appraise the applicability of the study findings and the validity of
conclusions and recommendations. The list represents an update of the first version, which was published in 2003.
More information can be found on http://www.equator-network.org/reporting-guidelines/stard.
Page 32 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on July 11, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2017-016028 on 10 July 2017. Dow
nloaded from
Top Related