Analysis and Optimization of a Sepsis ClinicalPathway using Process Mining

R.A. Quintano Neira1,2, B.F.A. Hompes2,3, J.J.G. de Vries2, B.F. Mazza4,S.L. Simoes de Almeida4, E. Stretton2, J.C.A.M. Buijs3, and S. Hamacher1

1 Industrial Engineering Department,Pontifıcia Universidade Catolica do Rio de Janeiro, Rio de Janeiro, Brazil

2 Philips Research, Eindhoven, The Netherlands3 Department of Mathematics and Computer Science,

Eindhoven University of Technology, Eindhoven, The Netherlands4 Hospital Samaritano de Sao Paulo, Sao Paulo, Brazil

ricardo.quintano@philips.com

Abstract. In this work, we propose and apply a methodology for themanagement and optimization of clinical pathways using process mining.We adapt the Clinical Pathway Analysis Method (CPAM) by takinginto consideration healthcare providers’ needs. We successfully appliedthe methodology in the sepsis treatment of a major Brazilian hospital.Using data extracted from the hospital information system, a total of5,184 deviations in the execution of the sepsis clinical pathway werediscovered and categorized in 43 different types. We identified the processas it was actually executed, two bottlenecks, and significant differences inperformance in cases that deviated from the prescribed clinical pathway.Furthermore, factors such as patient age, gender, and type of infectionwere shown to affect performance. The analysis results were validated byan expert panel of clinical professionals and verified to provide valuable,actionable insights. Based on these insights, we were able to suggestoptimization points in the sepsis clinical pathway.

Keywords: Clinical Pathways · Sepsis · Process Analytics

1 Introduction

Clinical Pathways (CPs) consist of well-defined care plans which describe theordering and timing of interventions and their expected outcomes. Healthcareproviders typically model their CPs after evidence-based medical guidelinessupplied by communities such as the Society of Critical Care Medicine, promotingthe improvement and standardization in patient care while reducing expensesand consumption of resources [4,13,25,39]. The correct implementation of CPs isknown to promote positive outcomes such as early disease recognition, correcttreatment, reduction of mortality and length of stay, and hospital costs [24].However, the management of CPs has proven to be a challenging task. Timeconstraints, lack of qualified resources, administrative burdens, and absence

2 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

of tools for evaluating and reporting are common contributing factors [8,19].The analysis of data extracted from hospital information systems provides aninteresting direction for the optimization efforts of healthcare professionals.

Sepsis is defined as a “life-threatening organ dysfunction caused by a dysregu-lated host response to infection” [32], and is considered a worldwide health issueand economic burden. The mortality rates from different countries varied in awide range, but can go as high as 64.5% [12,22,9,21,30,34,6,27]. Chalupka andTalmor found that the mean costs per case ranged from USD 4,888 (Argentina)to USD 103,529 (United States) [7]. Improving performance and compliance tosepsis CPs can simultaneously save lives and reduce financial impact.

Process mining is a research field concerned with the analysis of businessprocesses using data extracted from information systems [1,38,23]. It combinesmodel-based process analysis with data-driven analysis techniques. It enablesautomatic extraction of process models (process discovery), recognition of com-pliance to protocol (conformance checking), performance analysis, and manyother applications. Process mining has been successfully applied to the health-care domain in several studies [31,37,23]. For example, Augusto et al. presentedan automated method to simulate and evaluate CPs in a severe heart failurecase study [3]. Fernandez-Llatas et al. demonstrated the use of process miningtechniques to support the analysis of stroke cases in the emergency room [11].

In [5], Caron et al. presented the Clinical Pathway Analysis Method (CPAM).This method enables the acquisition of insights regarding CPs using event dataand process mining techniques. It is a flexible method supporting a plethoraof analyses such as workflow discovery, social network analysis, task allocation,and bottleneck analysis. In this paper, we adapt CPAM based on the needs andchallenges of the healthcare professionals, and apply it to the sepsis CP of HospitalSamaritano, a large private Brazilian hospital located in Sao Paulo. The proposedmethodology – entitled Simplified Clinical Pathway Analysis Method (SCPAM) –was intentionally kept simple to facilitate its adoption by healthcare organizationsand helps answer the following set of questions: How does the actual executedprocess look like? What are the deviations from protocol in the execution of theCP? What is the performance of the CP? What are the performance bottlenecksin the CP? Which factors affect the performance?, and How to optimize the CP?

The remainder is organized as follows. We describe the analyzed sepsis CPin section 2. In section 3 we describe the analysis performed according to theSCPAM methodology and the obtained results. In section 4 we discuss thevalidation of results by medical experts. In section 5 we conclude and discussfuture perspectives.

2 Adult Emergency Department Sepsis Clinical Pathway

Hospital Samaritano de Sao Paulo is a large private hospital in Brazil with overthree hundred beds. It stands out for excellence in healthcare [18,29]. One of themain CPs of the hospital is the sepsis pathway. Figure 1 depicts the sepsis CP

Analysis and Optimization of a Sepsis Clinical Pathway 3

Fig. 1. Prescriptive sepsis treatment process model of the emergency department,modeled using Business Process Model and Notation (BPMN).

as it should be executed in the adult emergency department. The process wasmapped and validated by interviewing the hospital staff during two visits.

The process starts when a patient arrives in the emergency department andreceives a queue number (activity 1). The patient waits until a receptionistcalls them. The receptionist registers the patient’s admission in the hospitalinformation system (2). The patient then waits at the reception until a nursecalls them to start the triage. During the triage, the nurse measures the patient’svital signs (temperature, blood pressure, heart rate and respiratory rate) (3),asks specific questions to the patient, prioritizes the patient according to theirseverity level, and registers all the triage information in the Electronic HealthRecord (EHR) (4). If the nurse suspects that the patient has sepsis, they muststart the formal sepsis pathway (5A). The nurse must fill out a specific sepsis CPpaper-based form, must register a clinical note in the EHR that the sepsis CP wasstarted and must immediately communicate with the first available physician thatthe patient is sepsis suspicious. The physician evaluates the patient (anamnesisand physical examination) and records all information in the clinical notes (6). Ifthe physician defines a sepsis diagnostic hypothesis, then antibiotics and volumeexpansion must be prescribed (7 and 9) and blood culture and lactate exams mustbe requested (8 and 10). If the sepsis CP was not formally started previously, thenthe physician must do so (5B). Finally, the patient goes to the medication roomand a nurse technician collects their blood for the blood culture and lactate exams(11 and 13) and administers the antibiotics (14) and the volume expansion (12).The designed CP represents all sepsis treatment steps until the administration ofantibiotics as described in the first bundle of the Surviving Sepsis Campaign [33].

4 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

3 Clinical Pathway Analysis and Optimization

In order to support the management and evaluation of the sepsis CP, we developedand applied the SCPAM methodology (see Figure 2), which is a simplification ofCPAM and aims to facilitate adoption by healthcare organizations. The maindifferences between the two methodologies are that SCPAM predefines a setof research questions, process perspectives, PM analyses and their executionorder and that it presents detailed instructions on how to perform each step andanalysis. In this work, all process mining analyses were conducted using ProMand Disco and statistical analyses were performed using R1. The results of thiscase study were validated by six healthcare professionals who all actively work inthe sepsis CP, using interviews and structured questionnaires.

Fig. 2. Overview of the SCPAM methodology followed to analyze the sepsis CP.

3.1 Defining the Project

In the first step we define the scope of the project (i.e. the CP). A prescriptiveCP model is created, mapping the selection and order of activities to be executedfor the treatment of a given disease. This model guides the data extraction andis used to identify CP compliance and to create the AS-IS model.

In this study, we focus on the sepsis CP described in section 2. We areinterested in the analysis of the compliance and performance of the process. Forexample, the administration of antibiotics must happen after the collection ofthe blood culture and, as prescribed by the hospital, this activity must happenwithin one hour after the sepsis presentation. The faster the patient receives theantibiotics, the greater their chance of survival [20].

3.2 Extracting and Pre-processing the Event Log

The aim of the second step is to extract all necessary data from support systemsand prepare the event log for analysis. This step is composed of the followingactivities. First, for each activity in the prescriptive CP model, the attributesthat are relevant for the analysis are identified (e.g. date and time of execution

1 See https://promtools.org, https://fluxicon.com, and https://r-project.org

Analysis and Optimization of a Sepsis Clinical Pathway 5

of the activity, the age of patient). Next, the selected attributes are mapped tothe available systems databases (e.g. hospital information system or EHR) andthe relevant data is extracted. For most process mining analyses, the minimal setof attributes per recorded event consists of the case identifier (e.g. the admissionor patient), the executed activity (e.g. triage, administration of medicine), andthe order in which the activities were executed (date and time). The event logis converted to the XES format supported by process mining tools [2,36]. Toolssuch as Disco and ProM provide this conversion. Ideally, the event log shouldbe as simple as possible. Duplicated or otherwise unnecessary activities for theintended analysis should be removed.

We extracted two years of hospitalizations of patients either diagnosed withsepsis or who died of sepsis, identified through the list of International StatisticalClassification of Diseases and Related Health Problems tenth revision (ICD-10)sepsis codes provided by the Canadian Institute for Health Information [9]. Casesin which the sepsis prescription template was used were added as well. All selectedpatients are over eighteen years of age, have two or more Systemic InflammatoryResponse Syndrome (SIRS) signals, and are identified as sepsis suspicious by aphysician. We enriched the event log with the following attributes: patient age,divided into 8 groups ([18-26], [27-35], [36-44], [45-53], [54-62], [63-71], [72-80]and [≥ 81]), patient gender, patient priority (following the Manchester triagesystem), type of infection (either community or hospital-acquired), presenceof SIRS signal for each of the six clinical parameters (temperature, chills andshivering, respiratory rate, heart rate, leukocytes, and rod cells percentage), andtotal number of SIRS signals present. This information was translated to caseattributes. The extracted data was converted to an event log that contains allcomplete sepsis cases that started and ended in the emergency department, aswell as cases that started in the emergency department and then were transferredto the wards. Regarding the event log creation, our main challenges were: therewas no registry of volume expansion in the EHR (activities 9 and 12) - we createdthese events taking in consideration the amount of saline solution prescribed toeach patient; event times were derived from the time that the action was enteredin the system rather than the time that the action was actually performed (thelatter one was rarely available in the EHR) - in our visits to the hospital weobserved that the staff registers the information in a very close time of executingthe action; registry of the collection of blood and the administration of antibioticswere in most cases performed in the same time - we added one minute to allevents of activity 14. The resulting event log contains 1,710 cases, 20,605 events,14 activities (those described in Figure 1) and 292 unique trace variants (activityorderings). The granularity of time is in minutes. The extraction phase is furtherdetailed in previous work [28].

3.3 Identifying the Adherence to the Clinical Pathway

The aim of the third step is to identify how well the process was actually executed,i.e. the identification of potential deviations in the execution with regards tothe prescribed CP. The deviations can be identified using conformance checking

6 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

tools (e.g. the “Replay a Log on Petri Net for Conformance Analysis” pluginin ProM), by discovering the AS-IS model and comparing it manually to theprescribed CP, or by looking for events performed by professionals with differentroles than expected (e.g. a receptionist registering the triage instead of a nurse).Conformance checking results are typically expressed using measures such asfitness and precision. Trace fitness for example is represented by a number betweenzero and one that indicates how well a case can be replayed by the model. Thehigher the fitness, the more compliant is the executed process.

Confronting the event log with the sepsis CP model defined in section 2, weidentified an average trace fitness of 0.85, indicating that the process as performedin the hospital is reasonably close to the one defined in its CP. In our study,deviations were associated to sub-sequences executed in a different order thanexpected, activities not executed, activities executed by different roles, and thenon-compliance to the target of one hour to give the antibiotics. As such, asingle case may exhibit multiple deviations. We identified 43 different types ofdeviations in the execution of the CP, constituting 5,184 deviation instances.The non-prescription and non-administration of volume expansion were amongthe most frequent type of deviations (89% of cases), followed by the start of theformal CP during triage (45%).

3.4 Identifying the AS-IS Process

The aim of the fourth step is to identify the process as it is actually executed.There are several process discovery algorithms that can be used to automaticallydiscover a process model from the event log. However, due to the complexity ofhealthcare processes and the variation typically found in the data extracted fromhospital information systems, these algorithms typically generate models that arenot easily understandable by clinicians. Furthermore, such discovered models tendto differ substantially from the prescriptive CPs, complicating further analysis.As such, we recommend manually and iteratively updating the prescribed CP byincorporating the most frequent deviations identified in the previous step andcalculating the trace fitness with respect to the updated model. The incorporationof deviations into the CP should be done in such a manner to bring the tracefitness as close to one as possible, without sacrificing other quality metrics, mostnotably simplicity. When only few cases deviate substantially from the processmodel, it is better to sacrifice fitness in favor of a more legible process modelwith less activities or infrequent paths.

To obtain a process model describing how the sepsis process was reallyexecuted, we manually adjusted the CP model by incorporating the most frequenttypes of deviations and observed behaviour from the event log. The adjustedmodel has an improved average trace fitness of 0.97, and can be seen in Figure 3.The differences between the prescribed CP and the AS-IS model are as follows.The first measure of vital signs (activity 3) usually happens before the registrationof triage (4), rather than in parallel with it. Furthermore, the formal start of thesepsis pathway (5A) usually happens during triage. Lastly, the prescription andadministration of volume expansion (9 and 12) are not always executed.

Analysis and Optimization of a Sepsis Clinical Pathway 7

Fig. 3. AS-IS sepsis treatment process model of the emergency department, modeledusing Business Process Model and Notation (BPMN).

3.5 Measuring Performance and Identifying Bottlenecks

In the fifth step, we measure the performance of the process and identify whichfactors contribute most to any identified bottlenecks. The step is composed ofthe following activities. First, the relevant performance measures and metricsare defined. Again, several process mining techniques exist that measure theperformance using an event log and a process model (e.g. the “Replay a Logon Petri Net for Performance/Conformance Analysis” plugin in ProM, andthe lifecycle-based technique presented in [14], which allows to define customperformance metrics). Next, each metric should be carefully analyzed by answeringquestions such as Does this waiting time make sense for this specific activity?, Isthe duration for this selected set of patients significantly different?. A techniqueto interactively analyze healthcare process performance using visual analyticsis presented and applied in [10,17]. High waiting times often indicate potentialbottlenecks. For example, while an average waiting time of three hours for theadministration of medicines is not out of the ordinary, one hour is a long waitingtime for a medical encounter with a physician in the emergency department, andas such might indicate a bottleneck. Contextual factors that affect performanceshould be identified as well. Process performance is generally affected by a plethoraof factors, such as the amount of scheduled staff, the number of patients undercare, or the clinical parameters of a patient [15]. The identification of these factorscan support clinicians in tailoring their CPs. The ProM plugin “Find context-based differences in process performance” supports this type of analysis [15,17]. Ittries to automatically discover statistically significant differences in performanceby looking at the context. Similarly, causal relations between factors can beidentified [16]. We further recommend removing outlying cases as they may affectperformance statistics.

8 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

To analyse the performance of the sepsis process, we looked at the duration ofactivities, the time between activities, and the total case duration. We removedcases with outlier performance using Tukey’s method [35] and did not considerthose pairs of activities with a case frequency less than 5% (for example, activity“13. Lactate Collection” was only followed by “7. Prescription of Antibiotic” in0.8% of cases). We identified two potential bottlenecks. Firstly, patients waitingin the reception before triage wait an average of 18 minutes between getting aqueue number (activity 1) and getting their first measure of vital signs (3). Next,between the registry of clinical notes (6) and treatment prescription (7, 8, and10), physicians spent an average of 5 minutes. These two bottlenecks may seemto represent a small amount of time. However, compared to the target time toadministrate antibiotics – which is one hour – they represent a significant amountof time (30% and 8% respectively).

In order to identify contextual factors correlated with performance, we dividedthe process in six phases: waiting in reception, triage, waiting for medical evalu-ation, medical evaluation, treatment, and observation. Again, Tukey’s methodwas used to remove outliers for each phase. We discovered that the triage phasetakes significantly more time for cases that started during a period of mediumor high demand compared to when the emergency department was not busy.Female patients and patients in the age groups [72-80] and [≥ 81] spent more timein the observation phase compared to male and younger patients, respectively.Patients that showed four SIRS signals and patients with a SIRS signal relatingto respiratory rate or that had an infection acquired in the hospital also exhibiteda significantly longer observation time.

3.6 Improving the Clinical Pathway

The aim of this step is to update the prescriptive CP to optimize its expectedperformance and outcome in terms of mortality, length of stay, costs, etc. Usingthe deviations and the performance results identified in the previous steps, thosedeviations that can improve the CP are identified. Per deviation, we applythe appropriate statistical tools. In case the identified deviation correlates withimproved performance or favorable outcomes, the healthcare provider may updatethe prescribed CP, incorporating those positive deviations. Alternative availableapproaches can be applied to improve the CP, such as the Multi-Criteria AnalysisTechnique [26] that identifies activities and sub-sequences that contribute in apositive or negative way to the outcomes. This technique can be very useful whenmultiple simultaneous criteria need to be considered in the optimization (e.g.minimize the length of stay and mortality rate).

In our study, the analysis of the impact on performance outcomes of theidentified deviations indicated that sepsis cases in which the registration of clinicalnotes occurs after the prescription of medicine and request of exam, spent onaverage 3.5 minutes less time waiting for the administration of antibiotics, witha median of 3 minutes1.

1 Two-tailed Mann-Whitney U test: U = 261, 472.5; n1 = 366; n2 = 1, 199; p < 0.001.

Analysis and Optimization of a Sepsis Clinical Pathway 9

4 Validating results with Hospital Staff

We performed the validation of the results with 3 physicians, 2 nurses, and 1quality analyst who all actively work in the sepsis CP. The validation happened in2017 by means of a group interview and structured questionnaires. We validatedthe AS-IS process, four deviations, two bottlenecks, five of the context factorscorrelated to performance, and the deviation that could potentially optimizethe execution of the CP. The panel considered three deviations as real, actualdeviations (registry of clinical notes after prescription of treatment, cases withoutprescription of volume expansion, and antibiotic not being administrated until onehour after identification). The panel considered blood culture being requested bya nurse technician as invalid since in practice, this should never happen. The panelvalidated that the AS-IS model in Figure 3 indeed represents the process as it isactually executed in the emergency department. Concerning the bottlenecks, thepanel considered both as real. Furthermore, they did not provide any additionalbottleneck from their own experience. Regarding contextual factors, the expertpanel agreed with three results, was neutral regarding the finding that the caseduration is higher for patients with a heart rate < 90 BPM, and disagreed withthe finding that female patients tend to stay in observation longer. The panelagreed that prescribing the treatment before registering the clinical notes clearlyreduces the administration target time since the delivery of medication processstarts early. However, in general, they believe that the CP should not be updatedsince this deviation is associated with severe patients. All professionals consideredit important to have access to the outcomes presented in this research as theyhelp hospital staff identify problems in the execution of the sepsis CP. For them,this is a key feature in managing their CP. Regarding the bottleneck of patientswaiting in the reception, the panel suggested a simplification in the registration oftriage information. In addition, the process will be updated such that the triageis performed before the registration of the patient. This will help the hospitalto quickly identify sepsis suspicious patients. With respect to the bottleneck ofthe prescription of the treatment, the panel suggested a simplification in theregistration of the prescription.

5 Conclusions and Further Considerations

The employment of Clinical Pathways (CP) is a viable direction towards improvingoutcomes of medical processes. Yet, the management and evaluation of theexecution of CPs presents multiple challenges. In this work, we adapted a data-based method employing process mining techniques in order to support themanagement and optimization of CPs and applied it to a sepsis process ofa large Brazilian hospital. We consider the results of this study to be quitepromising. Firstly, a simplified methodology can aid healthcare providers inthe management and optimization of their CPs as was confirmed by structuredinterviews and questionnaires with a panel of medical experts. Secondly, we wereable to successfully apply the methodology to real-life data from a sepsis process

10 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

and discover deviations from protocol, performance bottlenecks and contextualfactors that influence performance, leading to improvement suggestions.

As future steps, we suggest the application of SCPAM method in differenthealthcare facilities and for different diseases to verify its generalization andscalability. We believe SCPAM is particularly promising for the evaluation ofCPs in the ED such as stroke and acute chest pain. SCPAM could be used bycommunities (e.g. the Latin American Sepsis Institute) to create an overviewof the treatment of the diseases as executed in different healthcare facilities,supporting the update, management and deployment of clinical guidelines andclinical pathways. A prerequisite for the replication of this research study indifferent contexts is that the registration of healthcare information (e.g. triage,medical evaluation, prescription) is done using integrated systems, allowing thecreation and extraction of the event data.

Acknowledgments. The authors thank A. Medeiros, D. Brizida Dreux, M. San-tos, R. da Silva Santos, S. Barbosa, W.M.P. van der Aalst and all professionalsinvolved from Hospital Samaritano and Philips for their help in the developmentof this research. This study was partly financed by the Coordenacao de Aper-feicoamento de Pessoal de Nıvel Superior - Brasil (CAPES) - Finance Code 001,Conselho Nacional de Desenvolvimento Cientıfico e Tecnologico - CNPq [grantnumbers 140511/2018-0, 306802/2015-5, 403863/2016-3], and Philips Research.

References

1. van der Aalst, W.M.P.: Process Mining - Data Science in Action, Second Edition.Springer (2016)

2. Acampora, G., Vitiello, A., Di Stefano, B., van der Aalst, W.M.P., Gunther, C.W.,Verbeek, E.: IEEE 1849TM: The XES Standard. IEEE Computational IntelligenceMagazine pp. 4–8 (2017)

3. Augusto, V., Xie, X., Prodel, M., Jouaneton, B., Lamarsalle, L.: Evaluation ofdiscovered clinical pathways using process mining and joint agent-based discrete-event simulation. In: Winter Simulation Conference, WSC 2016, Washington, DC,USA. pp. 2135–2146. IEEE (2016)

4. Baker, K., et al.: Process mining routinely collected electronic health records todefine real-life clinical pathways during chemotherapy. I. J. Medical Informatics 103,32–41 (2017)

5. Caron, F., Vanthienen, J., Vanhaecht, K., Van Limbergen, E., Deweerdt, J., Baesens,B.: A process mining-based investigation of adverse events in care processes. HealthInformation Management Journal 43(1), 16–25 (2014)

6. Carrillo-Esper, R., Carrillo-Cordova, J.R., Carrillo-Cordova, L.D.: Epidemiologicalstudy of sepsis in Mexican intensive care units. Cirugia y Cirujanos 77(4), 301–8(2009)

7. Chalupka, A.N., Talmor, D.: The economics of sepsis. Critical Care Clinics 28(1),57–76 (2012)

8. Chawla, A., Westrich, K., Matter, S., Kaltenboeck, A., Dubois, R.: Care pathwaysin US healthcare settings: current successes and limitations, and future challenges.Am J Manag Care 22(1), 53–62 (2016)

Analysis and Optimization of a Sepsis Clinical Pathway 11

9. CIHI: Canadian Institute for Health Information: In focus: A national look at sepsis(2009), https://secure.cihi.ca/estore/productFamily.htm?&pf=PFC1564

10. Dixit, P.M., Garcia Caballero, H.S., Corvo, A., Hompes, B.F.A., Buijs, J.C.A.M.,van der Aalst, W.M.P.: Enabling interactive process analysis with process mining andvisual analytics. In: BIOSTEC: HEALTHINF, Porto, Portugal. vol. 5, pp. 573–584.SciTePress (2 2017)

11. Fernandez-Llatas, C., et al.: Analyzing medical emergency processes with pro-cess mining: The stroke case. In: International Conference on Business ProcessManagement. pp. 214–225. Springer (2018)

12. Finfer, S., Bellomo, R., Lipman, J., French, C., Dobb, G., Myburgh, J.: Adult-population incidence of severe sepsis in Australian and New Zealand intensive careunits. Intensive Care Medicine 30(4), 589–596 (2004)

13. Fujino, Y., et al.: Impact of regional clinical pathways on the length of stay inhospital among stroke patients in Japan. Medical Care 52(7), 634–640 (2014)

14. Hompes, B.F.A., van der Aalst, W.M.P.: Lifecycle-based process performance anal-ysis. In: OTM Confederated International Conferences ”On the Move to MeaningfulInternet Systems”, Valletta, Malta. Lecture Notes in Computer Science, pp. 336–353.Springer (2018)

15. Hompes, B.F.A., Buijs, J.C.A.M., van der Aalst, W.M.P.: A generic framework forcontext-aware process performance analysis. In: OTM Confederated InternationalConferences ”On the Move to Meaningful Internet Systems”, Rhodes, Greece. pp.300–317 (2016)

16. Hompes, B.F.A., Maaradji, A., La Rosa, M., Dumas, M., Buijs, J.C.A.M., van derAalst, W.M.P.: Discovering causal factors explaining business process performancevariation. In: International Conference on Advanced Information Systems Engineering.pp. 177–192. Springer (2017)

17. Hompes, B., Dixit, P., Buijs, J.: Using process analytics to improve healthcareprocesses. Data Science for Healthcare - Methodologies and Applications, pp. 305–325.Springer (2019)

18. Hospital Samaritano, Com 125 anos de atividades, o Hospital SamaritanoHigienopolis, destaca-se pela excelencia e humanizacao no atendimento a saude.(2019), http://samaritano.com.br/institucional/institucional/

19. Khalifa, M., Alswailem, O.: Clinical pathways: Identifying development, implemen-tation and evaluation challenges. In: ICIMTH, pp. 131–134. IOS Press (2015)

20. Kumar, A., et al.: Duration of hypotension before initiation of effective antimicrobialtherapy is the critical determinant of survival in human septic shock. Critical CareMedicine 34(6), 1589–1596 (2006)

21. Liao, X., Du, B., Lu, M., Wu, M., Kang, Y.: Current epidemiology of sepsis inmainland China. Annals of Translational Medicine 4(17) (2016)

22. Machado, F.R., et al.: The epidemiology of sepsis in Brazilian intensive care units(the Sepsis PREvalence Assessment Database, SPREAD): an observational study.The Lancet Infectious Diseases 17(11), 1180–1189 (2017)

23. Mans, R.S., van der Aalst, W.M.P., Vanwersch, R.J.B.: Process Mining in Healthcare- Evaluating and Exploiting Operational Healthcare Processes. Springer Briefs inBusiness Process Management, Springer (2015)

24. Palleschi, M.T., Sirianni, S., O’Connor, N., Dunn, D., Hasenau, S.M.: An interpro-fessional process to improve early identification and treatment for sepsis. Journal forHealthcare Quality 36(4), 23–31 (2014)

25. Panella, M., Marchisio, S., Di Stanislao, F.: Reducing clinical variations with clinicalpathways: do pathways work? International Journal for Quality in Health Care 15(6),509–521 (2003)

12 R.A. Quintano Neira, B.F.A. Hompes, J.J.G. de Vries et al.

26. Quintano Neira, R.A.: A multi-criteria process mining optimization tool and itsapplication in a sepsis clinical pathway. Ph.D. thesis, Pontifıcia Universidade Catolicado Rio de Janeiro, Rio de Janeiro (2018)

27. Quintano Neira, R.A., Hamacher, S., Japiassu, A.M.: Epidemiology of sepsis inBrazil: Incidence, lethality, costs, and other indicators for Brazilian Unified HealthSystem hospitalizations from 2006 to 2015. PLoS ONE 13(4) (2018)

28. Quintano Neira, R.A., de Vries, G., Caffarel, J., Stretton, E.: Extraction of data froma hospital information system to perform process mining. In: MEDINFO, Hangzhou,China. Studies in Health Technology and Informatics, vol. 245, pp. 554–558. IOSPress (2017)

29. Revista exame: Os hospitais brasileiros de excelenciaem 2014, https://exame.abril.com.br/seu-dinheiro/

os-hospitais-brasileiros-de-excelencia-em-2014

30. Rodrıguez, F., et al.: The epidemiology of sepsis in Colombia: a prospective multicen-ter cohort study in ten university hospitals. Critical Care Medicine 39(7), 1675–1682(2011)

31. Rojas, E., Munoz-Gama, J., Sepulveda, M., Capurro, D.: Process mining in health-care: A literature review. Journal of Biomedical Informatics 61, 224–236 (2016)

32. Singer, M., et al.: The third international consensus definitions for sepsis and septicshock (sepsis-3). Jama 315(8), 801–810 (2016)

33. Society of Critical Care Medicine: Surviving sepsis campaign, http://www.

survivingsepsis.org

34. The EPISEPSIS Study Group: EPISEPSIS: A reappraisal of the epidemiology andoutcome of severe sepsis in French intensive care units. Intensive Care Medicine30(4), 580–588 (2004)

35. Tukey, J.: Exploratory data analysis. Reading: Addison-Wesley (1977)36. XES Working Group: IEEE Standard for eXtensible Event Stream (XES) for

achieving interoperability in event logs and event streams. IEEE Std 1849-2016 pp.1–50 (2016)

37. Yang, W., Su, Q.: Process mining for clinical pathway: Literature review and futuredirections. In: 2014 11th International Conference on Service Systems and ServiceManagement (ICSSSM). pp. 1–5 (2014)

38. Yoo, S., et al.: Assessment of hospital processes using a process mining technique:Outpatient process analysis at a tertiary hospital. I. J. Medical Informatics 88, 34–43(2016)

39. Zhang, Y., Padman, R., Patel, N.: Paving the COWpath: Learning and visual-izing clinical pathways from electronic health record data. Journal of BiomedicalInformatics 58, 186–197 (2015)