Agile development model for certifiable medical device software
99
FACULDADE DE E NGENHARIA DA UNIVERSIDADE DO P ORTO Agile development model for certifiable medical device software Erica E. F. Lima Mestrado em Engenharia de Software Supervisor: Professor Doctor Gil Gonçalves October 27, 2020
Transcript of Agile development model for certifiable medical device software
FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO
Agile development model for certifiablemedical device software
Erica E. F. Lima
Mestrado em Engenharia de Software
Supervisor: Professor Doctor Gil Gonçalves
October 27, 2020
Agile development model for certifiable medical devicesoftware
Erica E. F. Lima
Mestrado em Engenharia de Software
October 27, 2020
Abstract
This study aims to illustrate the current status when the subject is medical device software usingagile. Introduces the needs of the regulation and the standards to be a certifiable medical devicesoftware and how this is the safer and quality way to do it.
The problem that many companies feel obligated to use the waterfall still when it is a projecttoo complex or have many regulations as the only way to build software. The medical area is notdifferent; it is challenging to put agile and still comply with all regulations and standards.
This dissertation illustrates efforts that others have already done using agile and try to find thebest way to proceed by validation, through an academic case and interviews and finally contributewith a final proposed process.
Keywords: Agile development, Agile methodologies, Medical device, Scrum, Process, Reg-ulation, Software life cycle processes, Risk Management, Software Process Improvement (SPI),FDA, Medical device industry.
i
ii
Resumo
Este estudo tem como objetivo ilustrar como está o estado atual quando o assunto é software dedispositivo médico utilizando agile. Apresenta as necessidades dos regulamentos e padrões paraser um software de dispositivo médico certificável e como essa é a maneira mais segura e dequalidade de fazê-lo.
O problema que muitas empresas se sentem obrigadas a usar a cachoeira ainda quando é umprojeto muito complexo ou tem muitas regulamentações como única forma de construir software.A área médica não é diferente; é um desafio ser ágil e ainda cumprir todos os regulamentos epadrões.
Este artigo irá ilustrar alguns trabalhos que outros já fizeram utilizando agile e tentar encontrara melhor forma de proceder por validação, através de um caso acadêmico e entrevistas e finalmentecontribuir com uma proposta de processo final.
Keywords: Desenvolvimento Ágil, Metodologias Ágeis, Dispositivos médicos, Scrum, Processo,Regulamentação, Processos do ciclo de vida de software, Gerenciamento de riscos, Melhoria deprocessos de software (SPI), FDA, Indústria de dispositivos médicos.
iii
iv
Acknowledgements
I dedicate my first thanks to my advisor, Gil Gonçalves, whose experience was invaluable in for-mulating research and methodology questions. He supported me with the topic from the beginningand helped me have discipline and guidance to finish the thesis.
I thank all the Master’s professors of FEUP, who made me learn even more and made me abetter professional.
I also thank Samira Tavares, Analia Irigoyen, and Carlos Rodriguez. They helped me withmeaningful advice and feedback that improve my final process by participating in the validationinterview. I also want to thank all academic case team members who welcomed me as a teammember and helped me with this research.
In addition, I would like to thank my parents and hope that they are proud of me once againwith this study. I’m also grateful to my fiancé, João Miranda, whom I thank for choosing to embarkwith me on this new adventure that was the Masters, and also for all the help to stay focused tofinalize this thesis. Finally, thanks to all my family and friends, who have always supported meand my choices.
Erica Lima
v
vi
“Continuous improvement is not about the things you do well -that’s work.
Continuous improvement is about removing the things that get in the way of your work.The headaches, the things that slow you down,
tha’s what continuous improvement is all about. ”
Bruce Hamilton
vii
viii
Contents
1 Introduction 11.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Document Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 State of the art - Regulation 52.1 Medical Devices Regulation in EU . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Medical Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Software as medical device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.4 Medical device standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4.1 Traceability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.4.2 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 State of the art - Process and Methodologies 133.1 Agile Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Scrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.3 Extreme Programming (XP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.4 Test Driven Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5 Behavior Driven Development (BDD) . . . . . . . . . . . . . . . . . . . . . . . 183.6 Compliance with IEC 62304:2006 . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.6.1 Configuration Management . . . . . . . . . . . . . . . . . . . . . . . . 183.6.2 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.6.3 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.6.4 Software Life Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.7 V-model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.8 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.8.1 CochlearTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.8.2 MDevSPICE R© . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.8.3 AV-Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4 Previous Work 294.1 Initial Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.2 Compliance with standard IEC 62304:2006 . . . . . . . . . . . . . . . . . . . . 30
4.2.1 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.2 Configuration Management . . . . . . . . . . . . . . . . . . . . . . . . 314.2.3 Traceability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.4 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.2.5 Audit Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
ix
x CONTENTS
4.3 Limitations to validate the process . . . . . . . . . . . . . . . . . . . . . . . . . 324.4 Proposals for the limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.5 Validation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5 Academic Case 355.1 The project - Hive Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.1 Academic Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2 Process Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385.2.2 Adaptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6 Interviews 436.1 Validation Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.1.1 Samira’s Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446.1.2 Analia’s Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.1.3 Carlos’s Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
7 Solution 537.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537.2 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.2.1 GDPR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547.2.2 Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547.2.3 Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567.2.4 DevOps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577.2.5 Non-Conformance Report . . . . . . . . . . . . . . . . . . . . . . . . . 58
7.3 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.3.1 Product Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.3.2 Product Delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607.3.3 Improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
8 Conclusions and Future Work 638.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
References 67
A Metrics Hive Health 71
B Interview Script 75B.1 Script Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
B.1.1 Introductory Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . 75B.1.2 Process Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75B.1.3 Validation Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
C Fast sheet 77
List of Figures
2.1 Software development processes and activities [37] . . . . . . . . . . . . . . . . 82.2 Definitions for probability and severity [31] . . . . . . . . . . . . . . . . . . . . 10
3.1 Life cycle XP process. [6] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.2 TDD principles [34] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 Configuration Management and Planning [10] . . . . . . . . . . . . . . . . . . . 193.4 Relationships between validation, verification and testing [40] . . . . . . . . . . 203.5 Design controls, verification and validation [40] . . . . . . . . . . . . . . . . . . 213.6 V-model [11] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233.7 Cochlear’s Design Control Process for software [33] . . . . . . . . . . . . . . . 243.8 MDevSPICE R© processes [28] . . . . . . . . . . . . . . . . . . . . . . . . . . . 253.9 Paper screening process [28] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263.10 AV-Model process. [26] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.1 Scrum for medical devices [42] . . . . . . . . . . . . . . . . . . . . . . . . . . . 294.2 Validation process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1 Product Development Plan [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . 375.2 My Health Diary - App and Web Application [18] . . . . . . . . . . . . . . . . . 39
6.1 Process First version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466.2 Process Second version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
7.1 Human factors affect outcomes of using medical devices [27] . . . . . . . . . . . 557.2 Summary of the most observable clauses in the standards with DevOps [24] . . . 587.3 Non-Conformance Rerport . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597.4 Final Process proposed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
A.1 Sprint 1 [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71A.2 Sprint 2[18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72A.3 Sprint 3 [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72A.4 Sprint 4 [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73A.5 Integration Sprint 1 [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73A.6 Integration Sprint 2 [18] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
xi
xii LIST OF FIGURES
List of Tables
2.1 Classification of medical devices [21] . . . . . . . . . . . . . . . . . . . . . . . 6
3.1 Advantages and disadvantages of several SDLC’s . . . . . . . . . . . . . . . . . 22
5.1 Metrics PMIE Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.2 Adaptations to the process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
6.1 Interviewee profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
xiii
xiv LIST OF TABLES
Abbreviations
AIMDD Active Implantable Medical Device DirectiveBDD Behavior Driven DevelopmentDSDM Dynamic systems development methodGDPR General Data Protection RegulationIVD In-vitro DiagnosticMDD Medical Device DirectivePMIE Project Management, Innovation and EntrepreneurshipPO Product OwnerSDLC Software Development Life CycleSM Scrum MasterTDD Test-driven developmentWWW World Wide WebXP Extreme programming
xv
Chapter 1
Introduction
This thesis addresses Software in the Medical Device field. This chapter has the context, motiva-
tion, and document structure.
1.1 Context
There are many professions that can put the life of a person in danger with one mistake. Our
world is now becoming more technological, there are systems and mobile applications for almost
anything. There are no major issues when it is only a game, but when it is about health or assisting
doctors it is a different picture. That is why there are so many regulations required in this new
scenario where software is made for medical applications. This thesis is a study in the application
of agile methods in the development of software for medical devices, to help developers building
products with quality and updates that are compliant with the regulations.
"Promoting good health is an integral part of the smart and inclusive growth objec-tives for Europe 2020. Keeping people healthy and active for longer has a positiveimpact on productivity and competitiveness. Innovation in healthcare helps take upthe challenge of sustainability in the sector in the context of demographic change"
[17]
The software itself is always evolving, and because of that, the legislation had to be updated
in 2017 to keep the software in the medical area safer, secure, and with standard quality. The
change involves putting more accountability in authorities, economy agents, notified bodies, and
health professionals. The importance of having these standards and regulations in medical device
software is to guarantee better processes and better quality medical device software. Depending
on the region there are different standards, regulations, and guidance. For the USA is the Food and
Drug Administration (FDA) and for the EU is the Medical Device Directive (MDD) 93/42/EEC,
the Active Implantable Medical Device Directive (AIMDD) 90/385/EEC, and the In-vitro Diag-
nostic (IVD) Medical Device Directive 98/79/EC —all three of which have been amended by
2007/47/EC. [28]
1
2 Introduction
There is no magic; it is a lot of work to build a software development life cycle (SDLC) that
could help to build the software with all the compliance equality needed. And even then, there are
a lot of risks and failures that could happen in the process. It’s not easy to do software properly, but
having an SDLC is not only required by standards but also greatly facilitates product management
and tracking. Choosing the right one for the software is challenging and will be discussed in this
study.
Agile has been adopted by companies around the world, especially software companies. This
adoption is because they get feedback to improve the software quicker and get better results with
the users when delivering continuously. By doing so, the software is more flexible to changes in
all phases, which is a better way, since the users are humans and change their mind with time
and with knowledge of the software tends to change functionalities that will improve the quality
and the value to the user. Furthermore, agile is only a mindset; there are many frameworks and
techniques to help build the product. This paper will be analyzed Scrum, DSDM, XP, TDD, and
BDD itself and how this can be used in the medical device field. Another scenario that it should
be a concern in software in general, but in a medical device in particular because of the risk, is
usability. Depending on who is going to use must the software, we should think about how the
software will guide the user to make as little mistakes as possible. There are reports of accidents
because of failure in the design of the software, and when the layout can cause wrong results and
even be fatal if the software is misused.
This research’s validation process will be to use the final process created in the previous work
to validate in an academic case. After that, the first updated version of the process will be prepared
and discussed with an one agile expert. After that, following the idea of continuous improvement,
another updated version will be developed and validated with two other agile experts, and finally,
as a result, the final process after these steps validation and improvement steps were taken.
1.2 Motivation
Every project can be developed by using an agile methodology, and this methodology is growing
in popularity. In the 14th annual state of the agile report says that more than 95% of respondents
report their organizations to practice agile development methods. [22] organizations now follow-
ing an agile approach in 2020. Even those which are regulated or especially those. It is a need
in every project to continuous improvement and learns fast so that it can avoid the same risk or
mistake again. When people think about doing software is not just programming at all. There
are a lot of processes to ensure quality that if the project follows, it will generate a large amount
of value for everyone. Especially when it is a software that could cost a life must have more re-
liability. Because of that, the methodology that is more used nowadays is waterfall, which is a
methodology already used for longer and in the comfort zone of companies. But if companies stay
in their comfort zone the improvement in their processes will be difficult.
Due to the challenge to do a process that includes the benefits of agile and the guarantee of
the regulations altogether, this study aims to demonstrate a thorough study between these areas
1.3 Document Structure 3
and proposals already made to solve the challenge by illustrating the process that needs to be done
to be compliant with the regulations and standards without harm the usability, quality, and effort
of the software and developers. Furthermore, this study will propose a process not only with the
medical regulations in mind but also with the compliance with General Data Protection Regulation
(GDPR).
1.3 Document Structure
Below is a brief explanation of each chapter in this document:
The first chapter is the introduction of the thesis, the context, motivation, and the structure of
the document.
In the second and third chapters are state of the art and the research made by this study. Divide
by regulation, process, and methodologies.
In the fourth section the starting point for this study is presented, together with a set of im-
provement opportunities.
The fifth paragraph presents the initial solution where the context of the problem is explained,
along side the contributions and the solution. It also includes the presentation of the validation
plan for the process.
In the sixth paragraph is the validation of the process using an academic case. Which is
detailed the process, how was done, the adaptations made, and the results.
The seventh paragraph is the validation interviews made with agile experts to receive feedback
about the process made. This chapter has the three interviews detailed and the new process created
by the review of the interviewees.
In the eighth paragraph and the last one are presented the conclusions and future work of this
thesis.
4 Introduction
Chapter 2
State of the art - Regulation
There was a radical change in Europe in the subject of medical devices with the publication the
directive 2007/47/EC [38] by the European Council in 2007 after which software was considered
to be a medical device. After this, the more recent directive was the 2017/45/EEC [15] which is
an update consolidating software as medical devices in the industry. [41]
Software is considered an active device by the regulation 2017/45/EEC, so every consideration
about an active device should be respected for software. [15]
From the user or patient perspective, it is clear that it must have standards to assure the safety
of the product. This helps in the confidence that the product works. If we think about the context
of 2020, the development of a vaccine for COVID-19 requires many phases to assure that the
vaccine is safe and can be taken by everyone. In our universe of medical devices, any equipment
or devices used in health care also needs to meet a high level of safety standards. [30]
2.1 Medical Devices Regulation in EU
Regulations are the instruments that governing bodies use to assure the safety and efficacy of
medical devices.[12] But most importantly, as it is mandatory for the company to follow the rules
to enter the market with the certificate in medical device software, it ensures the minimum of
safety for the users and standard between others. In 1993 the European Council released the
Medical Device Directive (MDD) so that it could ensure the safety of medical devices placed on
the European market. [32]
When it is about the regulation, even being MDD or FDA, the challenge to development re-
mains the same. There are some in the list below [28]:
• Adherence to many regulatory requirements specified in various international standards;
• Establishing a full traceability schema from stakeholder requirements to code;
• Performing changes to process artifacts (requirements, code, documents) in a traceable way;
• Being able to embrace change during development;
• Producing development evidence for auditory purposes consistently and continuously and
managing the documentation process in an effective way so that it is not overwhelming;
5
6 State of the art - Regulation
Table 2.1: Classification of medical devices [21]
Risk Classes Risk levelClass I Low riskClass IIa Medium riskClass IIb Medium riskClass III High risk
• Ensuring reliability, safety, and correctness of products;
• Improving the quality of products and productivity of teams;
• The necessity of clinical software validation to be done manually in some cases.
These are all problems that persist, and this paper will try to illustrate the situation and possible
solutions for them.
2.2 Medical Device
First, a medical device is any instrument, apparatus, appliance, software, material, or another ar-
ticle, whether used alone or in combination, including the software intended by its manufacturer
to be used specifically for diagnostic and/or therapeutic purposes and necessary for its proper
application, intended by the manufacturer to be used for human beings for the purpose of diagno-
sis, prevention, monitoring, treatment or alleviation of disease, diagnosis, monitoring, treatment,
alleviation of or compensation for an injury or handicap, Investigation, replacement or modifi-
cation of the anatomy or of a physiological process, Control of conception, and which does not
achieve its principal intended action in or on the human body by pharmacological, immunological
or metabolic means, but which may be assisted in its function by such means. [21].
So, the main concern is the people that are going to use these medical devices. Software for
other areas there isn’t so much risk as to the medicine area because if a bug happens, its normal,
but imagine a fatal bug in a real-life at stake. That’s why they came up with regulations only for
medical devices, and it must always be updated cause technology is always changing. Even with
this purpose in mind, it is natural to try to improve how to build these devices so it will cost less
effort and still respect the regulation and the safety of the users. Because of that, the regulation
has imposed a classification of medical devices.
This classification was calculated by taking into account the factors: [21]
• Length of time in contact with the human body (Transient, short-term, and long-term);
• Degree of invasion of the human body (Invasive, non-invasive);
• Part of anatomy affected by its use (Brain, heart, lower limbs, etc.);
• Potential risks stemming from technical design or manufacture
Depending on which classification the medical device is applied, there are rules and good
practices to follow.
2.3 Software as medical device 7
The International Medical Device Regulators Forum (IMDRF) recommends using this IEC
62304: 2006 standard, as the cause provides a solution for compliance with relevant legislation
requirements with European standards. [20]
This standard IEC 62304: 2006 has the security classification determined by these criteria[41]:
• In the case of a product classified as Class A, there is no injury or damage to the health of
the user in the event of product failure;
• In the event that the safety classification is Class B, some type of injury may occur, but this
is not serious and does not endanger the user’s life;
• If the classification is Class C, the software is considered to be high risk and may cause
injury serious or even death of the actors.
There is some indirect correspondence criteria between these classifications, since the IEC
62304:2006 is only applied to software the comparasion would be that A Class classification would
correspond to a Class III classification. To conclude, that classification of software security can
never be higher than the security rating of the device. [16]
2.3 Software as medical device
For a software to be accepted as a medical device must be specifically intended by the manufacturer
to be used for one or more of the medical purposes set out in the definition of a medical device.
Software is also considered an active device that must have interoperability, which is the ability of
two or more devices from the same manufacturer or from different manufactures to:
a. exchange information and use the information that has been exchanged for the correct
execution of a specified function without changing the content of the data, and/or
b. communicate with each other, and/or
c. work together as intended. [15]
Another concern for the regulation is the risks associates with the possible negative interaction
between the software and the computing environment in which works and interacts.
The validation and verification of the software must have a summary of the results of all both
at the level as in the user’s environment, simulated or actual, before the final release. [15]
As was informed before, there are a lot of points that need to do in the development of this
software to be certified or at least be approved to go to the medical market. The presence of
software in the medical area is undeniable. More than 50% of existing medical devices depend on
the operation of software systems in any way. [41]
With that in mind, it is needed a process to help the life cycle of software development so that
it would be possible to respect the regulation and also do the software in an agile way with lost
of updates and still continuing to be certified to be in the market with the changes. This is the
challenge, have the balance between the regulation and the process to keep the regulation even
with the new releases.
8 State of the art - Regulation
2.4 Medical device standards
According to the International Organization for Standardization(ISO): “Standards are documented
agreements containing technical specifications or other precise criteria to be used consistently as
rules, guidelines or definitions of characteristics, to ensure that materials, products, process, and
services are fit for their purpose.”
When the subject is the medical device, it would be strange to not have a standard or regulation
to take care of, even more, when it is well known that software is not trustworthy in other fields, so
when is about medical area one software failure can have catastrophic consequences. That’s why
it is indispensable to have many standards and regulations about it.
One standard is the ANSI/AAMI/IEC 62304:2006 Standard, also title as “Medical Device
Software-Software Life Cycle Processes,” [37] is an FDA recognized standard. In figure 2.1 it is
showed the activities and processes that the standard recommends, so it will be possible for the
software to be compliant by it.
Figure 2.1: Software development processes and activities [37].
This standard is the only one for the purpose of medical device software and is considered one
critical standard for software developers cause the main processes that development should have:
2.4 Medical device standards 9
[42]
• The Development Process: The standard requires the creation of development planning,
responsible for orchestrating process activities, and appropriate to the size and security rating of
the system. The model used in the cycle should be detailed in the planning, which should also
detail the deliverables of the traceability strategy between system requirements, software, testing,
and risk management measures. Other recommended activities in the document are requirements
analysis, architectural design, detailed design, implementation, integration, system testing, and
delivery.
• The maintenance process: Although sometimes classified as an activity, the standard consid-
ers maintenance as an independent process. The producer should develop maintenance planning
where attention is given to the feedback the product receives after it is delivered. Strategies for
receiving, documenting, evaluating, resolving, and monitoring this feedback.
• The risk management process: The producer should implement a risk management system
and include it in the planning of the development process. Because of that, the ISO standard must
be followed. 14971: 2007 and IEC / TR 80002-1: 2009 (which guides the implementation of the
previous in the context of the software)
• The Configuration Management Process: Must Be Included in Development Planning con-
figuration management information. This includes all tasks and actions in this area, a list of items
to be checked, versioning, and traceability of all changes each item.
• The problem-solving process: The standard forces the producer to create a report for each
problem identified in the product, which must include a type, scope, and criticality. Besides that,
strategies should be prepared to investigate and solve the problem, as well as to maintain their
traceability. Finally, testing activities should be created, and documentation updated when prob-
lems are encountered.
2.4.1 Traceability
Traceability is one requirement that the standard IEC 62304 demand and came with the need to
target the safety in medical device software. It is common to use traceability when it is to link a
requirement with a test case. It is a way to organize even more the project and the life cycle and
follows every phase the artifact that the traceability came from or going to. Now the traceability
has evolved to more than just using in the requirements but also in areas like defect management,
change management, and project management. [32]
There aren’t many study’s about how to do the traceability properly, so was developed the
method Med-Trace which are a lightweight software traceability process assessment and improve-
ment method for the medical device industry. [12]
Even if this or other standards are full of best practices, they also gave liberty to use a different
organizational structure, which allows trying the challenge of doing with agile development.
10 State of the art - Regulation
2.4.2 Risk Management
Every project has a risk. Risk of delay the schedule, risk of overtaking the budget. . . they is so
many that it is harder to list them all at once. But why is needed to identify? If you know that has
a chance for the risk to happen; then it will be good to be prepared to find out a solution or even
a way to avoid the risk from happening. This would apply for simple things in real life; knowing
what could happen, it can be prepared to deal with it. In the scenario of medical devices software
there are elements of risk management:
• Patients;
• Operators;
• Third parties;
• Environment. [31]
On the way to do risk management is also managing the responsibility of the risk and try to
level in the same way as in the figure 2.2.
Figure 2.2: Definitions for probability and severity [31].
2.4 Medical device standards 11
There are some foundation standards that help using risk management in the life cycle of the
software. One of them is ANSI/ISO/AAMI 13485, “Medical Devices—Quality Management Sys-
tems—Requirements for Regulatory Purposes, Int’l Standards Org., 2003” it provides the frame-
work for a quality system for medical device manufacturers and requires establishing a risk man-
agement process based on ISO 14971 and using it throughout the product realization process. And
the ISO 14971, Risk Management—Application of Risk Management to Medical Devices, Int’l
Standards Org., 2000 which defines several risk management terms and provides a framework for
an effective risk management process. [31]
Having standards are important, as said before, to obligate the company to follow the best
practice possible. But still have many problems, the risk management should start with the plan
of the project, not in the end or when some problem happens and must act. It should be followed
throughout the project.
12 State of the art - Regulation
Chapter 3
State of the art - Process andMethodologies
This chapter focuses on the background of the process and methodologies to help in this thesis.
3.1 Agile Methodology
Agile software development has been adopted over the years by many companies all over the
world. Agile is a mindset that involves adaptability and delivering value to the customer. More
than optimizing the process of development is a way to prioritize and improve the software with
time. Agile can be one alternative for the development of medical devices, but the biggest chal-
lenge is to assure that the software maintains the compliance to all required regulations in each
release of this software.
There are not many studies about a process that can guarantee compliance with the regulations
and standards and still improve the deliveries. One aspect is that the main goals of the FDA are
efficacy, safety, and security. Doing that with only pure Scrum, for example, is the challenge.
It is difficult for the company to change from waterfall to agile, especially if the clients are
antiquated and have difficulty changing. Who works with development is used to changes, always
has a new technology or a new approach to adapt to the project. If the developer does not follow
the new tendencies is going to be behind in the market. So it is going to be the projects with
software for medical devices. It will become more challenging to find a developer who wants to
still work with the waterfall model.
Adapt is necessary, but has to be adequate for the regulation and standards as well. It is
necessary to understand more about the agile methodologies that are used today.
The easiest way to understand agile is by analyzing the agile manifesto:
"Individuals and interactions over processes and tools Working software over com-prehensive documentation Customer collaboration over contract negotiation Re-sponding to change over following a plan"[9]
13
14 State of the art - Process and Methodologies
These sentences seem to be the opposite of what the regulations and standards advise to do it.
People tend to think about a lot of documentation and evidence that it was done what they demand.
However, not necessary is this, of course, need to put evidence but still can use with agile. The
next topics will show the possibilities to help with the regulation and standards in medical device
software.
3.2 Scrum
According to the scrum guide[36], Scrum is a framework within which people can address com-
plex adaptive problems while productively and creatively delivering products of the highest possi-
ble value. Scrum is:
• Lightweight
• Simple to understand
• Difficult to master
Scrum can be used to research and identify viable markets, technologies, and product ca-
pabilities to help with software medical devices. Although it has not a structured practice for
implementation, the idea of sprints and continuous delivery is handy for the challenge to put that
in the market of software medical devices.
The Scrum Team consists of a Product Owner, the Development Team, and a Scrum Master.
[36]
The Product Owner is responsible for maximizing the value of the product resulting from the
Development Team’s work. How this is done may vary widely across organizations, Scrum Teams,
and individuals. The Development Team consists of professionals who do the work of delivering a
potentially releasable Increment of "Done" product at the end of each Sprint. A "Done" increment
is required at the Sprint Review. Only members of the Development Team create the Increment.
The Scrum Master is responsible for promoting and supporting Scrum as defined in the Scrum
Guide. Scrum Masters do this by helping everyone understand Scrum theory, practices, rules, and
values. [36]
There are special events like the Sprint, Sprint Planning meeting, Daily Scrum, review meet-
ing, retrospective meeting, and refinement. To optimize the time always has a time box in each
meeting to be more productive in the development.
Scrum’s heart is a Sprint, a time-box of one month or less during which a "Done," useable, and
potentially releasable product Increment is created. Sprints have consistent durations throughout
a development effort. A new Sprint starts immediately after the conclusion of the previous Sprint.
[36]
The work to be performed in the Sprint is planned at the Sprint Planning. This plan is created
by the collaborative work of the entire Scrum Team. [36]
The Daily Scrum is a 15-minute time-boxed event for the Development Team. The Daily
Scrum is held every day of the Sprint. At it, the Development Team plans work for the next 24
hours. [15] A Sprint Review is held at the end of the Sprint to inspect the Increment and adapt the
3.3 Extreme Programming (XP) 15
Product Backlog if needed. During the Sprint Review, the Scrum Team and stakeholders collab-
orate about what was done in the Sprint. Based on that and any changes to the Product Backlog
during the Sprint, attendees collaborate on the next things that could be done to optimize value.
Daily Scrum is an informal meeting, not a status meeting, and the increment presentation is in-
tended to elicit feedback and foster collaboration. [36] The Sprint Retrospective is an opportunity
for the Scrum Team to inspect and create a plan for improvements to be enacted during the next
Sprint. The Sprint Retrospective occurs after the Sprint Review and before the next Sprint Plan-
ning. [36]
They are also the scrum artifacts that follow in the process. Like the product backlog, sprint
backlog, Increment. Many get confused with the difference between the product backlog to the
sprint backlog. The Product Backlog is an ordered list of everything that is known to be needed
in the product. It is the single source of requirements for any changes to be made to the product.
The Product Owner is responsible for the Product Backlog, including its content, availability, and
ordering. [36]
The Sprint Backlog is the set of Product Backlog items selected for the Sprint, plus a plan for
delivering the product Increment and realizing the Sprint Goal. The Sprint Backlog is a forecast
by the Development Team about what functionality will be in the next Increment and the work
needed to deliver that functionality into a "Done" Increment. [36]
So, the product backlog is every scope that the product has, and the sprint backlog is only the
scope that is going to enter the Sprint, so it is part of the product backlog. The Increment is the
sum of all the Product Backlog items completed during a Sprint and the value of the increments
of all previous Sprints. At the end of a Sprint, the new Increment must be "Done," which means
it must be in usable condition and meet the Scrum Team’s definition of "Done." An increment is a
body of inspectable, done work that supports empiricism at the end of the Sprint. The Increment
is a step toward a vision or goal. The Increment must be in usable condition regardless of whether
the Product Owner decides to release it. [36]
3.3 Extreme Programming (XP)
It is a software development methodology that has evolved from problems caused by long devel-
opment cycles of traditional development methods. The term ’extreme’ comes from taking theses
commonsense principles and practices to extreme levels. (Beck 1999) [6]
The extreme level came to facilitate changes and to produce higher-quality software. They can
also improve the time of development because the developers work in pairs, which improves the
quality of the code and the maintenance.
Like other Agile Methods of development, XP aims to provide iterative and frequent small
releases throughout the project. The paradigm of Extreme Programming was built with five fun-
damental values: Simplicity, communication, feedback, respect, and courage. [8] Also focuses on
the phases shown in Figure 6: Exploration, Planning, Iterations to Release, Productions, Mainte-
nance, and Death.
16 State of the art - Process and Methodologies
Figure 3.1: Life cycle XP process. [6]
In the course of planning, a series of requirements-gathering practices are carried out where
the technical team is expected to gain business knowledge of the solution. These requirements
manifest themselves in the form of user stories and map directly to tasks. The client is directly
involved in the construction of the project and the specification of acceptance criteria and tests.
[42]
3.4 Test Driven Development
The first step o implement TDD is to write the tests before the software is developed. Then the tests
are automated, so it will be able to run anytime. For the regulation of medical device software, it is
an excellent choice to use for the verification and validation part since the tests are extensive and
with this method could save a lot of a time of testing. For the context in medical device software,
this can prove to be a significant ally since it is needed to be done many tests to consider the
software done. If the tests are thought before the developer of the code itself, this could also help
estimate the effort more appropriately. Some projects related even that the coding is not the part
that takes more time to be done, the tests are, and it is impossible to think that a medical device of
any kind puts the focus of testing behind in the process. It is fundamental to have all the tests, so
TDD can improve the time spent on the testing and divide some with the coding. Validation is one
requirement that is fundamental for the regulations, and TDD was born validating the code itself
and with that have the consequence of more quality and clean code.
TDD has three phases: [34]
• Red: First, write a unit test in failure. The impossibility of compiling is a failure.
3.4 Test Driven Development 17
• Green: Write as soon as possible the production code enough to pass this unit test even if it
allows the "worst" solutions. Of course, if a clean and straightforward solution appears immedi-
ately, it must be realized, but otherwise, it is not severe, the code will be improved incrementally
during the refactoring phases. The aim here is to obtain the green bar of success of the unit tests
as soon as possible.
• Refactor: This phase is often neglected but is essential because it eliminates possible code
duplication but also makes it possible to make changes in architecture, factorization, presentation.
This refactoring concerns both the production code and the test code and must not modify the
program’s external behavior, which is materialized by a test execution bar that remains green.
The Figure 3.2 below exemplifies the phases above descriptive.
Figure 3.2: TDD principles [34].
With this process, the productivity of the developers also increased. TDD helped in the case
this paper mention in section 8 that they concluded that it would not be possible for them to apply
agile without TDD because of the volume of testing and the Sprint time. TDD could be combined
with other agile methods to integrate the compliance needed for the medical device software.
18 State of the art - Process and Methodologies
3.5 Behavior Driven Development (BDD)
BDD is a software development practice to write scenarios from the user perspective that evolved
from TDD. It uses the Gherkin language to write the cases, and with that, it is possible to au-
tomatize the latter into tests. One of the many advantages of using BDD in the project is that the
scenarios should be written together with the stakeholders and the developer team, so the entire
team knows what the project is about.
For the case study in Healthtech [11] that changes from waterfall to agile using BDD, they
experience an increase in communication, which results in better collaboration. It was easier to
formalize a standard in the test requirements that help with the system validation. Traceability
was a goal, too, because of the transparency to all stakeholders. Another benefit was that the
time-consuming tasks were better harnessed for other tasks since BDD is mostly automated. To
ensure continuous compliance before every release, they generated comprehensive testing reports
that allowed the stakeholders to review and confirmed that defined test scenarios were meaningful
and covered the product requirements.
The automation available using BDD saved time and cost in that project, critical to small
organizations such as ours. Finally, significant changes can be deployed with thorough validation
and test requirement coverage to accelerate the development process. [11]
Using BDD could be a way to maintain the releases with the proper standard and regulation
that MDD requires. It has automation as an ally to keep managing the changes and keep the
traceability alive even with the changes.
3.6 Compliance with IEC 62304:2006
To develop quality software is needed so much more than just coding. There are areas of the
software that need to be included in the process to comply with the standard. In this chapter, the
goal is to explore some of these crucial areas and complement developer medical devices.
3.6.1 Configuration Management
Software configuration management is the control of the evolution of complex systems [14], which
means that every software should be known, tracked, and managed.
This area was adapted from other institutions and was thought in the context of software. There
are several benefits when the project uses well the CM:
1. Easily recovery with rollbacks;
2. Reliability with less chance of downtime;
3. Easily scaling;
4. Knowledge of services. [10]
3.6 Compliance with IEC 62304:2006 19
There is a process to follow to develop a good CM. First, the phase of configuration iden-
tification needs to identify which needs to be maintained. Second, the configuration control is
responsible for control the changes that will happen. Third, there is the configuration status ac-
counting, which provides the current state of the development, and the history can be traced.
Fourth, the configuration audit is to keep register the changes that can be made. More information
in the image 3.3.
Figure 3.3: Configuration Management and Planning [10]
Configuration Management is a perfect fit for agile methodologies since it helps to keep the
velocity high and is also inside the process of DevOps.
3.6.2 Validation
Validation is not only testing the software. If only has validation in this phase, then it is counting
that the test will be so perfect that it would validate the entire project. Verification also is not only
testing, included but also has other activities. It is clearer this representation in the Figure 3.4.
In agile, two questions teach what validation is:
"Are we building the right product?" and the question for verification: "Are we build-
ing the product right?".
There is regulatory guidance that defines validation as "Confirmation by examination and pro-
vision of objective evidence that software specifications conform to user needs and intended uses,
and that the particular requirements implemented through software can be consistently fulfilled"
and verification as "Software verification provides objective evidence that the design outputs of a
particular phase of the software development life cycle meet all of the specified requirements for
that phase" [39]
20 State of the art - Process and Methodologies
Figure 3.4: Relationships between validation, verification and testing [40].
It is secure to affirm that needs evidence to prove that the validation and verification were done
correctly. But must be objective evidence is something to be based on a fact.[40]
3.6.3 Verification
Verification and validation go side by side, they are related, but both are fundamental to deliver-
ing the software with the required requirements and quality. Verification activities include review,
evaluations, or actual tests that are graphically shown to verify that the design outputs are consis-
tent with the requirements. [40]
In figure 3.5 can seem that verification and validation are redundant, but the same way that is a
good practice to have the front and back end prepare for some exception in the process works too.
So the validation will confirm that what was proposed in the first place at the end was the same,
and the verification works in the middle to guarantee that everything will work correctly.
3.6.4 Software Life Cycle
A software life cycle is a way to organize building with some milestones, metrics, or control points
in a more organized way. There are some common characteristics:[40]
• Identifiable phases;
• Input to each phase;
• Activities to be performed in each phase;
• Outputs of each phase.
When the subject is the medical device, the life cycle must be adapted because of the com-
plexity and must emphasize the development and maintenance of the software. [40]
3.7 V-model 21
Figure 3.5: Design controls, verification and validation [40].
A single life cycle for software in a company is not ideal because it should change with the
software type that it is developing. Also, it is essential to have one because it is a way for outsiders
to know that it has a process to guide and control the process. If it has control, it will be easier to
identify where the problems are and how to improve even the process itself to help build quality
software. It can be economical to know the schedule and the budget in a more precise way.
One way to find out the problems in the life cycle is with metrics. How many bugs were found
in a specific phase; for example, it could help study more and understand what happened to cause
bugs. If it has a few bugs, it would be an excellent example to put in more projects.
How to choose the best life cycle is a question that many companies are struggling with. The
problems are always the same, requirements changing, and mistakes like lack of communication
between the team and many more. There is not yet one solution for all of these, but the life cycle
can help solve some of them. Nevertheless, it still must be careful about it because sometimes
the process itself can be the problem if it is not adequate for the project and who was supposed to
follow the activities start not to be doing it as the process demand it. So, there are some options to
choose the more adequate in each case.
In this table 3.1, there are four options of the life cycle and has a summary of the pros and cons
of each one.
As was illustrated, there is no perfect approach, and it will depend on the software and what
would work better in that case. After choosing the best-fit life cycle, the plan must create a quality
system; with the review, coding software, verification, and validation testing, and more are all part
of the software life cycle plan.
3.7 V-model
Even if the regulation does not determine, which SDLC to follow, medical device software devel-
opment organizations typically follow the V-model. The V-Model shows the relationship between
22 State of the art - Process and Methodologies
Systemsdevelopmentlife cycle
Pros Cons
TraditionalWaterfall
Intuitive, easily understood.Established and well known.Enforces design before implementation.Control points clearly demarcated.Leaves “clean” design history trail ofdocumentation.
Not realistic for more than simple software.First software available late in life cycle.Testing starts late in life cycle.Requires interpretation and modification to make itwork for real projects.
ModifiedWaterfall
Intuitive, easily understood.Established and well known.Enforces design before implementationin a slightly more realistic way.Control points clearly demarcated.Leaves “clean” design history trail ofdocumentation and updates.
Still slightly unrealistic . . . but better.First software available late in life cycle.Testing starts late in life cycle.Still requires interpretation for changesrequiring more than single-phase updates.
SashimiWaterfall
More indicative of how some projectsare done (vs. should be done?)Recognizes the need to “peek ahead”in life cycle.Design-before-implement still presentbut to lesser extent.First software pieces available earlierin life cycle.
Control points less clear. Documentation ofdesign history challenging.Configuration management (synchronization of designdocuments, code, and test documents) more difficult.Progress metrics and forecasting difficult
Spiral
May be best model for softwarewhose requirements need tobe discovered iteratively.Gets prototype software in users’hands most quickly.Opportunities for more formaldevelopment in later stages of the spiral.Good for user interface intensive softwarein which user preferenceis a key driver of requirements.Integration of risk management is explicitand in-line, not supplementary.
Not appropriate for software whose early prototypescould harm users.Tracing evolution of requirements back to user needsmay be challenging.Temptation to end the life cycle at the last “good enough”prototype withoutadequately reviewed design or adequate testing.Metrics for project management, forecasting, and processimprovement challenging
Table 3.1: Advantages and disadvantages of several SDLC’s
3.8 Related work 23
the two sides of the development process. This relationship is used to determine whether the stage
has been completed successfully. [26]
The V-model has a series of linear stages represented in Figure 3.6.
Figure 3.6: V-model [11].
3.8 Related work
3.8.1 CochlearTM
There was one case in 2008 at the company CochlearTM [33] that build solutions for severely to
profoundly deaf people. They needed to adhere to the highest level of the medical device standards
and comply with Traditional Waterfall.
The Cochlear than tries to build with agile and developed his product development process
based on the V-model, shown in Figure 3.7.
Even with this process, they had issues on introducing agile in the company like:
• the disparity between using an Agile methodology in the software department versus using
a waterfall/iterative process throughout the rest of the organization
• the identification of the recipient safety aspects of a product and the mitigation of such risks
in a structured and traceable way
• managing the evolution of requirements in a traceable way
24 State of the art - Process and Methodologies
Figure 3.7: Cochlear’s Design Control Process for software [33]
• the long validation cycles required to validate the marketing claims and ultimately the user
requirements
• the extensive manual verification testing required to verify that the electrical stimulation of
the implant under software application control is within safe limits
• device drivers with a large amount of legacy code that was neither unit tested nor conducive
to any other automated testing type. [33]
The project definition had complicated setting the timeline with all the estimations and expe-
rience based on a different development process.
In the product definition, they change the use of the Software Requirement Specification (SRS)
for user stories based on XP.
In the case, they use sprints of 4 weeks, and for the user, story be considered done, it has to
be implemented and tested. The definition that it showed impossible to accomplish in the time of
the current Sprint since they had to do safety testing as the regulation demands. Because of that,
they needed to decide if they were going to spend time automating testing or keep going without
delivering and degrading the velocity. That was the point that they implement TDD using the
framework FIT. When they made this decision, they got it right the timing and started to see the
sprints happening.
Although they could implement TDD, they still did the clinical validation manually to ensure
that they match all the requirements. [33]
This case was fascinating because it could be analyzed to create a process complementing
their issues and improve their difficulties. However, it has proved that it is possible to work with
agile and regulations for medical device software, and with agile, it will improve the efficacy of
the developer team and the quality of the code.
3.8 Related work 25
3.8.2 MDevSPICE R©
This article is the most recent case published. The work was done in 2019 with a framework
MDevSPICE R©, XP, Scrum, and DSDM.
They first analyzed the XP with Scum for the management and technical processes perspective.
However, the mapping of these two methods was not enough to cover all medical requirements’
needs. Then they performed a new mapping with the Dynamic System Development Method
(DSDM), which provides practices and the work products to cover the full software development
life cycle. Its emphasis on governance, adequate documentation, and specific roles. [28]
This study was an extension of the previous mapping studies done with XP and Scrum to in-
clude DSDM analysis. It was revealed that have ways for the methods XP, Scrum, and DSDM be
combined, how these agile models can support each other, and which additional system and soft-
ware processes and practices should be considered to ensure medical regulatory compliance.[28]
MDevSPICE R© is a framework developed to help with all the regulatory requirements speci-
fied in the standards. It is an integrated framework of medical device software development best
practices.[28]
MDevSPICE R© was based upon the old ISO/IEC 15504/SPICE, the ISO/IEC 33000 series.
To build a process assessment standard that includes requirements from a complete number of
medical software development and software engineering standards. [28]
The process created can be illustrated in Figure 3.8.
Figure 3.8: MDevSPICE R© processes [28]
26 State of the art - Process and Methodologies
Another process that was used in this case was the Dynamic Systems Development Method
(DSDM). It is an agile project framework that provides practices and products to cover the full
software development life cycle. It was initially developed in 1994 by a large group of practitioners
for rapid application development. DSDM advocates fundamental agile principles such as Enough
Design Upfront, Never Compromise Quality, and Collaborate. [28]
It was performed in the case of a systematic literature review (SLR) to specify the agile meth-
ods used in the medical device software development domain. The study that was done was in 126
papers and reduced to 33, as illustrated in Figure 3.9.
Figure 3.9: Paper screening process [28]
Their analysis has revealed that Scrum, Extreme Programming, and Test-Driven Development
are the most used agile methods in the medical device software development (MDSD) domain.
Other research was that Lean Development, Feature Driven Development, and DevOps are the
other agile methods referred to in the papers. However, there was no evidence on the utilization of
DSDM in the MDSD domain.
They also discovered that the scaled agile models: Disciplined Agile Delivery and Scaled
Agile Framework was not provided in the specific solutions for safety-critical projects. It was an-
alyzed by them that performing daily stand-up meetings and using burn-down charts enable teams
to uncover issues earlier, monitor the effect of changes, and estimate better. Iterative development
enables accommodating changes more comfortable and getting early feedback from the customer,
and managing scope creeps. [28]
On the other hand, according to the papers, it was also analyzed which agile method was
not suitable to implement in the medical device software development. R-Scrum augments the
Scrum method with specific roles and artifacts. In addition to the Scrum roles, R-Scrum defines
a "Vice President (VP) of Quality" role who verifies that the output(s) from each Sprint adhere
to the required procedures and a "Quality Control" role who produces system test documentation
and executes system test scripts in line with required standards and product specification and
documents all test results for release review. [28]
One of their goals was to find what could extend their framework MDevSPICE R©. It was
analyzed that Scrum covers project management, requirements specification, integration test, and
system test stages/activities. XP presents solutions for requirements specification, design, code,
unit test, integration test, and system test stages/activities. DSDM, however, covers the entire life
cycle. While Scrum and DSDM provide practices for project management, XP does not. [28]
3.8 Related work 27
This paper was fascinating because it could join other papers in one. It is theoretical to prove
that combining more agile methods will help cover all the standards and regulations’ requirements.
Still have more agile methods to explore, and the job is still not over, but at least with this work
is not from zero the study of new processes to full fill everything to give a better medical device
software with safety.
3.8.3 AV-Model
It is an agile adaptation for the V-model. It could be more comfortable for the companies to adopt
because of the process’s similarity since any change must be the phase of adaptation and learning.
It was a hybrid model that, in the mapping study was identified 13 practices such as "Iterative
Development," "Use case/User Stories." [26]
Figure 3.10: AV-Model process. [26]
The objective of the development of this model is to resolve problems associated with fol-
lowing a plan-driven software development life cycle while reaping the benefits of utilizing agile
practices.[26]
This AV-Model also has two stages of validation:
1) Expert Opinion: Once the model has received validation from the industry, it will be dis-
tributed to experts in medical device software development. An agreement has already been made
with members of the IEC 62304 committee and members of the TIR 45:2012 committee to provide
validation of the model. By eliciting this form of feedback, the model aims to gain acceptance in
the standards community.
2) Implementation: Once the model has undergone the Expert Opinion validation, it will be
adopted by a medical device software development organization. A medical device software orga-
nization has agreed to implement the model once it is completed and passed through each of the
steps of validation .[26]
28 State of the art - Process and Methodologies
Hybrid agile methods have been shown as a viable option to secure standards and regulations.
This case was interesting because it adapts one model that is one of the most used for medical
devices software.
Chapter 4
Previous Work
This study is a sequel to the thesis made by Zamith [41] named "Desenvolvimento de Software
para Dispositivos Médicos: Uma Abordagem Ágil". His study observed many improvements and
had a solution to the process based on this study. It was based on the idea of the methodology
Scrum to improve the process in the medical devices field. The proposed solution is summarized
in figure 4.1.
Figure 4.1: Scrum for medical devices [42].
29
30 Previous Work
4.1 Initial Process
This process was made to comply with the standard IEC 62304:2006 and still using agile to im-
prove the quality in every release. The process was started by doing the product vision, where it is
about to work in the product’s global idea. This part should have identification of which classifica-
tion of security the medical device has. The identification of the tools, standards, and deliverable
that the project will have. Also, the backlog should be ready and an architectural document.
After this, it will be the Sprint 0, a revision of the product backlog, update the documentation,
and have the definition of done to all teams. [42]
Every sprint should have a time box between two and four weeks. As a result of every sprint
should have an increment of the product. The user stories that will be developed in the sprint
should not have changed since the sprint started. Here the process is equal to SCRUM. QA check-
point was one phase added to audit the deliverable and evidence compliant with the standard.
After this, the checkpoint should have a report of non-compliance, and the measures should be-
come tasks. The QA checkpoint was based on the R-SCRUM.
The process created by Zamith still involved all ceremonies of SCRUM in making the team
more capable of following the life cycle of SCRUM. [41]
Another change was a QA Team that is not part of the development team, which is accountable
to do the QA checkpoints.
Was proposed several practices that should be applied in the process like TDD, Refactoring,
Build Process Automation, Automatic Deployment process, Automatic tests, and Continuous In-
tegration. [42] These were actions that could improve the quality and assist the agile process with
standard compliance.
4.2 Compliance with standard IEC 62304:2006
It was observed in the study that are some tools that allowed evidence that can be useful for the
standard. Like Version Controls, Issue Tracking Platforms, and Integrate Environment Develop-
ment. [41]
In the field of requirements, user stories are proposed, which is a common practice when
using an agile methodology. One of the many advantages of using this is that the stakeholders are
involved.
The definition of done is essential to be aligned with the team once it provides the developed
work quality. Some practice is advised to be done at this point:
a) Code produced;
b) Success compiled project;
c) Unit tests, integration, and system;
d) Deployed in a test environment without errors;
e) Refactoring activities;
f) Update the documentation;
4.2 Compliance with standard IEC 62304:2006 31
g) Code Review;
h) Fix all anomalies in the process of verification. [41]
4.2.1 Verification
Verification is fundamental for the standard IEC 62304:2006 one way to reach the requirements
demanded. One practice that should be with Pull Requests which have evidence to prove to the
standards.
Must have a rigid verification in Requirements, Architectural, and Planning documents, espe-
cially if it is the sprint zero. [41]
4.2.2 Configuration Management
The propose was to have an automation process to do it and document. Automation is fundamental
to the changes that the project can have. So it is necessary to have version control.
4.2.3 Traceability
It is fundamental to have links between the artifacts and data of the project. If the software is crit-
ical, safety is a priority and must have traceability between the artifacts. Traceability contributes
to more understanding of the product itself. [42]
It is essential to connect every area with requirements, development, tests. It was advised to
use the platform issue tracking. [41]
4.2.4 Test
The standard IEC 62304:2006 demands three levels of testing:
a) Integration
b) System
c) Regression
These items should have evidence and better if it is automatic.
4.2.5 Audit Checklist
Each section of the checklist corresponds to a chapter IEC 62304: 2006 standard and were use to
validate in the academic case of this study. [41] The checklist have the audit information, project
details and types that will evaluate each item to compliant until observation.
The audit scope were Project Managment, Configration Managment, Software Planning, Risk
Managment, Verification, Delivery, Requirements analysis, Safety Classification, Software De-
sign, Software Testing and Software Construction.
32 Previous Work
4.3 Limitations to validate the process
In the thesis of Zamith [41] were observed some limitations when he validated the solution in an
academic case and the interviews. Like every project with a previous similar one, it is an excellent
practice to learn the previous lessons learned to avoid them from happening again. So it was
punctuated by him some limitations that he had to deal with due to the academic case environment
that the author could learn before validated mine.
• The study’s validation was made with an academic case, which was only in the project’s
final moments. Because of that, it was difficult to analyze every step and observe;
• It was not possible to find elements to perform all the necessary roles and the absence of a
Product Owner made some important actions impossible during the process;
• It is difficult for companies to put a separate team to do the intern auditing;
• The process is tied with the TDD, and not many developers have experience on this;
• The validate case does not have many of the scenarios of tests made.
Even with the solution proposed, the process cannot be fully validated due to the limitations
above.
4.4 Proposals for the limitations
Based on the experience of the previous work with the objective to improve the process already
created, and more study in the agile area, this thesis proposes improvements to get a more adapted
process that could work at any company and not only focus in the regulation but also improve the
process by an in-depth study in the agile methodology so that the developers and the work team
can also have benefited by using the process.
It is essential to follow the process at the conception part to understand more of the product and
what process will work better. The previous work was not possible to follow from the beginning
of the case because the process was still being developed. Otherwise, this thesis could follow the
academic team in the conception part of the process already made.
Another issue was about quality assurance in the process, which is a primordial area to guar-
antee compliance with the medical area’s regulations, security, and safety. Another proposition
beyond had the quality test outside to audit also includes a QA Team inside the developer team
to follow and respect the regulation. Having a QA member inside the team will help limit getting
more visibility of test scenarios if the tester is in the developer team.
Today it is possible to find several technologies and processes that will help in the test automa-
tion. Know developer TDD, unfortunately, could not be the knowledge in the developer team, so
it is useful to have other techniques with agile that are more common for developers to use than
TDD. So that will solve the technical debt that the team could have, and the team can be trained to
other projects.
4.5 Validation Process 33
4.5 Validation Process
For a project, be agile is not necessarily one methodology but the change of the mindset. Because
of that, when this study was started, the plan was to use the same project and compare the results.
However, it is natural that overtime on a project, people will gain more experience, and this is what
happened in this thesis; why compare the same thing if the author already learned some necessary
improvements to work in the process?
Because of that question in this study, the validation was done by the author with continuous
improvement.
The first step is to tested in the academic case using the process created by the previous work
presented in this chapter. Next were analyzed what could have been better and created a first
version of the process by this study. Later on, the first version was validated by one agile expert,
and with the feedback of one interview were made a second version of the process and validate by
the last two interviewees and then with the feedback of the interviews was generated the third and
last version of this study process presented in chapter 7.
This thesis will use this approach to validate the study, and it is exemplified in the diagram
below 4.2.
Figure 4.2: Validation process
34 Previous Work
chapter 5
Chapter 5
Academic Case
This chapter presents the academic case experience validating the proposed process and recogniz-
ing improvements for this study.
5.1 The project - Hive Health
Because of the pandemic conditions in 2020, this study could not be validated in a real com-
pany, but it was possible to observe and act in an academic project in the course PMIE (Project
Management, Innovation, and Entrepreneurship) from the Master in Software Engineering at the
University of Porto.
The client of this group of students was the Hospital Pedro Hispano, which had the objective
and motivation to create an information circuit from the patient’s headache diary to the doctors to
reduce the number of appointments.
The main functionalities would be developed in two systems, one mobile app for the patient
integrate with a web application in which the doctor would have access to the data made by the
patient in the app.
5.1.1 Academic Validation
It was possible to use the process done by the previous work done. That provided that the study
could have started to validate the process at the beginning of the project. Not giving the students
more tasks but still following the process so that the validation will be an inside job of quality. To
ensure the process’s precision, the author was part of the team and helped with the quality.
First, the team evaluated which methods they already have done and which were necessary to
add to the full process. The author aligned with the Leader of the project that they were doing
already the following practices:
• Product Vision
• User Stories
• Definition of Done (Verification)
35
36 Academic Case
• Code Review
• Versioning
• Traceability (Issues from Gitlab)
Second, it was possible to identify the need of one member to do the integration testing and
regression testing during each Sprint, and at the end of the project was to verify the checklist of
non-compliance, which had, in result, one template of a Non-Conformance Report. The work
done here simulates both quality teams inside the developer team and outside with the idea of
audit the project.
Third, it was identified by the author that because it was an academic project, still were some
practices that weren’t able to be observed in this case:
• Unit test
• Automation with CI/CD
• TDD
The product was made with this mindset, to empower the connection between doctors and
patients by developing a digital headache diary, aiming to provide health professionals constant
analytics and insights regarding their patients’ health.
During the sprints, it was possible to do all the Scrum ceremony’s except the daily scrum
because it was an academic project. However, in the Retrospective, improvements of the team and
compliments about the quality work done during the sprints were observed. [18]
The calendar of the project was developed in 5.1:
Sprint 1: March 20 - Apr 3, 2020
Sprint 2: Apr 3 - 17 Apr, 2020
Sprint 3: Apr 17 - 1 May, 2020
Sprint 4: May 2 - 16 May, 2020
Integration Sprint 1: May 15 - 22 May, 2020
Integration Sprint 2: May 22 - May 28, 2020
5.2 Process Validation 37
Figure 5.1: Product Development Plan [18]
5.2 Process Validation
The author was inside the team as a QA member and a QA Audit. As it was an academic study,
the author’s fairest proposal was to help make the process happen. For that it was presented,
the process proposes for the Leader and the CTO responsible for the Hive Company. After that
meeting, the students compromised to make the items in the list presented before.
Nevertheless, they would not do a unit test because the primary purpose was to deliver a
functional MVP fast. Another difficulty for them would be integrating testing and regression
testing because they did not have the time and resources for it; that is what the author could help.
The author’s role would be doing to assure the integration and regression tests for each Sprint.
Besides assuring to do a Checklist as a QA Audit as well to see what points the product would not
be compliant and generate the Non-conformance Report. After their approval, the author started
to keep up with the team since the Sprint Review of the first Sprint.
Due to the COVID-19, the class was all made online, so the author was always called in the
meetings in google [2], and the communication was via Slack [3].
In every ceremony was analyzed if the process has been followed by talking to the students.
The author was accountable to determined which user stories were truly delivered by testing them
and open issues in the Git Lab when needed. The author also proposes improvements for the
product, but they were more difficult due to its schedule.
There was not a Scrum Master to follow the metrics, but the author also did that during the
Sprints, and the Leader always helped the team evaluate the stories by days.
38 Academic Case
5.2.1 Results
5.2.1.1 Conception
The conception phase was presented for the team, the goal of the product. With this mindset, the
team produced the following documents:
• Project Charter
• Product Vision
• Architecture Diagram
• Use Case Diagram
• GDPR Research
At the beginning of the project were concern about the product being GDPR compliance, and
with that, the team was able to identify functionalities to compose the product backlog that was
fundamental for the first version in production.
5.2.1.2 Development
During the development were done user stories inside the issues in the Git lab [1], which were
traceable with tags. The team also estimated the issues and defined what would make the issue
considered done.
In each Sprint was made tests in the delivery and regression tests. The team was very grateful
for having a member who had focused on the quality area. According to members, if there were
not someone focused on quality, many defects would have gone unnoticed, causing significant
problems during the delivery phase. These reassure that it is fundamental to have a team focus on
quality inside.
The developer team developed two systems. The systems are illustrated in the image 5.2.
5.2 Process Validation 39
Figure 5.2: My Health Diary - App and Web Application [18]
5.2.1.3 Closure
The project’s closure was made more integration testing with both systems to validate if they were
working together. In this phase were found some divergences because different developers made
the systems but always had an agreement which place would change for both systems be in the
same information for the user.
Was delivered the following documents:
• Business Model
• Metrics and Findings
• Handover Report
• Requirements Updated
Also was made the release of the MVP for the hospital.
The author observed the Sprint metrics, presented at the table 5.1.
40 Academic Case
MetricsPMIE Case Weight Issues
Closed AVG MergeRequests
FixedBugs Observations
Sprint 1 75 9 49.8 32 0Sprint 2 133 13 92.4 42 0Sprint 3 41 30 12.1 32 7
Sprint 4 7 38 3.5 47 8Not all issues were estimatedcausing discrepancy in the average.
IntegrationSprint 1 14 30 7 14 9
Last sprint to add functionalitiesand integrate both systems.
IntegrationSprint 2 28 15 11 17 9 Code freeze for deploy
Table 5.1: Metrics PMIE Case
5.2.1.4 Lessons Learned
• It was identified improvements in the process had to change in some aspects. At the end of
the project was observed that it would be better if they focus on usability was made at the
beginning of the project and not at the end.
• It was very little about the phase product discovery, about deciding what to build. In the
end, were some adjusts in the MVP context? What was in the backlog of the project? The
Product Owner’s role was played by the team leader of the team and the client to build the
final backlog.
• Another problematic experience for the team was that they didn’t preview the risk of the
pipeline and the effort to do it. If the developer team had CI/CD implemented, this risk
could have been mitigated.
5.2.2 Adaptions
At the end of the MVP were analyzed based on the checklist [41] the non-conformity’s and what
adaption or solution made to be compliance at table 5.2
Non-conformity’s AdaptationDaily meetings /Sprint Burn-down Board of Git lab and slack communicationNo records of Retrospective Write on the wiki of Git labNo CI/CD Review of Merge RequestsNo unit tests More exploratory testsNon non-functional requirement Written in the user storiesNo Architectural report Written an architectural documentNo Configuration management report Review of change requestNo Backlog Refinement The backlog refinement was done inside the Planning.
Table 5.2: Adaptations to the process
5.3 Conclusion 41
5.3 Conclusion
Following the team’s routine with the ceremonies, the difficulties during the project, and is in-
cluded as a quality member and a consulting member that they could turn to understand more
other areas were exciting, at least.
Because despite the author not have the same responsibility for the grade they had to deliver,
the author had the responsibility to deliver a project that would follow the process proposed by
the previous work and do the tests that would guarantee the software’s quality. The author’s role
was internal and external at the same time. After all, internally, it was performed regressive,
integration, and exploratory tests that made a difference in their work and external because the
author could still assess the difficulties of the process and improvements that could change to
facilitate the development work.
The improvements for the process that was observed in the case was:
• GDPR;
• Non-conformance Report;
• Usability.
42 Academic Case
chapter 6
Chapter 6
Interviews
This chapter focuses on the interviews made with agile experts about the process created in this
study.
6.1 Validation Interviews
Three agile experts were selected for validation interviews and for professional feedback about
this thesis. The main purpose was to:
• Collect information about their experience in agile methodologies;
• Validate the ideas of the contribution in the process without showing the process;
• Collect experiences in processes which are regulated;
• Validate the tools or techniques used in the process;
• Validate the new process and collect feedback about improvements.
The interviews were via Teams because of the scenario of the pandemic COVID-19. The au-
thor has chosen the professionals by the experience in agile methodology and regulated projects
besides the experience of change mindset in companies. This thesis aims to change the mindset
that medical devices must only be developed with waterfall processes. This study had a focus
on improving the process [42] based on the standard IEC 62304:2006 [19] by keeping the pro-
cess compliance and evolve in the agile process. That was presented by each interviewee at the
beginning of the interview to be acquainted with the thesis.
Below is the table 6.1 follows the profile interviewees mentioned:
43
44 Interviews
Interviewee Experiences Interview Date
Analia Irigoyen
Role: CEO at PromoveCertified Scrum Product Owner (CSPO) by Scrum AllianceCertified Scrum Master (CSM) by Scrum AllianceCertified Project Management Institute (PMP)Certified PSMI by Scrum.orgCertified Lean Kanban KMP I (LKU) by Lean Kanban Inc.Certified Lean Kanban KMP II (LKU) by Lean Kanban Inc.Certified Facilitator DevOps by Adapt NowCertified Exin DevOps Professional by EXINCertified ICP-AHR-ICAgile Certified Professional by ICAgileCertified LeSS Practitioner by The LeSS CompanyCertified MGT 3.0 by Management 3.0
July 28, 2020
Carlos Rodriguez
Role: Scrum Master and Agile Consultant at RadixCertified as Scrum Master Foundation by Scrum.asCertified as Scrum Master (CSM) at Scrum AllianceCertified Product Owner Foundation by Scrum.as
August 10, 020
Samira Tavares
Role: Agile Expert at Knowlodge 21Certified Scrum Product Owner (CSPO) by Scrum AllianceCertified Scrum Master (CSM) by Scrum AllianceCertified Kanban KMP Foundation I by Kanban UniversityCertified Kanban KMP Foundation II by Kanban UniversityCertified Kanban KCP by Kanban UniversityFlight Levels Systems Architecture(FLSA)by Flight Levels AcademyCertified MGT 3.0 by Management 3.0
July 26, 2020
Table 6.1: Interviewee profiles
6.1.1 Samira’s Interview
The first that was interviewee had a duration of forty minutes and was with Samira Tavares, who
is an agile expert from the company Knowledge21, which is licensed to apply certifications from
Scrum Alliance and to change the culture inside the company to become agile, for this study was
essential to have her participation because of the experience inside projects and now as a consultant
that transform the companies.
Samira had in 2001 experience with development in 2005 was QA in an airline and was Re-
quirements for the company Petrobras. After that, she started with agile in 2008. She was pro-
moted and gained experience as a coordinator and project analyst. In the future, it was PO and now
is an agile expert working as a consultant and advising companies on how to improve processes.
At the beginning of the interview was asked preliminary questions. Samira truly believes that
every project can be done by using agile. She said "Being agile means being focused on what you
are going to deliver, delivering value to the customer, quickly, continuous improvement.". About
the benefits, she mentions value delivery, learn and miss fast, culture change, and work with a
process establish and clear with beginning and ending.
6.1 Validation Interviews 45
She had experience with working with a certifiable project that had compliance with bank and
money. In this experience, she thought that it was necessary to adapt the agile methodology. She
used Scrum and Kanban together to focus on the flow. Because of that, they had to focus on the
client, short cycle, and continuous improvement.
For the process questions, Samira liked the first question because everyone thinks that agile
does not have risk management. Nevertheless, the truth for her is that actually, the risk appears
sooner in the project, and cause of that, it is possible to verify sooner and manager. In agile
understanding, what is the smaller share of the value that can mitigate this risk—agile help by
being transparent and clear with the risks.
Samira advised that in the regulation project’s context would be good to have during the entire
development cycle, and one specific ceremony has to be carried out to deal with the risks. It should
be a ceremony to look at the risks, and the entire scrum team should be involved. There is also in
some companies the manager to help mitigate the risks during the project. She suggests a tool the
technique flyte levels for risk management.
For Samira, usability is how to design the product, as the minority looks, usability has a bias
of including people in the product. It must be essential if the client wants to have a differentiated
product, not a copied product.
When the matter is a QA team inside or outside, Samira affirmed that she has to be inside.
Even if there is only one person accountable for it makes the entire team works for quality as well,
and she had some experiences with that and worked. However, an audit is different from assuring
quality; the audit should look for the regulations.
At the end of the interview, the second version of the process was presented, which was devel-
oped after the study in the previous work done and observing the academic case detailed above.
To explore all the experiences that the interviewed had to improve even more the process. How
this part is the most important of the interview, it will be presented with more details and separated
by the interviewee. The first interview was with Samira Tavares, and explained the first version of
the process that could see more details in the image 6.1.
After explained the process, was given some feedback and suggestions:
• The product owner must not be isolated, put him also represented in other ceremonies;
• Te usability should be in the product vision;
• DevOps is not well located, should be near the team, who will put in practice;
• One improvement would automate the checklist;
• Create a new ceremony for risk management;
• Analyse which metric to use to evaluate the risk;
• Go more deep in the product discovery where the ideas emerge;
• Put automated testing in the process;
• Lookup for fit for purpose, distinct metrics (efficiency, atmosphere, ecosystem, and techni-
cal);
46 Interviews
Figure 6.1: Process First version
• Change the colors of the team to be more representative.
After this interview, the author made a third version of the process, and it is illustrated in image
6.2.
The author presented version three of the process in the last two interviews reported below.
6.1.2 Analia’s Interview
The second interviewee had a duration of one hour and was with Analia Irigoyen, a CEO from the
company Promove, which is a company with more than ten years of operation, and 200 customers
served in Brazil and abroad. The main goals are to help companies be certifiable like CMMI 5 and
improve the general process, for this study was essential to have her feedback about creating new
processes.
Analia’s experience with agile began in 2006 at the PUC University, where she made an exten-
sion, but since 2000, she was an audit for process MPS. Nowadays, she is Promve’s CEO, where
she can teach that agile is a good practice in companies.
In the preliminary questions, she said that she liked the agile methodology, but she preferred
the hybrid approach better when the company has compliance to follow and standards. Agility
does not preach that it cannot have others artifacts with it and with the interpretation of agile
is more Agile benefit is for her are quality, values, increasingly better increments focus, care
more about the flow, the traditional is no longer concerned, people, continuous feedback and
transparency.
6.1 Validation Interviews 47
Figure 6.2: Process Second version
She had well-defined the ready and done criteria, traceability, and backlog items in her expe-
rience with certifiable projects. She still did not saw any task that some regulation has that could
not be applying in agile methodology or with quality management. She gave an example of one
standard that had the security as a non-functional requirement that had to be for the definition of
ready. Also, the LGPD had to be compliant too to put the product in the market. She uses check-
lists of security for the PO to remember the requirements needed for security like automation tests,
and if the automatic tests do not succeed, they have to do the test cases. According to her, having
the standard is just one more requirement to worry when developing the product, but it is just that.
The good practice is there. Agile is not not to do documentation but doing useful documentation,
well-documented, like using BDD. Every time she had a standard, she compares and evidence that
the project could be agile.
She recommends that until the team reaches the pike, it is needed to do hybrid until the team
reaches the skill or even the project has the budget to do it. In her experience, she still did not
found one company that has automatized all types of testing, and she even does not know that it
is worth it the cost versus benefit. One client of hers did only automate testing in the critical part
of the system, only in the financial part. Because it has some problems in other areas does not
have much impact. Every bug they found, they correct and create another automatic test to cover
this bug for never happening again. It was a strategy to guarantee quality but not have to automate
everything because it could not be worth it.
Manages risk in agile is all the time; the methodology itself has risk management; whether the
team will deliver or not, prevention is taking the scope out. Doing Negotiation at planning, at the
Daily identifying impediments. "The methodology is already risk management itself.". In agile
contracts must contain the risk of not delivering as the Scrum has too. For tools, she recommended
48 Interviews
the burndown chart and Monte Carlo kanban to try to predict the future.
To Analia, usability must be inside the process. It is imperceptible to analyze personas and
needs to meet software quality.
The QA team member should be on the team since the beginning of the project. For compli-
ance in the project, it is vital to have a vision outside, so the product is stable and confident about
its development. One advice is to have the tool Sonar at least 70%.
When the question was about DevOps, Analia said she invested in a start-up in the health
area, and she is accountable for doing the DevOps. She defends that DevOps brings the culture
of automation, with the mindset that if it is repetitive, it should be automated. Her words were:
"Accelerates the flow from left to right, and right-to-left flow." Improve the quality of the product.
Moreover, collaboration to understand the flow and automation. She advised us to use it in every
project, if possible.
After explained the second version of the process, was given some feedback and suggestions:
• The checklist can be the definition of done;
• Risk management in the planning where the team breaking into tasks rather than a new
ceremony because she already has done this with clients;
• Put automate commit policy, traceability to the code to associate, meeting items of audit
criteria
• Every change is history, so it always works in the history risk;
• When the risk is high, needed to be approved by the PO and needed to run every automate
tests;
• Agreed with the Scrum Master monitoring the Burndown chart;
• Agreed that having DevOps in the process would make a difference to the project and added
that many projects that she worked were saved because DevOps had. To have less bureau-
cracy adding DevOps to the project is fundamental;
• GDPR or LGPD in Brazil is needed to protect the data in the software.
In addition to this experience, Analia Irigoyen stands for a hybrid process. This practice is
challenging not to adjust something, and it is all right to do it as long as to follow some criteria
like the definition of done and backlog well-defined and traceability. Other techniques that are
important, as well:
• DevOps;
• GDPR;
• Data Security;
• Automation.
6.1 Validation Interviews 49
6.1.3 Carlos’s Interview
The third and last interviewee was Carlos Rodriguez, an Agile Consultant at Radix, a company
Software factory and engineering with many projects using agile methodology, and this is impor-
tant for this thesis because of the experience routine inside the projects.
Carlos has experience in agile about four years and knew agile when it emerged—worked
in the company Nexo when he changed the company’s mindset to be agile and had great results
out of it—improved transparent with clients and colleagues. Then he was a Scrum Master and
Consultant at Radix, and PSM and CSM certify it.
For Carlos, every project could be agile, but not every project should be. It depends on the
client’s acceptance of this methodology because it needs commitment, and the customer has to be
willing to work with agile, not only to collect the benefits that agile has but to be there to work
together in the product and not every client can do that. Another point was that Carlos compared
Civil Engineering doing agile, it would not work because the build must be incredibly defined
from the beginning and cannot change in the middle like agile allowed. Agile benefits for Carlos
are the transparency and communication that is insured and the quality of deliveries because of
adaptations.
He worked in a project that needed to be certified in work security and health. In the beginning,
this was the case that the waterfall and then changed to agile and was proved to have a better result
when changed it. It was useful, especially when there are external factors.
Risk management in Carlos’s vision is done in agile in the Review and Retrospective cere-
monies, where at the Review, it is identified, and in the Retrospective is healed the risks. Besides
that, there is expectation management, which helps to mitigate the risk of client deception. Carlos
compared with a waterfall that it is not part of the process to learn from lessons learned and learn
with the mistakes made, and there is no continuous improvement. To manage the risks, he uses a
kanban board in front of the desk, which gives him visibility to look at what could have a problem
that he needs to act.
Usability for Carlos is an essential topic in the process, and it is typical for the company not
to give importance until the delivery to the client when it should be one of the most critical roles.
Usability should be analyzed before the project started and follow during the process.
Carlos believes that the QA should be inside the developer team even if it needs to be an
audit. Carlos has an initial experience with DevOps, but he already sees that should be part of the
process, but only if the team is capable; if it does not have DevOps, it will be an impediment and
will not work. "People more than tool".
After explained the second version of the process, was given some feedback and suggestions:
• Validated the process and gave a compliment about it;
• Advised to go deeper in how the risk assessment ceremony would be made and evaluated;
• Validated that DevOps must be part of the process;
• Agreed and like it that would be a QA team inside and apart as well;
50 Interviews
• Suggest to look up about the Manchester Protocol as a way to evaluate the risks.
6.2 Conclusion
The interviews were done with agile experts in the market and had to experience with different
projects included project, which is certifiable or regulated. All interviews were done by following
the Script in the appendix B and had an average one hour of discussion. All the participants agree
that all projects can be done by agile methodology, but Carlos Rodriguez made one reservation
that all can be agile, but not every project should be. Like in civil Engineering, it is difficult for a
start over in the middle of building construction. Although, not one said that because of regulation
or certifiable projects should be doing in a waterfall or similar. When it is about the benefits of
using agile methodology in projects, all of them agree that deliver value for the client, and the
other both believe in the quality of the delivery. Other benefits were mentions, like:
• Know the process from start to finish and learn.
• Make mistakes fast so it could be sorted out fast too because of constant feedback.
Furthermore, there is a change of culture inside the company that helps to change the mindset. All
the interviewees had experience in projects that need to be certifiable or regulated. All of them in
experience had to adapt the process to be possible for been compliance. Samira Tavares advice
that been agile is also adaptive to the projects, the situations and search for the best result for the
client and the team. She had one experience that merged the Scrum and Kanban process to focus
on the flow, client, short cycle, and continuous improvement; this helped in the project, which was
regulated.
All interviewees agreed that inside agile methodology already has risk management linked
inside the process, and it is undeniable that it is necessary for all projects, mainly if it is regu-
lated. There were although two different perceptions about it, in the experience of Carlos, the risk
management is done in two moments: The first moment at the Review with the management of
expectations with the risk being mitigated and avoiding customer disappointment, and the second
at the Retrospective that the lessons learned and the project’s continuous improvement is made
remedying the risks. Comparing with the waterfall that is almost certain that will not have lessons
learned, the agile methodology has the advantage of always improve from the last project or sprint.
Analia has already declared: the methodology itself is the management of risks in Daily planning.
For Samira, the advantage of using agile methodology to manage risks is that it is natural to work,
and the risk arises earlier than in a cascade project, and therefore it is easier to control and miti-
gate, besides being more transparent for everyone and clear. She also mentions that the regulated
scenario would be good to have one ceremony that these risks could be identified and evaluated. In
this study, this feedback was very appropriate since there is a standard accountable when the sub-
ject is a medical device. The author changed the process version. In the middle of the interview, it
was interesting for the author to hear the opinion of which person on the project should do the risk
management because at the time, in the author’s mind, it was clear that it should be the PO, but the
6.2 Conclusion 51
answer that all the interviewees gave was the integral team together, and they convinced the inter-
viewer too. It was asked it had any tools for doing this risk management they recommend, and all
of them said Kanban and others were mentioned like burndown chart and flight level. This study
had worried about usability because it is always downgraded in projects, and when it is delivered,
it is alarming cause it is exactly where the customer will criticize. In addition to having an even
greater risk, which is the end-user not knowing how to use the product and the medical context, it
is a very high risk to be taken when they can have lived at risk. So in the interview, we were asked
the importance of including usability at the process, and they all agreed that it is essential and
involves the software quality. The usability being a part of the process was also a lesson learned
observing the academic case; they also worried only in the delivery about the usability and was
too late to change it. So the proposal here is to put the usability at the beginning of the process.
Another question was about the two processes proposed, the previous work proposed to be a QA
team a part of the development team, and the present process was studied with the base that has
the QA team apart. All interviewees agreed that must have the QA member inside the team cause
needs to follow the product during the whole process to assure the quality, but it is also understand-
able to have another QA team or member outside to have an audit eye of the project, especially
if is regulated or certifiable. The process was updated to have a QA team inside and outside with
distinct but complementary roles because of this feedback. There was also advice to have tools
to assure quality like the software Sonar, a tool to analyze static code. Another contribution of
this thesis was to put DevOps inside the process because that was asked to the participants as if
they had already experience in DevOps, only two of them had and were asked the different they
experience in projects that had DevOps. Analia, who has breathed DevOps and is certified in it,
said it is a big difference to have DevOps in the process that ensures quality, accelerates the flow
from left to right, and right to left, which go along with the flow and the automation. However,
even Carlos, who has only an initial experience in DevOps, said that should be part of the process
if the team has the skill or trains them to have to implement this.
52 Interviews
Chapter 7
Solution
This chapter presents the difficulties when developing the medical device’s final solution, the areas
that were analyzed, and more deeply studied to be incorporated in the process created. This initial
solution will have them based on the lessons learned from the previous work to try to solve these
limitations:
• Start following the process at the conception part to understand more of the product and
what process is going to work better;
• Test a process with the QA Team inside the developer team.
• Try other techniques with agile that are more common for developers to use, than TDD;
• Improve the visibility of test scenarios by having one tester in the developer team.
7.1 Problem
Compliance with Medical Device Framework (MDF) can be a burden when it comes to costs of
resources and labor.[30] However, it has a profit later since the medical area is always growing,
especially software. More than 50% of existing medical devices depend on software in some form
or another.[7]
The waterfall methodology is standard when the subject is project regulated. It is a safe plan,
with scope fixed, which has more years of inexperience. Use agile in regulated projects is still a
mindset that will be poor in documentation and evidence that the certificates are needed. Besides
that, we still have a problem with many releases that the waterfall is not so suitable to do it.
7.2 Contribution
This section presents the areas that will solve the previous limitations and were analyzed to inte-
grate the process with medical devices, so it would be possible to use agile and still be compliant
with the regulation.
53
54 Solution
7.2.1 GDPR
Data is valuable, and that’s why the world had to change to protect personal data. There are many
cases that the software is free, but the user gives the data that will earn money with it. This medical
field scenario is much more sensitive, especially when the data is about the user’s health situation.
This data must be in control because it can have a significant impact on patient’s lives. The main
goals of GDPR are to ensure the protection of privacy data by giving a choice a way to access the
data and elimination.
The academic case that was observed by the author in this study had an MVP compliance with
GDPR. GDPR is a right for every citizen. According to EU data protection law, organizations must
not process personal data unless they can provide a justification expressly recognized by law.[13]
In the context of the medical field, health data is classified by this article as "data related do
health," as described in Article 4(15) of the GDPR. [29] This demonstrates how sensitive it is to
deal with data that are related to health information. Processing health data is prohibited unless
the data subject has given her explicit consent for one or more specified purposes or the processing
is necessary for health or medical purposes. [13]
Moreover, there is also the medical confidentiality that the patient has to trust in the doctor to
tell all the truth about his condition. In the context of software, these details are fundamental to
delivering product safety for the patients’ data. Because of that, the author was treated with items
in the backlog so that the software would be compliant with all.
GDPR will contribute to the data’s security, especially when about a patient. It is essential that
the data is protected and more than that is to guarantee that the patient has access and management
with the possible data created. This improvement was observed in the academic case and was
essential to contribute to the process and be a part of it.
7.2.2 Usability
Usability usually is the less thing to think about in software. Even projects that should have some
innovation in the design are left for the last minute. Functionalities are more critical than ’waste’
time on this.
Thinking only in a beautiful design indeed doesn’t have the same impact as thinking about a
complex screen for an inexperienced user to use on a patient. In this scenario, it is absurd that who
built it didn’t think about the complexity and life at risk.
A growing number of medical devices are being used to monitor and treat patients, and errors
in use, leading to patient harm, have been increasingly a cause for concern. Such errors may be
due to poor device design, particularly where a complex user interface is involved. [27]
Healthcare evolves and could have less skilled or even unskilled users, including the patients.
It must be a concern when building a medical device software that is going to use and try to
make the best usability possible because one mistake from the user could be fatal. In Figure 8 it
is possible to see how human factor affects the outcomes. This image was adapted from FDA’s
guidance for usability.
7.2 Contribution 55
Figure 7.1: Human factors affect outcomes of using medical devices [27]
To do a good usability engineering process to prevent such errors from happening, need to
follow these stages:
• Identification of uses, users, use environments, operational contexts of use and training (Use
specification);
• Identification of know use problems;
• Risk assessment of use and use error;
• Formative and summative evaluation;
• Selection of tasks for evaluation;
• Develop user interface specification and human factors engineering plan;
• Formative testing during product development;
• Summative testing (Validation of the instructions for use);
• Validation of device or system;
• Training;
• Summative testing reporting (all use errors identified);
• Human factors summary report. [27]
Usability engineering should be incorporated into the product design from conception to the
device’s final validation. [27] It must be inside the process. In this study, it was observed in the
case reported in chapter 5 that if the developer team does not consider the usability at the beginning
56 Solution
of the project, it will have rework later. If the user does not know how to use the product, it will
not be used and will not create the confidence and security that a medical field system must-have.
Analyzing this is common to think that needs to be a flexible approach to sustain all the
changes in this product life cycle. An agile approach could be the answer to help with these
changes since work with iterations and adaptability. With better usability, the software will be
more safety to use.
Usability being a part of the process will mitigate the risk and the limitation of Delivery for
the client, which no one wants to use.
7.2.3 Risk Management
There is a false common sense that in agile methodology, there isn’t a management of risks.
Nevertheless, which items are going to the Sprint backlog and the boundary of not include changes
when the sprint started or create changes and re-prioritize the items in the backlog with the possible
changes. When the team estimated the points, every planning is analyzing if it could deliver or not
have the negotiation with the PO, and there is an alignment of the risks if it put more points or less
for the team to work. This approach is risk management inside of Scrum.
Although when the subject is standards is common thinking that will be needed well-documented
evidence but the manifesto agile don’t say that documentation is forbidden, just "Working soft-
ware over comprehensive documentation" [9]. However, it is vital for being regulated, and it is
possible to do it with an agile methodology.
With this mindset creating a ceremony to deal with risks may sound redundant, but if the
project depends on been certifiable or standards by regulations, it is essential to evidence what it
demands. Even in the last work done had a concern for risk control during the sprints. If there
were a ceremony, it would be easier to manage and be part of the process itself, like doing the
Sprint Planning.
Because of that, in the context of the medical field, there is a standard 14971 that is especially
for medical devices, which must be respected. There are two outputs that the standard [4] the risk
management plan and the risk management file, wherein the risk management file must have the
following items[40]:
• The risk analysis;
• The risk evaluation;
• The implementation and verification of risk control measures;
• The assessment of the acceptability of any residual risks.
These items can be a guide when developing the project that must have this standard. When
controlling the risks in a system that could have lived in danger, it is a minimum requirement.
Risk management is vital for the product to be compliant with the standard IEC 62304:2006
[19]. That is why it must be part of the process even when agile is already an exemplary process
7.2 Contribution 57
for risk management because of the standard, and it was needed more pieces of evidence and
practices inside the process.
7.2.4 DevOps
DevOps came from Development plus IT Operations, so the meaning is the junction of both to
work together to improve the collaboration and productivity by automating everything. DevOps
enables the organizations to create a safe system of work, where even small teams can quickly and
independently develop, test, and deploy code and value quickly, safely, securely, and reliable to
customers. [23] If this is so good, why not every project has it? Needed skills, change of mindset,
and tools to organize and make it happen and not always is possible the investment. However,
the result is exact, and the company would be more competitive in the market and Delivery even
more value to the client. The DevOps practices and culture are aligned with and complement
agile software development practices by integrating, testing, and deploying applications at a rapid
pace. [35] In the context of regulated software in medical devices, DevOps could be even more
appreciated since it could automate some demands and assist in training a developer in a project
based on IEC 62304. This standard does not specifies the practical execution of theses processes
and activities [24]. Thus it is possible with DevOps to apply the compliance. In this journal article,
they proposed the following list[25]:
• Item tracking across tools should be standard practice;
• Tools should include standard templates that comply with regulatory requirements;
• The tools should work hierarchically;
• The tools should guide the developer to follow the workflow in a fashion that complies with
the regulatory requirements.
The article also mentioned that the documentation should be integrated into the delivery
pipeline to increase the confidence of regulatory authorities that all the documentation required
has been created. [25]
To accomplish that, DevOps be a natural choice in regulatory projects. It should be clear that
with that in the process, it would be easier for the developer to deal with the requirements of
tracing, documentation, and deployment, and automating tedious but necessary activities. [25]
In the study of DevOps approach with the regulation IEC 62304 and IEC 82304-1 was deep an-
alyzed that has some obstacles but also has benefits that would help to accomplish with automated
processes. This is illustrated in the image below from the article. [24]
58 Solution
Figure 7.2: Summary of the most observable clauses in the standards with DevOps [24]
Therefore, DevOps is an excellent option to improve parts of the process but still need other
tools to help in some aspects like traceability. But, it would be fulfilled as automated environment
deployment tools.[24] Because of that, DevOps contributes to this study and part of the final
process proposed by this thesis.
7.2.5 Non-Conformance Report
During the academic case analysis, the author needed a report to summarize the evidence of the
non-compliance, so it would be possible for the team to put the project compliance with the reg-
ulation. Besides, the previous work done by Zamith [41] had inside the checklist a executive
summary that inspired the author to do a separated non-conformance report.
So it was needed a formal report to illustrate what was missing and a possible solution for the
non-compliance, and with that, it was able to put in the backlog so it would be developed by the
team and incorporate into the process. In the academic case, was adapted to the process illustrated
in the chapter 5.
However, in terms of contribution was made one more item in the flow illustrated in the Fast
Sheet in the appendix C. This report helped to summarize and expose non-conformities so that the
development team treated them according to the product owner’s priority.
Below is an example of the report done by the author when was analysing the academic case.
7.3 Solution 59
Figure 7.3: Non-Conformance Rerport
7.3 Solution
The process created was joined between the topics cited before in this chapter, the research of
process and regulations in the chapters 2 and 3 the process was made to be compliance with:
• Regulation IEC 62304:2006[19]
• ISO 14971:2019 [4]
The process will be detailed in this section, and the fast-sheet could be visualized in the ap-
pendix C. This section will contemplate the ceremonies, the management needed, roles and re-
sponsibilities, and the proposed flow.
7.3.1 Product Discovery
Building a product is necessary for the phase called product discovery. This phase is where the
idea and the product are molded, and the vision is formed. It is a critical phase that must be taken
into consideration in the project. Because of that was thought to create a product vision and the
product backlog with the Product Owner’s assistance. This phase should also clarify the items that
must be in the Backlog to make the product compliance. This phase could be one month or more,
should also have meetings with the stakeholders to decide priorities. Besides that, we must have
an interview with the end-user, whether patient or doctor, should also be carried out in the medical
field.
60 Solution
The product owner is accountable for this phase, and the medical device must have the security
classification [19], one Plan Document which must contain evidence about the decisions made for
the development process, the product backlog, and an architecture document. [41] Besides that, it
will have product prototypes with high fidelity and low fidelity to preview the product’s usability
and functionalities.
7.3.2 Product Delivery
It will be similar to the methodology Scrum after the product discovery has the iterations to develop
the product. The developer team will decide the scope of each iteration. This ceremony includes
the developers and also the quality assurance member. The audit member outside the team will
participate in some ceremonies.
Each Sprint will start by Sprint Planning, where the developer team will evaluate each item
prioritize by the Product Owner for the Sprint. If it is the first Sprint with the team, it will be some
reference from previous teams in the company, but if there are not any references, it will be with
time how many points the team can Delivery. The author recommends starting with reasonably
Fibonacci points or any metric that can be evaluated later. The quantity of the metric used will
always be depending on how many weeks will work the project; it could be two to four weeks, and
the total points will fluctuate depending on this and the team. A mature team will always deliver
the same metric per Sprint, which means that the team is mature and remained stable for changes
in members.
The Sprint Planning should be a timed box according to the number of weeks chosen. Not
controlling the time box is to waste the team’s time in a meeting when they can develop. During
the planning, the team will vote for each backlog item, and the whole team has to reach a consensus
on each story’s complexity. After voting the stories and understanding them, it will result in the
Sprint Backlog.
During the Sprint, the team should meet for the Daily Sprint with the time box of 15 minutes,
where they will talk about the impediments and exchange advice between them to always improve
technically. The development team is accountable for automate the flow and manage the DevOps,
besides the quality of the code using tools like Sonar [5] to analyze static code and doing code
review to assure the quality and every story should be tested by the QA member inside the team to
be considered Delivery including usability tests and GDPR concerns.
After the weeks of Sprint, it will happen the milestone of the Sprint Review. The Review will
have the following members: the Product Owner, developer team, Audit QA, Scrum Master, and
possible stakeholders will see the Sprint result.
After the Delivery at the Review should do the QA Checkpoint, which the Audit QA will
validate the product by doing the checklist. The following checklist will guarantee that the product
is compliant with regulations; after doing the checklist, if the QA Audit found any item that should
have been done by the developer team or the process to be compliant, the QA Audit will write a
non-conformance report reporting what should have been done in the project. This report will be
a deliverer to the PO, prioritizing the other items in the Product Backlog.
7.3 Solution 61
After the Sprint Review, all team members should do the Sprint Retrospective, where contin-
uous improvement happens. In the Retrospective, it is essential to point out the good, what could
be better, and the bad things during the Sprint. The good things will be indicated by the team to
encourage to continue the excellent work, and the improvements parts are to get even better the
process of work or anything, and for last, the bad things should take actions that will help to solve
the problem, and the Scrum Master can help in the process to remove the impediments and help
the team.
After the Sprint Retrospective to achieve a more quality work about risks when the context is
regulated, medical device’s products, the author created a ceremony Risk Assessment Ceremony
to discuss the risks lifted during the project, think about solutions, evaluate them, and mitigate if
necessary. Some risks exist only to call attention. The ceremony will assist the team. Everyone
on the team should participate, included the Audit QA that will help in the evidence necessary for
this ceremony.
The Scrum Master will act near the developer team, analyzing the efficiency metrics, and help
out, and the product owner should always work for the next Sprint Backlog to be ready for the
next Sprint. When the PO prioritizes the Backlog, the PO should do a Refinement to explain the
user stories, and the developer teams have a voice to say more scenarios or improvements or even
technical unfeasibility.
This process is illustrated in the image below.
Figure 7.4: Final Process proposed.
7.3.3 Improvements
This initial process comparing to the final process was that included the folow items:
62 Solution
• Usability was most of the time deprioritized by projects and was not part of the process, and
the improvement was that Usability be a part of the product discovery in the final process.
Include the Usability in the process improves the product by any user to use it correctly and
intuitively. Use the software correctly is always essential, but when the user’s software is not
from the IT area, like doctors or nurses it is crucial. Usability will help avoid functionalities
that could be used in the wrong way and avoid accidents in treatments, at least from users’
errors.
• DevOps was added to the process by the author, so it would help to automate the process
that would bring agility for the development flow and help the development process. It also
would full-fill some requirements demands by the regulation like deployment repeatability.
• The new ceremony Risk Assessment Ceremony was necessary because the regulation needed
a more deep process about risks. It must have risk analysis, risk evaluation, implementation,
and verification of risk control measures and acceptability of any residual risks. This cere-
mony would generate and evolve the management of risks and contribute to the whole team
is responsible.
• Non-conformance Report was created in the academic case because the author needed a way
to summarize the audit checklist to show the team they had to do to become certifiable. So
the report was done only with what they need to implement in the project.
• GDPR was another improvement that became with the observation of the academic case.
GDPR was not a requirement from the regulation, but everyone has the right to own their
data. Especially about health. Because of that, GDPR was part of the process to contribute
to quality and security for the users.
These items were to solve some limitations found in the previous work done, the academic
case, and the interview’s feedback.
Chapter 8
Conclusions and Future Work
This chapter discusses the results obtained by the approaches used: the research, the academic
case, and the validation interviews. It presents the conclusion and future work.
8.1 Conclusions
At the planning of this study, one goal was to use the process in a project being developed in a
company, but due to the COVID pandemic, it was not possible. This thesis was in the middle of the
pandemic, so the alternative was to observe an academic case, which was very much appreciated
because the students were very pleased to help since the author would contribute to the quality.
The students were well organized and willing to do a good job even with the distance, which
was another consequence of the pandemic. The meetings were always online, and so were the
ceremonies. The result would be better if it were able to put the process in a real company but
still some outstanding improvements that in the previously proposed process were not considered
like GDPR (that was a concern from the beginning of the project) were observed. Even with all
that, the author still would recommend testing the final process in the market to be more faithful
to the deadlines and pressure. The subject was still a deadline for delivery, which helps with the
pressure part. To not overcharge the team, the proposal was for the author to do it an inside job
helping the quality so it would be possible to observe the process in action and to see the possible
improvements and observations:
• Usability be part of the project, preferably at the beginning;
• GDPR and Security of data also included in the process;
• Quality team must be included in the process and also needed an audit team outside the
process;
• There were difficulties in implementing CI/CD;
• For the team was natural to version the code and to have issues to mapping them with the
commits using Git Lab;
63
64 Conclusions and Future Work
• For the team was easy to understand what should have been done with the user stories and
the definition of done defined by them.
In the second validation, the interviews, the choice was made since the previous work already
validated with engineering experts who had experience with critical systems. [42] To change the
validation and go deep in the study of the methodology was chosen for this thesis, some agile
experts also had experience with regulated projects. The interviews were a way to collect more
experiences and validate if the process was too far from reality. The results were very satisfactory,
were suggestions and opinions that were divergent. However, all of them contribute to the final
version proposed in this thesis. In every company, in typical days, the workers must adapt to
changes, adapt to the client, and adapt to projects. One thing that they should not adapt is the
process itself. The process has to change to be adequate for the way people work. This mindset is
not an easy change because it is a change of mindset that the agile methodology and others wanted
to change.
In the interviews, it was clear that none of them do only one framework or process. They
joined several of them, collecting the substantial parts of each, and applying where it should. That
was this thesis was all about, change one process so it could be improved with more techniques and
embrace the commitment to be compliant with the standards even being also committed to people.
People made the process, product, or project works. So if the process is improved to think about
people and helped facilitate compliance itself, it is the best solution possible. Many techniques can
solve most of the problems shown in the previous process and projects. Agile teaches that needed
to evolve people and processes to fit the purpose of the goal. Discussing the experts’ solution
was possible to see the improvements made were life-changing because it helps developers to
program with security and tools that they were used to work independently if the project needs
regulation. There is nothing wrong with using a hybrid solution if the goal is archived and all
of the interviewees agreed with that. That is why it is essential to put this theoretical process in
practice because it will need to adjust to each project’s case.
Maybe the team has not to skill in DevOps, so until they are trained, the items that DevOps
accomplish at the project needed to be adapted. Another example, maybe the company sold wrong
the project and cannot afford two quality teams. What than? There are no right answers, only
mitigating the possible risks the project itself could have. In the first case, the developers would
demand to track the code and commits to secure the delivery. In the second case, the inside
member of the quality team would be more demanding of being the two roles. At the beginning
of each sprint, reviewed the last one as a QA audit and, during the sprint, could follow the tests
according to the developers’ delivery.
Another issue was deciding if should have a separate ceremony for risks or not, for one inter-
viewed was not necessary, she already had this experience and adapt one ceremony from Scrum
itself to prolong and discuss the risks. However, the other two interviewed were clear that it would
accomplish more knowing the ceremony’s goal with a separate ceremony. The weight of this
decision was difficult. In the author’s experience, it would prefer a separate one but got always
thinking about the goal. To clarify more, the author asked one colleague that is a systems architect
8.2 Future Work 65
and developer what the developer would prefer: extend the already existing ceremony or making
a new one. The answer, surprisingly, was to make a new one because of the time box. The already
existing ceremonies sometimes take too long to end, and by the end of it, the developers are tired
and do not even think much more to make such sensitive decisions that are risk management. So
it was it, the decision was for creating a new one, and discussing this with the experts was clear
that all teams should be in the ceremony. Because of that, the process has a new ceremony called
Risk Assessment Ceremony (RAC).
The author took two certifications in agile during the thesis: Certified Scrum Master and "Cer-
tified Scrum Product Owner." These certifications helped to understand more of the pure Scrum
because the author works in a company that uses a hybrid method. The course was beneficial for
this study and gave many insights that the author used in the contribution.
The author’s experience in the market was that not always the first phase, Product Discovery
has the vital importance of starting the product. If the first phase is not well constructed, there is an
excellent chance that in the end, the delivery will not be with value, and that is with the worst-case
scenario, which is that the software could be delivered on time, with all the stories but at the end,
the user did not need it and because of that will not use at all. So in the proposed process, the
product discovery is always on the flow with the sprints. This phase is where the Usability got to
be planned, and the product vision can be born and got to be changed and put this in the process,
the non-conformances can be explored and becoming user stories. This phase will learn in each
iteration how to create more backlog that changes with the product itself.
For the project to be compliant with most standards having metrics during the project is funda-
mental, and it seems appropriate that the role Scrum Master who facilitates the team should look
at it the efficiency metrics and analyze the risks and work to try to mitigate them during every
sprint.
Therefore, this research contributes to a final process for companies that would help the team
meet demands compatible with the IEC 62304: 2006 standard, and is updated with agile technolo-
gies and tools that facilitate the work of employees, and the company’s efforts and budgets.
8.2 Future Work
This study focused on an agile methodology and improved the process with techniques that would
help the developer team. Nevertheless, due to the scenario of COVID-19 was not possible to
validate the process in any company because all of them were in the home office and with other
problems due to the crises. Because of that, for future work, it would validate the final process in
this study in some company to observe if it was useful and make improvements if necessary and
them validate the process again until is found some process that will work very well in the routine
of the developer team and the company itself.
Another complement would be to analyze this process with different medical device classifi-
cations and improve or make adjustments if necessary, to adapt to every classification.
66 Conclusions and Future Work
References
[1] Git Lab. https://about.gitlab.com/.
[2] Google Meet. https://meet.google.com/.
[3] Slack. https://slack.com/intl/pt-pt/?eu_nc=1.
[4] ISO 14971:2019 - Medical devices — Application of risk management to medical devices,2019.
[5] Sonarqube. https://www.sonarqube.org/, 2020.
[6] Pekka Abrahamsson, Outi Salo, Jussi Ronkainen, and Juhani Warsta. Agile software devel-opment methods: Review and analysis. VTT Publications, (478):3–107, 2002.
[7] Stephen Allen. Medical device software under the microscope. Network Security,2014(2):11–12, 2014.
[8] Andrew Powell-Morse. Extreme Programming: What Is It And How Do You Use It?, 2017.
[9] Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Mar-tin Fowler, Ames Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, BrianMarick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas.Manifesto for Agile Software Development, 2001.
[10] Lou Bichard. Configuration Management: What Is It and Why Is It Important?, 2019.
[11] Stefania Bruschi, Le Xiao, Mihir Kavatkar, and Gustavo Jimenez. Behavior Driven Devel-opment ( BDD ) A Case Study in Healthtech. pages 1–12.
[12] Casey Valentine and Mc Caffery Fergal. Med-Trace: Traceability Assessment Method forMedical Device Software Development. Eurp SPI Conference 2011, 2011.
[13] Lothar Determann. Healthy Data Protection. SSRN Electronic Journal, apr 2019.
[14] Jacky Estublier. Software Configuration Management : A Roadmap. 2000.
[15] European Parlament and Council of the European Union. Regulation (EU) 2017/745 of theEuropean Parliament and of the Council of 5 April 2017 on medical devices. Official Journalof the European Union, 60(L117):2–175, 2017.
[16] McCaffery Fergal, Marion Lepmets, Trektere Kitija, Özden Özcan-Top, and MinnaPikkarainen. Introducing Agile Practices into MDevSPICE. International Journal on Ad-vances in Life Sciences, 8(1):133–142, 2016.
67
68 REFERENCES
[17] Gsma. mHealth and the EU regulatory framework for medical devices.Http://Www.Gsma.Com, pages 1–10, 2012.
[18] Hive Health. Product Vision, 2020.
[19] IEC. International Standard IEC 62304. Medical device software – Software life cycle pro-cesses, 2006.
[20] IMDRF Management Committee. Statement regarding Use of IEC 62304:2006 “Medicaldevice software - Software life cycle processes”. 2015.
[21] INFARMED. Infarmed - Medical Devices.
[22] Culture Is, Still A Thing, Reign Supreme, and Agile Empowers. 14th State of Agile. 2020.
[23] Jez Humble; Patrick Debois; John Willis; Gene Kim. The DevOps Handbook. 2016.
[24] Teemu Laukkarinen, Kati Kuusinen, and Tommi Mikkonen. DevOps in regulated softwaredevelopment: Case medical devices. Proceedings - 2017 IEEE/ACM 39th InternationalConference on Software Engineering: New Ideas and Emerging Results Track, ICSE-NIER2017, pages 15–18, 2017.
[25] Teemu Laukkarinen, Kati Kuusinen, and Tommi Mikkonen. Regulated software meets De-vOps. Information and Software Technology, 97:176–178, may 2018.
[26] Martin Mc Hugh, Oisin Cawley, Fergal McCaffcry, Ita Richardson, and Xiaofeng Wang. Anagile V-model for medical device software development to overcome the challenges withplan-driven software development lifecycles. 2013 5th International Workshop on SoftwareEngineering in Health Care, SEHC 2013 - Proceedings, pages 12–19, 2013.
[27] MHRA. Human Factors and Usability Engineering - Guidance for Medical Devices Includ-ing Drug-device Combination Products. Medicines and Healthcare Products RegulatoryAgency, (September):1–30, 2017.
[28] Özden Özcan-Top and Fergal McCaffery. To what extent the medical device software reg-ulations can be achieved with agile software development methods? XP—DSDM—Scrum,volume 75. Springer US, 2019.
[29] Parlamento Europeu. Regulamento (UE) 2016/679 do parlamento Europeu e do Conselhode 27 de abril de 2016. 2016(4):210–230, 2016.
[30] Paul Quinn. The EU commission’s risky choice for a non-risk based strategy on assessmentof medical devices. Computer Law and Security Review, 33(3):361–370, 2017.
[31] Steven R Rakitin. in Medical Devices. Risk Management, pages 40–45, 2006.
[32] Gilbert Regan, Fergal McCaffery, Kevin Mc Daid, and Derek Flood. Medical device stan-dards’ requirements for traceability during the software development lifecycle and imple-mentation of a traceability assessment model. Computer Standards and Interfaces, 36(1):3–9, 2013.
[33] Pieter Adriaan Rottier and Victor Rodrigues. Agile development in a medical device com-pany. Proceedings - Agile 2008 Conference, pages 218–223, 2008.
[34] Sylvain Saurel. Introduction to Test Drive Development.
REFERENCES 69
[35] Vlad Stirbu and Tommi Mikkonen. Towards Agile Yet Regulatory-Compliant Developmentof Medical Software. In Proceedings - 29th IEEE International Symposium on SoftwareReliability Engineering Workshops, ISSREW 2018, pages 337–340. Institute of Electricaland Electronics Engineers Inc., nov 2018.
[36] Ken Schwaber Sutherland. and Jeff. The Scrum GuideTM, 2018.
[37] Aami Sw, Medical Device, and Software Committee. American National Standard Medicaldevice software — Software life cycle processes. (800), 2006.
[38] The European Parliament and the Council of the European Union. Directive 2007/47/EC,of 5 September 2007 amending Council Directive 90/385/EEC on the approximation of thelaws of the Member States relating to active implantable medical devices, Council Directive93/42/EEC. Official Journal of the European Union, L(November 2000):21–55, 2007.
[39] U.S. Food and Drug Administration. General Principles of Software Validation; Final Guid-ance for Industry and FDA Staff (Document issued on: January 11, 2002). FDA Guidance,page 47, 2002.
[40] David A. Vogel. Medical Device Software Verification, Validation, and Compliance. 2010.
[41] Manuel Zamith. Desenvolvimento de Software para Dispositivos Médicos: Uma AbordagemÁgil. PhD thesis, Porto University, 2018.
[42] Manuel Zamith and Gil Gonçalves. Towards an agile development model for certifiablemedical device software: Taking advantage of the medical device regulation. ICSOFT 2018- Proceedings of the 13th International Conference on Software Technologies, pages 132–140, 2019.
70 REFERENCES
Appendix A
Metrics Hive Health
Burndown charts about what were observed during each sprint in the academic case.
Figure A.1: Sprint 1 [18]
71
72 Metrics Hive Health
Figure A.2: Sprint 2[18]
Figure A.3: Sprint 3 [18]
Metrics Hive Health 73
Figure A.4: Sprint 4 [18]
Figure A.5: Integration Sprint 1 [18]
Figure A.6: Integration Sprint 2 [18]
74 Metrics Hive Health
Appendix B
Interview Script
B.1 Script Plan
To validate the changes made in the process was selected this script with this plan:
1. Introduce the interview with the explanation of the problem and the previous work done;
2. Explanation of the possible solution in the process and the changes made;
3. Why is important the experience of the interviewee to add to the process;
4. Make the questions listed in this script;
5. Moment for the interviews to talk about what they think about the process;
6. Thank the participants.
B.1.1 Introductory Questions
Questions to get to know more about the interviewee.
1. What experiences do you have ?
2. Do you think that all projects can be use the methodology agile ?
3. What benefits you see using methodology agile ?
4. Did you do or have done any project with regulations or that have to be certifiable ?
5. In your experience, what do you think about adapting processes in a regulated area?
B.1.2 Process Questions
6. In your opinion, what do you think about the risk management ? How is apply in the of
agile methodology? Do you know any tools that help the management? Who should do it ?
75
76 Interview Script
7. What do you think about usability be part of the process?
8. What do you think will work better, the QA Team a part or include in the development team
?
9. Do you have any experience with DevOps? If yes, it is a big difference in the process?
Should be a part of the process?
B.1.3 Validation Questions
10. What do you think about the process proposed? Any suggestions? Improvements?
11. Have any suggestions ? Comments?
Appendix C
Fast sheet
77
78Fastsheet
Fastsheet79
