E-health and multiple sclerosis

Paul M Matthews1, Valerie J Block2, Letizia Leocani3

1 Department of Brain Sciences and UK Dementia Research Institute Centre, Imperial College, London, UK2Weill Institute for Neurosciences, Department of Neurology, University of California, San Francisco, USA Valerie.Block@ucsf.edu3University Vita-Salute San Raffaele, Milan; Neurorehabilitation Dep.t and Experimental Neurophysiology Unit, INSPE, Scientific Institute Hospital San Raffaele, Milan, Italy leocani.letizia@hsr.it

Address for correspondence:Prof. Paul M. Matthews

E515, Department of Brain SciencesImperial College LondonHammersmith HospitalDuCane Road, London WC12 0NNTel: 0044 207 594 2612

p.matthews@imperial.ac.uk

Abstract word count: 203Main text word count: 2146Bullet points word count: 124

2

Abstract

Purpose of review: To outline recent applications of e-health data and digital tools for improving

the care and management of healthcare for people with multiple sclerosis (MS)

Recent findings: The digitisation of most clinical data, along with developments in communication

technologies, miniaturisation of sensors and computational advances are enabling aggregation

and clinically meaningful analyses of real-world data from patient registries, digital patient-reported

outcomes and electronic health records (EHR). These data are allowing more confident

descriptions of prognoses for MS patients and the long-term relative benefits and safety of disease

modifying treatments (DMT). Registries allow detailed, MS-specific data to be shared between

clinicals more easily and providing the additional data needed to better the impact of DMT and,

with EHR, interactions between MS and other diseases. Wearable sensors provide continuous,

long-term measures of performance dynamics in the home. In conjunction with telemedicine and

online apps, they promise a major expansion of the scope for patients to manage aspects of their

own care. Advances in disease understanding, decision support and self-management using

these Big Data are being accelerated by machine learning and artificial intelligence.

Summary: Both health professionals and patients can employ e-health approaches and tools for

development of a more patient-centred learning health system.

Keywords: e-health, registries, real-world date, wearable sensors, artificial intelligence (AI)

3

Key points:

E-health refers to the use of electronic communication tools and computers by patients to

monitor or maintain their own health and by doctors to provide better care.

Patient registries provide real-world data to supplement that from clinical trials for better

understanding prognoses, the influence of co-morbidities and long-term benefits and safety

of disease modifying therapies.

Hospital or care system electronic health records complement registries with often more

comprehensive and representiative of data concerning the health of MS patients.

Wearable sensors are adding the potential for continuous, long-term measures of

performance and behaviour in the home environment.

Emerging themes concern responsible linkage of these data and use of machine learning,

artificial intelligence and modelling methods for personalisation of decision-support and

management approaches based on them.

4

IntroductionMedicine relies on close observation of patients to inform future practice. Advances in electronics

for miniaturisation and dramatic reductions in power demands have enabled a rapid growth in

computational capabilities that has transformed the nature of this “learning medicine” model. The

rapidly growing field of e-health describes the use of electronic communication tools and

computers by patients to monitor and maintain their own health and by doctors to provide better

care. Here we will briefly outline current e-health opportunities for multiple sclerosis (MS) care and

research with an emphasis on recent developments.

Real world dataMS Registries

Balances of risk to benefit vary across disease modifying treatments for MS (DMT). Previously,

guidance for their use came only from clinical trial data. However, these data describe outcomes

limited to small segments of the population followed for the minimal periods needed for

establishing efficacy (1). With the increasing harmonisation of routine clinical and patient centred

data and the potential for its digital capture, real-world data can be used to supplement this trial

data and address a range of important clinical questions (e.g., co-morbidities, patient prognosis,

the long-term safety of therapies and clinical effectiveness) (2). Nonetheless, real-world

observational data must be interpreted in the context of its potential sources of bias (e.g., consent

for participation is subject to bias, unmonitored data collection allows variations in its quality).

However, methods for quality control and validation of “big data” are being developed (3), e.g., the

large Swedish MS Registry has recently been audited for consistency of diagnoses through

linkage of individual-level MS registry data to other nationwide registers (4) and clinical chart

review (5). There are growing numbers of registries, increasing their aggregate

representativeness (6). Recognising this, the Europeans Medicine Agency has mandated their

greater use of these data for the discharge of new drug marketing authorisation requirements (7).

Analyses of real world registry data already inform better patient care. Studies based on these

data have made unique contributions particularly for understanding outcomes in smaller patient

groups poorly addressed in clinical trials, such paediatric-onset MS (8), or to contrast trajectories

for variously stratified patient groups (9).

Digitisation and standardisation of registries now makes them relatively easy to integrate for data

across centres and countries. The most important example of this is MSBase, by far the largest

internationally aggregated registry for MS and neuroimmunological disorders (10). Recent

applications using its database have improved estimates of the risk of development of secondary

progressive MS (SPMS) (11), more robustly demonstrated that prior use of disease-modifying

5

therapy (DMT) lowers this risk and delays SPMS onset (12) and re-enforced conclusions that

current DMT do not slow disability progression in primary progressive MS (13).

Digital resources managed by people with MS

Web-based applications also can be used directly by people with MS (pwMS) to collect patient

reported outcome data. Data from Patients Like Me, perhaps the largest of these (14, 15), has

been used in many ways, such as to assess access to medicines (16) and perceptions of efficacy,

side effects and overall satisfaction with treatments (17), monitoring disease progression (18)and

informing pharmacovigilance and comparative effectiveness research, particularly for off-label

uses of medicine (19, 20). Other websites capture data specifically from pwMS (e.g., (21, 22).

However, while the promise of digital patient-reported data is clear, it has significant limitations.

For example, there is a potential for distortion of the data by unscreened contributions from people

without the disease of interest (although see recent reassuring audit data (17). The

representativeness of those contributing data also is difficult to define. Site questionnaires are

variably complete, exacerbating this problem. Moreover, there are few high quality patient-centred

questionnaires (23).

Electronic health records

Real world data concerning MS also is becoming available to researchers from routinely acquired

clinical data in hospital electronic health records (EHR) (24). EHR reduce some of the sampling

bias in registries maintained by individual clinicians (25) and can be remarkably complete (the UK

Clinical Practice Research Datalink collects de-identified patient data from participating general

practices across the UK covering more than 45 million patients across the country (26). EHR can

be used to understand associations between MS and other diseases or the impacts of treatments

for them (27). EHR allow linkages between personal and clinical data to allow relationships such

as that between socioeconomic status and MS progression (28). Even missed appointment data

provides an important source of information (29). The scope for EHR to be used in the way is

being enhanced using algorithms that enable unstructured clinical notes to be interrogated (14).

Digital tools for individual patient assessment and rehabilitationA central challenge for MS care and new drug development still lies in the assessment of disability

and its change in ways that are clinically meaningful (30). Wearable sensors that provide

continuous, long-term measures of performance dynamics represent a new frontier for e-health.

Most progress has been made for the assessment of walking and gait, disturbances in which can

present early in the disease, but often are undetectable using routine clinical tests and only

reported by the patient subjectively. (31-34). Quantitative assessments which provide sensitive,

6

objective measures of gait across the full clinical spectrum of MS are well-precedented, but

traditionally have been limited to sophisticated movement laboratories, access to which is not

practical for most clinics. Wearable technologies that patients can use in their homes offers

benefits in terms of time, cost and, if worn for extended periods of time in the home setting,

generalizability to functions of everyday living. The data produced provides the clinician with the

ability to detect, track and promptly address any changes in ambulatory function remotely between

clinical visits throughout the disease course.

Early studies in remote ambulatory monitoring in MS recorded physical activity over shorter

periods (3- 7 days) and favoured devices such as pedometers or first generation bi- or tri-axial

accelerometers. (35-37). This provoked interest in the accuracy of these data. Most studies in

MS (75%) have used activity counts from a model associated with the research-grade ActiGraph

tri-axial accelerometer (38). A recent systematic review characterised the use of these tools and

concluded that multiaxial accelerometers worn on the ankle are the most reliable of the “simple”

devices for step counting in neurological populations (39).

Future applications should incorporate more or the raw data from the accelerometer

measurements directly rather after data reduction as summary single measures (such as step

counts per day). To be most useful to clinicians, reports should more routinely include the

duration, frequency, intensity and amount of energy expended during daily physical activity (39).

Movement monitors algorithms have the potential to provide even more granular features of gait

(such as step length, step width, and cadence). In a recent study, one such approach

differentiated between healthy controls and pwMS, even when performance-based clinic

measures like the timed walk (T25FW) showed no differences (40). Use of wearables in

conjunction with other kinds of remote sensing could further improve the precision for assessing

disability in ambulant pwMS, e.g., using GPS with accelerometery to better determine walking

endurance. Notwithstanding such promising applications, drawbacks to widespread use of the

more sophisticated monitors needed include their higher cost, shorter battery life, and much higher

data generation rates and the complexities of analyses needed for rich feature extraction.

There is a growing range of devices available. Which to choose depends on goals. Longer-term

physical activity monitoring requires devices that promote adherence, are economical and user

friendly for both research team and participant. In a disease with so much inter- and intra-person

variability as MS, being able to monitor activity continuously, in the home setting, for long periods

has high value. Commercial fitness tracker wearables have most of these characteristics and are

advancing into the research space. Recent relevant studies include: Floodlight (a downloadable

7

App using smartphone and smartwatch technology) (41), FITriMS (involving continuous, remote

activity measures using the proprietary Fitbit wrist-worn accelerometer over 1 year) and SPI2 (the

first phase 3, international placebo-controlled trial to use remote ambulatory monitoring as an

exploratory outcome to determine the efficacy of MD1003: high dose biotin) (38, 42) and Biostamp

(a recent study using novel skin-adherent sensors) (43, 44).

New directions for digital healthE-health tools and applications are growing. Another emerging area of growth for e-health in MS

involves extensions of telemedicine for care rehabilitation, e.g., for creation of a telehealth support

group participation to reduce loneliness in multiple sclerosis (45) or for cognitive rehabilitation (46).

Tele-rehabilitation appears to be well-accepted with high levels of adherence and satisfaction,

supporting its feasibility for home care of pwMS (47, 48). Even people living in urban communities

could benefit from being freed of the need to visit care centres, which become an increasing

burden with greater disability (47, 49). Although there is some evidence for its potential efficacy

(50, 51), other studies have been less encouraging (52).

However, with the almost ubiquitous use of mobile phones, health self-management apps are

increasingly dominating the telemedicine environment. More than 1000 smartphone apps for

medical self-management (with more than 100 unique applications) were identified recently, of

which almost a quarter are designed for problems of pwMS (53). Both telemedicine and smart

phone apps may require customization according to clinical and demographic features of patients

using them. Some pwMS, for example, have difficulty in navigating virtual reality environments

presented on a flat screen (54), while immersive visual presentation may lead to cybersickness in

others (55).

A few promising early studies have reported combining wearables with internet-based

interventions for gait rehabilitation (33, 56, 57). Feasibility was confirmed in a recent randomised

clinical trial (58). Early efforts using a smart tablet-based app for dexterity training have been

described (59)In the future, “gamification” of rehabilitation (and testing) apps promises could

promote adherence (60). There also is a potential to extend capabilities using robotics (61).

The largest single growth areas in technology for e-health make use of machine learning and

artificial intelligence. These will transform digital health from its currents relative focus on

advanced integration synthesis more towards decision support. Neuroradiology has been one of

the first medical beneficiaries of such tools. A recent application of quantitative radiomic

biomarkers discriminated well between neuromyelitis optica spectrum disorder and MS (62). More

8

interesting is the application of such methods to discrimination problems that are fundamentally

uncertain, e.g., to test whether pwMS can be distinguished from healthy persons based solely on

Optical Coherence Tomography features (63), for the selection of patient-reported (PROs) and

clinical-assessed outcomes (CAOs) predictive of the future progression (64, 65) or using postural

sway measures for fall risk prediction (66). A limitation of these approaches is that they are

based on previously defined features in the data and have uncertain generalisability. An

advantage is that they rely on definitions of meaningful features in the images based on prior

knowledge so that the resulting ‘signatures” or models can be explained.

By contrast, AI attempts to reproduce the behaviour of the human brain as it makes decisions

based directly on “real world” inputs rather than on pre-defined features. Applications of AI involve

training an algorithm on a representive dataset, followed by validation with independent data. As

new kinds of data are encountered, the algorithm can be tuned to progressively improve

performance. The approach already has been applied widely in neurology to enhance the

interpretation of complex signals, e.g., by dimensional reduction, classification or image

parcellation (67). The origins of these tools in computer systems performing tasks that normally

are done by the human brain makes them ideal for the quantitative perception or scoring of

images, as demonstrated in a recent pilot for automated rating of the Brief. Visuospatial Memory

Test-Revised (BVMT-R), a recognized method to measure optical recognition deficits and their

progression (68). Another emerging area in e-health that is beginning to have an impact on MS

care involves use of the “internet of things”, the potential network of inter-relatable sensors or

devices that can interact with patients as part their daily lives (69). We are not aware of

applications to MS at present, but they can be expected soon.

ConclusionsE-health is enabling the development of an increasingly robust “learning health system” for MS. It

has created opportunities for large scale integration of datasets across centres and countries to

generate datasets large and comprehensive enough to characterise less common populations and

events and support the development of more meaningfully predictive models of individual

prognoses. The latter promise greater personalisation of care and more rapid discrimination of

the relative benefits and risks of new DMT. Patient centred data and tools should better empower

patients and improve health outcomes by enabling greater self-management of care. E-health

tools also are contributing to improved clinical trial designs for early proof of concept studies (70).

Neurologists and other care providers thus need to develop both the skills needed to integrate e-

health tools fully into clinical care and for a full understanding of the limitations and risks of their

use (71).

9

AcknowledgmentsResearch underpinning this review was funded by the Progressive MS Alliance and the Medical

Research Council. Infrastructure was supported by the National Institute for Health Research

(NIHR) Biomedical Research Centre (BRC).

Declarations of InterestPMM acknowledges generous personal and research support from the Edmond J Safra

Foundation and Lily Safra, an NIHR Senior Investigator Award and the UK Dementia Research

Institute. He is reimbursed for service on a Scientific Advisory Board to Ipsen Pharmaceuticals.

He has received consultancy fees from Roche, Adelphi Communications, Celgene and Biogen

OrbiMed. He has received honoraria or speakers’ fees from Novartis and Biogen and has

received research or educational funds from Biogen, Novartis and GlaxoSmithKline. L. Leocani

has received honoraria for consulting services and/or speaking activity from Merck, Biogen,

Novartis, Roche, Almirall, Excemed.

References

1. Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. Jama. 2007;298(10):1209-12.

***An influential discussion of the limitations of clinical trial data that provides use of strong rationale for real world data to guide patient care.

2. Cohen JA, Trojano M, Mowry EM, Uitdehaag BM, Reingold SC, Marrie RA. Leveraging real-world data to investigate multiple sclerosis disease behavior, prognosis, and treatment. Mult Scler. 2019:1352458519892555.

3. Kalincik T, Kuhle J, Pucci E, Rojas JI, Tsolaki M, Sirbu CA, et al. Data quality evaluation for observational multiple sclerosis registries. Mult Scler. 2017;23(5):647-55.

4. Murley C, Friberg E, Hillert J, Alexanderson K, Yang F. Validation of multiple sclerosis diagnoses in the Swedish National Patient Register. Eur J Epidemiol. 2019.

10

5. Alping P, Piehl F, Langer-Gould A, Frisell T. Validation of the Swedish Multiple Sclerosis Register: Further Improving a Resource for Pharmacoepidemiologic Evaluations. Epidemiology. 2019;30(2):230-3.

6. Glaser A, Stahmann A, Meissner T, Flachenecker P, Horakova D, Zaratin P, et al. Multiple sclerosis registries in Europe - An updated mapping survey. Mult Scler Relat Disord. 2019;27:171-8.

7. EuropeanMedicinesAgency. Patient registries 2017 [Available from: https://www.ema.europa.eu/en/human-regulatory/post-authorisation/patient-registries.

8. McKay KA, Manouchehrinia A, Berrigan L, Fisk JD, Olsson T, Hillert J. Long-term Cognitive Outcomes in Patients With Pediatric-Onset vs Adult-Onset Multiple Sclerosis. JAMA Neurol. 2019.

9. Murley C, Mogard O, Wiberg M, Alexanderson K, Karampampa K, Friberg E, et al. Trajectories of disposable income among people of working ages diagnosed with multiple sclerosis: a nationwide register-based cohort study in Sweden 7 years before to 4 years after diagnosis with a population-based reference group. BMJ Open. 2018;8(5):e020392.

10. Kalincik T, Butzkueven H. The MSBase registry: Informing clinical practice. Mult Scler. 2019;25(14):1828-34.

11. Fambiatos A, Jokubaitis V, Horakova D, Kubala Havrdova E, Trojano M, Prat A, et al. Risk of secondary progressive multiple sclerosis: A longitudinal study. Mult Scler. 2019:1352458519868990.

12. Brown JWL, Coles A, Horakova D, Havrdova E, Izquierdo G, Prat A, et al. Association of Initial Disease-Modifying Therapy With Later Conversion to Secondary Progressive Multiple Sclerosis. Jama. 2019;321(2):175-87.

**A powerful demonstration of how aggregated real world electronic data from across the world can be used to address fundamentaily important questions in MS care.

13. Lorscheider J, Kuhle J, Izquierdo G, Lugaresi A, Havrdova E, Horakova D, et al. Anti-inflammatory disease-modifying treatment and disability progression in primary progressive multiple sclerosis: a cohort study. Eur J Neurol. 2019;26(2):363-70.

14. Van Le H, Le Truong CT, Kamauu AWC, Holmen J, Fillmore C, Kobayashi MG, et al. Identifying Patients With Relapsing-Remitting Multiple Sclerosis Using Algorithms Applied to US Integrated Delivery Network Healthcare Data. Value Health. 2019;22(1):77-84.

15. Williams III DW. The PatientsLikeMe ® Multiple Sclerosis Community: Using online marketing to shift the health data privacy paradigm. Journal of Communication in Healthcare. 2010;3(1):48.

16. Simacek KF, Ko JJ, Moreton D, Varga S, Johnson K, Katic BJ. The Impact of Disease-Modifying Therapy Access Barriers on People With Multiple Sclerosis: Mixed-Methods Study. J Med Internet Res. 2018;20(10):e11168.

17. Eichler GS, Cochin E, Han J, Hu S, Vaughan TE, Wicks P, et al. Exploring Concordance of Patient-Reported Information on PatientsLikeMe and Medical Claims Data at the Patient Level. J Med Internet Res. 2016;18(5):e110.

11

18. Bove R, Secor E, Healy BC, Musallam A, Vaughan T, Glanz BI, et al. Evaluation of an online platform for multiple sclerosis research: patient description, validation of severity scale, and exploration of BMI effects on disease course. PLoS One. 2013;8(3):e59707.

**A compelling illustration of the accuracy and clinical meaningfulness of data captured directly from patients using a web-based platform.

19. Pearson JF, Brownstein CA, Brownstein JS. Potential for electronic health records and online social networking to redefine medical research. Clin Chem. 2011;57(2):196-204.

20. Frost J, Okun S, Vaughan T, Heywood J, Wicks P. Patient-reported outcomes as a source of evidence in off-label prescribing: analysis of data from PatientsLikeMe. J Med Internet Res. 2011;13(1):e6.

21. Acclerated Cure Project. What is iConquerMS™? 2013.

22. UKMSRegister. MS Register. 2020.

23. D'Amico E, Haase R, Ziemssen T. Review: Patient-reported outcomes in multiple sclerosis care. Mult Scler Relat Disord. 2019;33:61-6.

24. Damotte V, Lizee A, Tremblay M, Agrawal A, Khankhanian P, Santaniello A, et al. Harnessing electronic medical records to advance research on multiple sclerosis. Mult Scler. 2019;25(3):408-18.

***An informative review describing methods for deriving MS data from electronic health records along with an assessment of their accuracy.

25. Briggs FB, Hill E. Estimating the prevalence of multiple sclerosis using 56.6 million electronic health records from the United States. Mult Scler. 2019:1352458519864681.

26. Herrett E, Gallagher AM, Bhaskaran K, Forbes H, Mathur R, van Staa T, et al. Data Resource Profile: Clinical Practice Research Datalink (CPRD). Int J Epidemiol. 2015;44(3):827-36.

27. Briggs FBS, Thompson NR, Conway DS. Prognostic factors of disability in relapsing remitting multiple sclerosis. Mult Scler Relat Disord. 2019;30:9-16.

28. Harding KE, Wardle M, Carruthers R, Robertson N, Zhu F, Kingwell E, et al. Socioeconomic status and disability progression in multiple sclerosis: A multinational study. Neurology. 2019;92(13):e1497-e506.

29. Gromisch ES, Turner AP, Leipertz SL, Beauvais J, Haselkorn JK. Risk Factors for Suboptimal Medication Adherence in Persons with Multiple Sclerosis: Development of an Electronic Health Record-Based Explanatory Model for Disease-Modifying Therapy Use. Arch Phys Med Rehabil. 2019.

30. LaRocca NG, Hudson LD, Rudick R, Amtmann D, Balcer L, Benedict R, et al. The MSOAC approach to developing performance outcomes to measure and monitor multiple sclerosis disability. Mult Scler. 2018;24(11):1469-84.

31. Hauser SL, Chan JR, Oksenberg JR. Multiple sclerosis: Prospects and promise. Ann Neurol. 2013;74(3):317-27.

32. Motl RW. Physical activity and irreversible disability in multiple sclerosis. Exerc Sport Sci Rev. 2010;38(4):186-91.

12

33. Motl RW, Dlugonski D. Increasing physical activity in multiple sclerosis using a behavioral intervention. Behav Med. 2011;37(4):125-31.

34. Motl RW, Sandroff BM, Sosnoff JJ. Commercially available accelerometry as an ecologically valid measure of ambulation in individuals with multiple sclerosis. Expert Rev Neurother. 2012;12(9):1079-88.

35. Ng AV, Kent-Braun JA. Quantitation of lower physical activity in persons with multiple sclerosis. Med Sci Sports Exerc. 1997;29(4):517-23.

36. Hale L, Williams K, Ashton C, Connole T, McDowell H, Taylor C. Reliability of RT3 accelerometer for measuring mobility in people with multiple sclerosis: pilot study. J Rehabil Res Dev. 2007;44(4):619-27.

37. Hale LA, Pal J, Becker I. Measuring free-living physical activity in adults with and without neurologic dysfunction with a triaxial accelerometer. Arch Phys Med Rehabil. 2008;89(9):1765-71.

38. Block VA, Pitsch E, Tahir P, Cree BA, Allen DD, Gelfand JM. Remote Physical Activity Monitoring in Neurological Disease: A Systematic Review. PLoS One. 2016;11(4):e0154335.

39. Casey B, Coote S, Donnelly A. Objective physical activity measurement in people with multiple sclerosis: a review of the literature. Disabil Rehabil Assist Technol. 2018;13(2):124-31.

40. Horak F, King L, Mancini M. Role of body-worn movement monitor technology for balance and gait rehabilitation. Phys Ther. 2015;95(3):461-70.

41. Midaglia L, Mulero P, Montalban X, Graves J, Hauser SL, Julian L, et al. Adherence and Satisfaction of Smartphone- and Smartwatch-Based Remote Active Testing and Passive Monitoring in People With Multiple Sclerosis: Nonrandomized Interventional Feasibility Study. J Med Internet Res. 2019;21(8):e14863.

42. Block VJ, Bove R, Zhao C, Garcha P, Graves J, Romeo AR, et al. Association of Continuous Assessment of Step Count by Remote Monitoring With Disability Progression Among Adults With Multiple Sclerosis. JAMA Netw Open. 2019;2(3):e190570.

43. Moon Y, McGinnis RS, Seagers K, Motl RW, Sheth N, Wright JA, Jr., et al. Monitoring gait in multiple sclerosis with novel wearable motion sensors. PLoS One. 2017;12(2):e0171346.

44. McGinnis RS, Mahadevan N, Moon Y, Seagers K, Sheth N, Wright JA, Jr., et al. A machine learning approach for gait speed estimation using skin-mounted wearable sensors: From healthy controls to individuals with multiple sclerosis. PLoS One. 2017;12(6):e0178366.

***Description of innovative skin-mounted accelerometers and how machine learning algorithms could be used to derive quantitative gait speed data from them.

45. Leavitt VM, Riley CS, De Jager PL, Bloom S. eSupport: Feasibility trial of telehealth support group participation to reduce loneliness in multiple sclerosis. Mult Scler. 2019:1352458519884241.

46. Mura G, Carta MG, Sancassiani F, Machado S, Prosperini L. Active exergames to improve cognitive functioning in neurological disabilities: a systematic review and meta-analysis. Eur J Phys Rehabil Med. 2018;54(3):450-62.

47. Shaw MT, Best P, Frontario A, Charvet LE. Telerehabilitation benefits patients with multiple sclerosis in an urban setting. J Telemed Telecare. 2019:1357633x19861830.

13

48. Isernia S, Pagliari C, Jonsdottir J, Castiglioni C, Gindri P, Gramigna C, et al. Efficiency and Patient-Reported Outcome Measures From Clinic to Home: The Human Empowerment Aging and Disability Program for Digital-Health Rehabilitation. Front Neurol. 2019;10:1206.

49. Charvet LE, Yang J, Shaw MT, Sherman K, Haider L, Xu J, et al. Cognitive function in multiple sclerosis improves with telerehabilitation: Results from a randomized controlled trial. PLoS One. 2017;12(5):e0177177.

50. Fjeldstad-Pardo C, Thiessen A, Pardo G. Telerehabilitation in Multiple Sclerosis: Results of a Randomized Feasibility and Efficacy Pilot Study. Int J Telerehabil. 2018;10(2):55-64.

51. Jonsdottir J, Bertoni R, Lawo M, Montesano A, Bowman T, Gabrielli S. Serious games for arm rehabilitation of persons with multiple sclerosis. A randomized controlled pilot study. Mult Scler Relat Disord. 2018;19:25-9.

52. Khan F, Amatya B, Kesselring J, Galea MP. Telerehabilitation for persons with multiple sclerosis. A Cochrane review. Eur J Phys Rehabil Med. 2015;51(3):311-25.

53. Salimzadeh Z, Damanabi S, Kalankesh LR, Ferdousi R. Mobile Applications for Multiple Sclerosis: a Focus on Self-Management. Acta Inform Med. 2019;27(1):12-8.

**A useful review of available health apps for MS.

54. Leocani L, Comi E, Annovazzi P, Rovaris M, Rossi P, Cursi M, et al. Impaired short-term motor learning in multiple sclerosis: evidence from virtual reality. Neurorehabil Neural Repair. 2007;21(3):273-8.

55. Weech S, Kenny S, Barnett-Cowan M. Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Front Psychol. 2019;10:158.

56. van Kessel K, Wouldes T, Moss-Morris R. A New Zealand pilot randomized controlled trial of a web-based interactive self-management programme (MSInvigor8) with and without email support for the treatment of multiple sclerosis fatigue. Clin Rehabil. 2016;30(5):454-62.

57. Tallner A, Pfeifer K, Maurer M. Web-based interventions in multiple sclerosis: the potential of tele-rehabilitation. Ther Adv Neurol Disord. 2016;9(4):327-35.

58. Paul L, Renfrew L, Freeman J, Murray H, Weller B, Mattison P, et al. Web-based physiotherapy for people affected by multiple sclerosis: a single blind, randomized controlled feasibility study. Clin Rehabil. 2019;33(3):473-84.

59. van Beek JJW, van Wegen EEH, Bol CD, Rietberg MB, Kamm CP, Vanbellingen T. Tablet App Based Dexterity Training in Multiple Sclerosis (TAD-MS): Research Protocol of a Randomized Controlled Trial. Front Neurol. 2019;10:61.

60. Bove RM, Rush G, Zhao C, Rowles W, Garcha P, Morrissey J, et al. A Videogame-Based Digital Therapeutic to Improve Processing Speed in People with Multiple Sclerosis: A Feasibility Study. Neurol Ther. 2019;8(1):135-45.

61. Dixit S, Tedla JS. Effectiveness of robotics in improving upper extremity functions among people with neurological dysfunction: a systematic review. Int J Neurosci. 2019;129(4):369-83.

62. Ma X, Zhang L, Huang D, Lyu J, Fang M, Hu J, et al. Quantitative radiomic biomarkers for discrimination between neuromyelitis optica spectrum disorder and multiple sclerosis. J Magn Reson Imaging. 2019;49(4):1113-21.

14

63. Cavaliere C, Vilades E, Alonso-Rodriguez MC, Rodrigo MJ, Pablo LE, Miguel JM, et al. Computer-Aided Diagnosis of Multiple Sclerosis Using a Support Vector Machine and Optical Coherence Tomography Features. Sensors (Basel). 2019;19(23).

64. Jackson KC, Sun K, Barbour C, Hernandez D, Kosa P, Tanigawa M, et al. Genetic model of MS severity predicts future accumulation of disability. Ann Hum Genet. 2020;84(1):1-10.

65. Law MT, Traboulsee AL, Li DK, Carruthers RL, Freedman MS, Kolind SH, et al. Machine learning in secondary progressive multiple sclerosis: an improved predictive model for short-term disability progression. Mult Scler J Exp Transl Clin. 2019;5(4):2055217319885983.

66. Sun R, Hsieh KL, Sosnoff JJ. Fall Risk Prediction in Multiple Sclerosis Using Postural Sway Measures: A Machine Learning Approach. Sci Rep. 2019;9(1):16154.

67. Raghavendra U, Acharya UR, Adeli H. Artificial Intelligence Techniques for Automated Diagnosis of Neurological Disorders. Eur Neurol. 2019:1-24.

68. Birchmeier ME, Studer T. Automated Rating of Multiple Sclerosis Test Results Using a Convolutional Neural Network. Stud Health Technol Inform. 2019;259:105-8.

**A description of how artificial intelligence can be used to improve the efficiency of MS monitoring with a conventional performance test.

69. Jagadeeswari V, Subramaniyaswamy V, Logesh R, Vijayakumar V. A study on medical Internet of Things and Big Data in personalized healthcare system. Health Inf Sci Syst. 2018;6(1):14.

***A look towards the future and how the home "internet of things" can be harnessed to improve healthcare for chronic diseases.

70. Mullins CD, Vandigo J, Zheng Z, Wicks P. Patient-centeredness in the design of clinical trials. Value Health. 2014;17(4):471-5.

71. Wicks P, Chiauzzi E. 'Trust but verify'--five approaches to ensure safe medical apps. BMC Med. 2015;13:205.***An important argument for care in properly striking a balance between innovation and caution with the introduction of medical apps to ensure that patient safety is maintained paramount.