DEBATE Open Access

Empowering patients through eHealth: acase report of a pan-European projectEmanuele Lettieri1*, Lia P. Fumagalli1, Giovanni Radaelli2, Paolo Bertele’1, Jess Vogt3, Reinhard Hammerschmidt3,Juan L. Lara4, Ana Carriazo5 and Cristina Masella1

Abstract

Background: This paper crystallises the experience developed by the pan-European PALANTE Consortium indealing with the generation of relevant evidence from heterogeneous eHealth services for patient empowermentin nine European Regions. The European Commission (EC) recently funded a number of pan-European eHealthprojects aimed at empowering European patients/citizens thus transforming the traditional patient/citizen role inthe management of their health (e.g., PALANTE, SUSTAIN, CARRE, HeartCycle, Empower). However, theheterogeneity of the healthcare systems, of the implemented services and of the target patients, the use of ad-hocdefinitions of the salient concepts and the development of small-size experiences have prevented thedissemination of “global” results and the development of cumulative knowledge. The main challenge has been thegeneration of large-scale evidence from heterogeneous small-size experiences.

Discussion: Three lessons have been collectively learnt during the development of the PALANTE project, whichinvolves 9 sites that have implemented different eHealth services for empowering different typologies of patients.These lessons have been refined progressively through project meetings, reviews with the EC Project Officer andReviewers. The paper illustrates the ten steps followed to develop the three lessons.The first lesson learnt is about how EC-funded projects should develop cumulative knowledge by avoidingself-crafted measures of outcome and by adopting literature-grounded definitions and scales. The second lessonlearnt is about how EC-funded projects should identify ambitious, cross-pilot policy and research questions thatallow pooling of data from across heterogeneous experiences even if a multi-centre study design was not agreedbefore. The third lesson learnt is about how EC-funded projects should open their collections of data and makethem freely-accessible to the scientific community shortly after the conclusion of the project in order to guaranteethe replicability of results and conclusions.

Summary: The three lessons might provide original elements for fuelling the ongoing debate about the capabilityof the EC to develop evidence-based policies by pooling evidence from heterogeneous, local experiences.

BackgroundPatient empowerment can be defined as the acquisitionof motivations and abilities that patients might use toimprove the participation in decision-making, and thusimprove their power in their relationship with profes-sionals [1]. Patient Empowerment has become a keypriority for policy-makers and professionals, who assumethat it will increase the sustainability of the present para-digms of care delivery [2]. Coherently, many initiatives

have been implemented in the last years for empoweringpatients in self-management [3] and shared decision-making [4]. Several providers, in particular, have pursuedeHealth solutions for patient empowerment, i.e., thedelivery or enhancement of health care services orhealth care information through the Internet and relatedtechnologies [5]. eHealth might contribute to radicallyrethink the patient-professional relationship because itovercomes spatial and temporal constraints, developshome-based solutions integrated within the National/Re-gional Healthcare Systems, integrates different function-alities, e.g., information, education, and communication[6–9]. Following this premise, the European Commission

* Correspondence: emanuele.lettieri@polimi.it1Department of Management, Economics and Industrial Engineering,Politecnico di Milano, 4/B Lambruschini Street, Milan 20156, ItalyFull list of author information is available at the end of the article

© 2015 Lettieri et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide alink to the Creative Commons license, and indicate if changes were made. The Creative Commons Public DomainDedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in thisarticle, unless otherwise stated.

Lettieri et al. BMC Health Services Research (2015) 15:309 DOI 10.1186/s12913-015-0983-0

(EC) recently funded a number of pan-European eHealthprojects aimed at empowering European patients/citi-zens to take a more active role in the management oftheir health (e.g., PALANTE, SUSTAIN, CARRE, Heart-Cycle, Empower).Despite such wealth of experiences, however, it is still

difficult to build a consistent story on the role thateHealth might play in empowering patients [10, 11],primarily because the concept of “patient empowerment”remains ambiguous [1], and has acquired over time mul-tiple meanings for professionals [12, 13]. Psychologists,hospital managers, nurses have promoted their specificviews on what empowerment is and how it might beachieved, limiting the possibility to reach a sharedunderstanding of this concept and agreement on itspractical development. The heterogeneity of patients andof the social contexts of European Countries have addedfurther complexity, leading to the implementation ofseveral small scale, country-specific eHealth-basedservices. Their harmonisation is now a policy priority toenhance the dissemination of local knowledge and expe-riences on patient empowerment and avoid the upcom-ing projects constantly ‘reinvent the wheel’.Large-scale, nation-wide eHealth-based services for

patient empowerment cannot be regarded the desired“silver-bullet” solutions, since previous attempts likeNPfit or the Whole System Demonstrator have failed torecognise and develop local diversities. Similarly, previ-ous EC-wide projects on patient empowerment stum-bled because of three main limitations. First, they haveadopted heterogeneous and self-crafted definitions ofpatient empowerment, which limited the possibility tocompare local experiences and learn from a commonpool of knowledge. Second, they have collected small-size and pilot-specific data on outcomes, costs, satisfac-tion or access, struggling to develop acceptable levels ofevidence. Third, scales employed to measure significantconstructs―such as patient empowerment, ease of use,perceived value―are self-crafted by each project team,more intended for local impact assessment exercises,than for the accumulation and dissemination of know-ledge in the scientific community. As a result, a lot ofpotentially valuable evidence about the role of eHealthsolutions to empower patients has struggled to informevidence-based policies from the EC. Considering theselimitation, we thus ask: in the context of EC projectinvolving different Countries and organisations, how canwe knowledge development and sharing be organised toallow the comparability of results and facilitate thetranslation of local experiences in EC-wide policies?To address this question, we report the experience of

PALANTE (PAtients Leading and mANaging theirhealThcare through Ehealth) project, which involves 9sites (namely, “pilots”), each of which has implemented a

different eHealth service for empowering differenttypologies of patients (cf. the next sections). This hetero-geneity has facilitated the cross-fertilisation amongdifferent eHealth experiences and generation of collect-ive knowledge, but has also the Consortium to many ofthe limitations described above.Reflecting on the experience of the PALANTE Con-

sortium, the paper develops three lessons regarding thepossibility to pool data from 9 diverse pilots and informthe EC about the role of eHealth solutions in empower-ing patients.This paper is organised as follows. The next section

briefly describes the methods implemented by thePALANTE Consortium to collect high-quality evidenceand the case study. Then, the main results are displayed.The last section summarises the main lessons of thisstudy that will be confirmed with the completion of theproject evaluation.

DiscussionStudy design approachThis section details the phases undertaken to design animpact assessment methodology that matches the needfor a robust research design with the contingencies ofthe 9 eHealth services implemented, in terms of norma-tive constraints, available resources, own goals, maturityof the eHealth service. This research is not based onpatients’ clinical data, but on discussions held during themeetings with Consortium partners, the EC ProjectOfficer and three EC reviewers (two academicians fromInformation Science and one senior manager from aleading, global IT company). All PALANTE pilotsadhered to the European and national legislations (e.g.,EC Directive 46/95 on data protection) protectingprivacy, security and ethical issues and obtained theauthorisation, when needed, from the related ethicalCommittees.The lessons learnt that will be illustrated later emerged

from a journey that the PALANTE Consortium under-took along ten phases in 42 months (lifetime of theproject).First, an urgent and relevant policy and research ques-

tion was agreed with the nine pilots; namely providingEC decision-makers with evidence on how eHealth con-tributes to patient empowerment. This clear, syntheticgoal was the product of several one-to-one and collect-ive teleconferences and was formalised in a collectivevis-à-vis meeting with all partners during the firstprogress meeting held in June 2012 in Milan (Italy).Second, the Consortium reviewed a list of etymologic-

ally similar concepts (e.g., patient empowerment, engage-ment, activation, involvement, participation, enablement)to avoid uncertainty among pilots about the mainendpoint of their activities and in further dissemination

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 2 of 12

activities. This choice has been informed by the results ofa systematic literature review carried out by Politecnico diMilano as academic partner of the PALANTE project.Methods and results of this review are available in anothermanuscript [1]. Based on the results of the systematic litera-ture, the PALANTE Consortium adopted the concept of“patient activation” and its Patient Activation Measure(PAM) scale [14] to operationalise the broad concept of“patient empowerment”. An activated patient knows howto manage her condition, maintain functioning and preventhealth declines; she has the skills and behavioural repertoireto manage her condition, collaborate with her healthproviders, maintain her health functioning and accessappropriate and high-quality care [14]. This concept wasfound more suitable to explain and represent the mainpurposes of the eHealth services implemented by the 9pilots.Third, the significant heterogeneity of the eHealth

services discouraged an immediate direct comparison(Table 1). This limitation was overcome by deconstructingthe services into a combination of three basic functional-ities (e.g., information, education, communication). It wasthen possible to compare the 9 implemented services interms of their basic functionalities. Additionally, datacollected at the pilot level could be pooled according to thespecific functionality. As result, the goal of the PALANTEproject has been reframed as providing evidence on if andhow eHealth-enabled information, education and commu-nication might contribute to a positive/negative variation inpatient activation. To our best knowledge, this is an originalsolution to data pooling in multi-centre studies on eHealth-based care delivery.Fourth, patients’ willingness to engage with the eHealth-

enabled functionalities was measured, consistently with thewell-established Technology Acceptance Model (TAM)[15], through patients’ perceived value and ease of use.Fifth, a sample of control variables was identified from

the literature to control potential confounding factorsand gather better explanations of results.Sixth, the theoretical framework linking PAM-related and

control variables to TAM-related outcomes was discussedwith all pilots in order to collect potential criticalities (e.g.,in the case of Norway the legal framework does not allowidentification of the respondent because of privacy con-cerns, and this inhibited a longitudinal research design).Seventh, a very detailed protocol for data collection

was produced and pilots were trained on its strict appli-cation to assure data pooling and significant evidencegeneration.Eighth, the research design was discussed during the

first review meeting with three reviewers appointed bythe EC and the Project Officer in April 2013 in Brussels.The methodology was approved, being regarded capable(i) to generate significant evidence through data pooling

from a multi-centre pan-European project, (ii) to betranslated to other projects about eHealth and patientactivation, and (iii) to be methodologically sound withinthe literature and academic standards.Ninth, three lessons learnt about how EC-funded projects

could behave to generate significant evidence were crystal-lised during the fifth progress meeting in September 2014in Istanbul. Project Officer and reviewers provided thePALANTE Consortium with the recommendation to pro-duce lessons which could be shared with other EC-fundedprojects. The three lessons were agreed by all partnerswithin the Consortium.Tenth, the lessons were shared in an open conference

organised by the PALANTE Consortium in June 2015 inBrussels. Feedbacks collected from attendees―academi-cians as well as practitioners―allowed the refinementand improvement of the lessons.

Case studyTable 1 shows the heterogeneity in terms of main goals,services offered, target patients, and size of the 9 pilotsinvolved within the PALANTE project.Such heterogeneity was intentional, since the Consortium

sought the opportunity to collect insights from very differ-ent experiences and provide the EC with diverse narrativeson how eHealth could be used to activate patients. Pilotswere then selected to account for different types of disease,levels of interaction between patients and healthcareprofessionals and types and innovativeness of eHealth-enabled solutions. The selection of polar-extremes cases[16] generated local expectation of pilot-specific impactassessment exercises, with the consequence of limiting anygeneralizability of results. This kind of expectation is com-mon in EC-funded project since pilots are more interestedin gathering an in-depth understanding of the pros andcons of their eHealth service rather than formalise moretheoretical lessons from the comparison of heterogeneousexperiences. This is particularly true when the conductionof a multi-centre study has not been agreed before. In thisregard, we discussed with pilots the opportunity of gather-ing more robust evidence by pooling their data through thedesign of an ad-hoc impact assessment framework.In order to counter such expectations, address pilots’

heterogeneity and guarantee the collection of high-quality, larger-size evidence, three challenges (and con-sequent lessons learnt) arose in the Consortium:

� The crystallisation of the main concept/phenomenon under observation (i.e., patientactivation) and of the main explanatory variables(i.e., ease of use and perceived value);

� The development of an ad-hoc research design forthe impact assessment that allows data pooling fromheterogeneous pilots;

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 3 of 12

Table 1 Pilots’ features. Shows the heterogeneity in terms of main goals, services offered, target patients, and size of the 9 pilotsinvolved within the PALANTE project

Pilot # Main services offered Target users

Pilot 1 – Andalusia(Spain)

Objective. To empower patients with diabetes and improve quality of life and diabetesmanagement through a new eHealth solution that comprises:

Diabetes 7,000 patients 6,500 Type2500 Type 1

▪ Patient access to Personal Health Record and management of personal healthinformation (medications, appointments, exams, allergies, clinical reports)

▪ Chronic disease management support services: Recording and tracking of patient’s ownmeasurements (blood sugar levels, weight, blood pressure) through tensiometers andglucometers

▪ Tailored education and lifestyle guidance through (virtual) games

▪ Secure messaging between patients and health care professionals

Pilot 2 – Lombardy(Italy)

Objective. to empower patients with chronic heart failure disease by involving them inand integrate care pathways where a specialist follows them.

Chronic Heart Failure Disease3,400patients

This is done through the SISS-DCPA tool available for producing Clinical Care Pathdocument at the hospital, publishing and versioning on the EHR to share it with GP andpatient. It comprises:

▪ Patient access to Personal Health Record and management of personal information:citizens can see his/her clinical documents and upload documents

▪ Tailored education and lifestyle guidance (guidelines, recommendations on lifestyle toadopt, diet, physical activities to do)

▪ Timeline view providing bird’s eye view of the HER

▪ OTP access (OTP sent via Mob. Phone) & Access with mobile devices

▪ Better integration with new services of booking and payment

Pilot 3 – Turkey Objective. To empower patients that suffer from ankylosing spondylitis, by follow themthrough a service that permits to have:

Ankylosing Spondylitis2,000 patients

▪ Tailored education and lifestyle guidance through Educational exercise videos ondisease and treatments

▪ Messaging service between patient and doctor

▪ Diary service which allows patients to record their daily or weekly health status andinform the doctors if necessary (Patient Diary)

▪ Remote follow-up

▪ Chronic disease management

▪ Remote consultation

▪ Medication and non-pharmacological treatment follow-up and medication andnon-pharmacological treatment benefit analysis

▪ Reminders

Pilot 4 – Norway Objective. To empower patients with diabetes and hospitalised patients for any disease byproviding the following eHealth services:

Diabetes1,550 patientsDischargeNotes1,3 million of patients

▪ Electronic discharge notes in the local electronic health record

▪ Modules tailored for chronically ill patients (diabetes)

▪ Consultation form and self-reporting tools to support self-management

▪ A guidance tool that helps visualizing factors (food, insulin and physical activity) thatinfluence blood sugar levels in a 24 hour clock, also used as a communicational tool

▪ Recording and tracking of patient’s own measurements (blood sugar levels, weight,blood pressure, HbA1c)

▪ Patient access to Personal Health Record

▪ Secure messaging between patients and health care professionals

▪ Integrated with MinJournal (Patient Portal)

Pilot 5 – Styria(Austria)

Objective. To empower patients on their condition of X-ray exposure for estimating therisk level in making a new exam through an eHealth services that comprises:

X-ray exposure1,183 patients

▪ Patient access to his/her personal eX-ray-Record that reports personal achieved level ofX-ray exposure

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 4 of 12

Table 1 Pilots’ features. Shows the heterogeneity in terms of main goals, services offered, target patients, and size of the 9 pilotsinvolved within the PALANTE project (Continued)

▪ Storing of individual information about radiation exposure data for all examinations intothe existing Hospital Information System (HIS)

▪ Develop an informative and comprehensible presentation for the radiation exposuredata

▪ Provide access for patients to their personal eX-ray-Record via a patient-portal on theinternet (and for health professionals via the HIS)

Pilot 6 – Czech Rep. Objective. To empower parents in taking care of their children through a eHealth servicethat permits to:

Children health management3,352patients

▪ Scheduling appointments

▪ Having access to Personal Health Record

▪ Monitoring individual parameters related to children growth

▪ Accessing to the Vaccinations program

▪ Monitoring the Preventive check-outs calendar

▪ Monitoring the child growth (height/weight)

▪ Sending alert reminders via SMS

Pilot 7 – BasqueCountry (Spain)

Objective. To empower patients with COPD by including them in a telemedicine programwith educational module to follow. The eHealth service includes:

Asthma & COPD150 patients

▪ Integration of all the data and information that a patient generates through the healthservices (health habits, treatment, symptoms, etc.)

▪ Patient access to Personal Health Record

▪ Tailored education and lifestyle guidance

▪ Telecare, Monitoring and Chronic disease management support services

▪ Real time health information uploading

▪ Secure text & voice messaging and videoconference with the healthcare team andalerts

▪ Telerehabilitation (physical exercises)

▪ Multi-platform: TV/Kinect & mobile

Pilot 8 – France Objective. To empower citizens providing the access to the personal information throughthe national health portal where it is possible:

All Citizens/ Patients

▪ To view data and to be informed about new documents

▪ To correspond securely with a healthcare professional, after his consent

▪ To enter information: self-monitoring results, their wishes regarding arrangements at theend of their lives, etc.

▪ To download all or some of contents of the personal health folder

Pilot 9 Denmark Objective. To empower citizens providing the access to the personal information throughthe national Advanced eHealth portal with the following services:

All Citizens/ Patients

▪ Patient management of personal health information

▪ Patient access to Personal Health Record

▪ Chronic disease management support services

▪ Lifestyle guidance

▪ National web-solution for citizens/patients and health professionals, integrating datafrom app. 100 sources

▪ Online EHRs from hospitals, cross-sectorial personal electronic medicine profile, accessto lab/test results

▪ List of contacts with public hospitals and publically subsidised contacts in PHC

▪ Organ Donor Registration and Living Will

▪ Patient’s audit of any access to his/her data

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 5 of 12

� The delivery of original, valuable, timely and robustanalyses of collected data within a very constrainedtime span.

Finally, all these challenges had to be faced throughthe implementation of generally accepted academic stan-dards in order to facilitate both the dissemination ofresults and the legitimacy of EC’s policy-making [17].Henceforth, we detail the three main challenges and how

they have been tackled by the PALANTE Consortium.

Crystallisation of the main concept/phenomenon andexplanatory variablesIn the first project meeting, it emerged clearly that the con-cept of “patient empowerment” had different meanings forthe people sitting at the table. This ambiguity limited thepossibility to “speak the same language” both inside andoutside the project (e.g., with the SUSTAINS Consortiumthat was working on patient empowerment through theaccess to and use of Electronic Medical Records), inparticular with healthcare professionals. The same concernemerged with respect to the explanatory variables. Whileall pilots agreed that concepts such as user-friendliness ofthe eHealth service, patient’s perception that pros outweighcons, patient’s self-efficacy with the eHealth service wouldhave been salient in predicting the success of any eHealth-enabled care delivery, there was not an alignment on themetrics for their measurement and analysis. Each pilot hadinstead an individual list of metrics employed in past EC-funded or publicly funded projects, with unclear scientificfoundations.The Consortium addressed this issue reviewing past

contributions on eHealth-enabled solutions for caredelivery and on patient empowerment. This allowedfurthering the discussion with the pilots and positioningthe expected results according to past research [18].On the one hand, past studies agreed that in order to

take advantage of an eHealth-enabled solution for caredelivery, patients must be willing and able to interactwith the new system (e.g., personal health record andsecure messaging [19], eHealth service [20]). This argu-ment underlines the importance of the patients’ percep-tion of the system that has to be used. This perception isinfluenced by two main elements, i.e., the patient shouldexpect to reach some benefits thanks to the use of thatsystem; and to use the system without difficulty. Bothelements can be estimated through the TAM scale [15],which was developed to predict and evaluate users’acceptance of different information technologies. Thescale includes the construct of usefulness, which repre-sents “the degree to which a person believes that using aparticular system would enhance his or her job perform-ance” [21]; and ease of use, which represents “the degreeto which a person believes that using a particular system

would be free of effort” [21]. The literature alsosuggested relevant control variables to be taken intoaccount for possible confounding factors, i.e., gender,age, education level, ICT experience, experience withdisease and trust in regional/national healthcare system[22–24].On the other hand, the literature did not provide the

Consortium with a straightforward solution to oper-ationalise patient empowerment, since this concept isstill under discussion and has important overlaps withneighbour concepts―such as activation, involvementengagement, enablement, participation. By carrying outa systematic literature review [1], the Consortiumaddressed this issue selecting the concept of “patientactivation” [14] to operationalise “patient empowerment”and to position the expected results within the ongoinginternational debate.

Data pooling from heterogeneous pilotsA second core issue related to the capability to pool datafrom very different pilots. As shown in Table 1, pilotswere implementing significantly different eHealth-enabledsolutions, which inhibited a direct comparability of results.During the first progress meeting in Milan (June 2012),pilots discussed the pros and cons of designing a multi-centre study in such a context. Pilots claimed that theirheterogeneity required pilot-specific impact assessmentexercises in order to take into account the contingenciesof each pilot. Furthermore, their previous experiences withEC-funded projects suggested that small-sise, pilot-specific impact assessment exercises were more efficientand valuable for local dissemination. On the other hand,the literature did not offer examples of how to cope withthis heterogeneity since past studies compared and pooleddata from similar eHealth services (e.g., telemonitoring[25], Electronic Medical Records [26, 27]).The Consortium regarded these arguments as an op-

portunity to develop an innovative approach that couldbe valuable, for instance, to all projects that receivepublic funds and share the same heterogeneity, but stillhas to gather evidence that might matter at the EC-level.The most practical solution for a common assessment isdeconstructing each eHealth-based service according toits basic functionalities since all services seek to activatepatients by informing, educating and communicatingwith them. Specifically, information refers to all kind ofinformation shared through the eHealth service, such asalerts and notifications about treatments, reminders onappointments, opportunities to book medical visits or otherhealth services, exchanges of personal health informationor documents. Education refers instead to all aspects thataffect the education of patients, who learn by using devicesand educational materials (video, suggestions, documents,care plan), learn the meaning of their health results, how to

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 6 of 12

increase their skills and to manage their own diseases on adaily basis. Finally, communication refers to all kind ofcommunication between healthcare providers and patients(e.g., texts, video or e-mail, calls).In this view, the eHealth solutions implemented by the

pilots have been conceptualised as an organised, coher-ent bundle of these three functionalities (Table 2).Observing the role that eHealth functionalities might play

in enabling patient activation, the PALANTE Consortiumachieved two benefits. On the one hand, data collectedfrom heterogeneous pilots could be pooled together into acommon dataset. This increased the capability to generatereliable evidence that might be of value for the EC toground its policy on the promotion of eHealth solutions forpatient activation. This solution was of paramount rele-vance because it showed that more fine-grained observationof functionalities can overcome the shortcomings of thetraditional pilot-constrained impact assessment. Further-more, an analysis based on functionalities will provide theopportunity to disentangle their impact on patient activa-tion and to understand which functionalities―or combina-tions of them―are more likely to positively activatepatients. This knowledge can provide researchers, policy-makers, professionals, suppliers with more detailed guide-lines for the development of cost-effective eHealth solutionsfor patient activation.

Delivery of the analyses within a very constrained timespanThe PALANTE project has a lifespan of 42 months.Within this timeframe, only the last six months could bededicated to data analysis. This situation is very likely tohappen in many EC-funded projects since the develop-ment/implementation of the eHealth-enabled solutionsas well as data collection require significant portions ofthe whole project lifespan. As result, at least two short-comings arise with respect to the exploitation of thehuge mass of data collected. On the one hand, thePALANTE Consortium will have to focus its attentionon the analyses agreed with the EC at the beginning ofthe project, when it is not possible to anticipate all pecu-liarities, issues and variables that could play a role withinthe pilots. The risk is not to include possible relevantanalyses which are not formally signed with the EC. Forexample, while analyses could be carried out to verify if thethree eHealth-enabled functionalities affect patient activa-tion with respect to the set of pooled data, questions aroseabout the possibility to perform pilot-specific analyses. Onthe other hand, collected data could be investigated only bypartners of the PALANTE Consortium, losing the oppor-tunity to collect points of view of other experts that couldidentify and suggest new way to explore data. This has twospecific main limitations. First, the capability to read thesedata from different, original angles was affected. If other

researchers, professionals, policy-makers can access thesedata they could probably bring new research and policyquestions, enabling the generation of additional knowledge.Second, opening the dataset to others significant stake-holders would guarantee the reliability of results because itwould allow the replicability of all analyses. This wouldreinforce the strength of the final policy recommendations,so that the EC would be justified in leveraging on theseresults and orienting future research and projects. Thegrowing concerns about the quality of the evidence that un-derpins healthcare decision-making [28–32] obliges morethan ever EC-funded projects―being publicly funded bytaxpayers―to consider the possibility to open their datasetsto the scientific community in order to guarantee thereliability of their results. In this regard, the EC has alreadymoved the first steps in this direction, giving the oppor-tunity to winners of H2020 grants to apply to the opendata protocol (http://ec.europa.eu/digital-agenda/en/open-data-0). This initiative needs to be treated with attentionin order to manage sensitive aspects like privacy thatinvolves the laws of different Countries. Making data setspublicly available is a delicate matter, particularly whenthe data set is sourced from a wide-range of memberstates. Legal concerns regarding confidentiality of patientdata and permission requirements need to be well exam-ined within the state legislation for each member stateinvolved. Additionally, rules on using the data for publica-tion and the process of gaining consent from the data’ssource also need to be taken into consideration; particu-larly, when the data yields a negative result which maymake its use politically sensitive.Such concerns are being taken into consideration by the

PALANTE Consortium as it required the definition andsubmission of informed consent documents to patients atthe enrolment in the project, and it is still discussingmethods for maintaining and making available the datasetfollowing the conclusion of the project.A chief concern of the Consortium is considering the op-

portunity for further analyses post-PALANTE lifetime,which would generate additional value for the EC as well asfor all healthcare eco-system. A first step towards this goalis sharing data within the Consortium to increase the trans-parency of methods and results with partners and improvethe explanation of the results. This internal sharing andopening of the dataset is also an opportunity to learn andunderstand potential issues that can arise and that requireto be treated increasing the knowledge needed for man-aging future external access to the dataset.

ResultsAt the moment the PALANTE project is in progress.The administration of the baseline survey has been com-pleted and the collected data are under analysis. A sec-ond survey is currently in the field to gather data on

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 7 of 12

Table 2 Deconstruction of pilots’ eHealth service in its main functionalities. Reports the organization of the eHealth solutionsimplemented by the 9 pilots according with the three functionalities identified: information, education and communication

Pilot # eHealth functionalities

Information Education Communication

Pilot 1 – Andalusia (Spain) ● Patient access to Personal HealthRecord and management ofpersonal health information(medications, appointments,exams, allergies, clinical reports)

● Tailored education andlifestyle guidance through(virtual) games: Diabetesself-management Education(diet counseling, dailyactivities, measurements,games)

● Communicationwith their healthcareteam: Secure messagingbetween patientsand health careprofessionals

Chronic disease managementsupport services: Recording andtracking of patient’s ownmeasurements (blood sugar levels,weight, blood pressure) throughtensiometers and glucometers

Pilot 2 – Lombardy (Italy) ● Patient access to Personal HealthRecord and management ofpersonal information: citizens cansee his/her clinical documentsand upload documents

● Tailored education andlifestyle guidance (guidelines,recommendations onlifestyle to adopt, diet,physical activities to do)

● Access to Integrated Care Pathways(for healthcare professionals andpatients) and data entry andupdates (only for healthcareprofessionals)

● Timeline view providing bird’seye view of the HER

● Better integration with newservices of booking andpayment

● Patient summary andstructuration

Pilot 3 – Turkey ● Access to Personal Health Recordand Electronic Health Record

● Tailored education and lifestyleguidance through Educationalexercise videos on disease andtreatment

● Messaging servicebetween patientand doctor

● Chronic disease managementsupport service

● Diary service which allows patientsto record their daily or weeklyhealth status and inform thedoctors if necessary (Patient diary)

● Remote follow-up

● Remote consultation

● Medication and non-pharmacologicaltreatment follow-up and Medicationand non-pharmacological treatmentbenefit analysis

● Reminders

Pilot 4 – Norway ● Electronic discharge notes in thelocal electronic health record

● A guidance tool thathelps visualizing factors(food, insulin and physicalactivity) that influenceblood sugar levels in a24 hour clock

● Consultation formand self-reportingtools to supportself-managementInternet-basedpatient/ providercommunicationand interaction

● Modules tailored for chronically illpatients (diabetes)

Secure messagingbetween patientsand health careprofessionals● Recording and tracking of patient’s

own measurements (blood sugarlevels, weight, blood pressure,HbA1c)

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 8 of 12

Table 2 Deconstruction of pilots’ eHealth service in its main functionalities. Reports the organization of the eHealth solutionsimplemented by the 9 pilots according with the three functionalities identified: information, education and communication(Continued)

Patient access to Personal HealthRecord

Pilot 5 – Styria (Austria) ● Patient access to his/her personalX-ray-Record that reports X-rayexposure

● Storing of individual informationabout radiation exposure datafor all examinations into theexisting hospital informationsystem (HIS)

● Develop an informative andcomprehensible presentationfor the radiation exposure data

Provide access for patients totheir personal eXrayRecord via apatient-portal on the internet(and for health professionalsvia the HIS)

Pilot 6 – Czech Republic ● Scheduling appointments ● Sending alert reminders via SMS

● Patient access to PersonalHealth Record

● Monitoring individual parametersrelated to children growth

● Accessing to the Vaccinationsprogram

● Monitoring the Preventivecheck-outs calendar

● Monitoring the child growth(height/weight)

Pilot 7 – Basque Country (Spain) ● Integration of all the data andinformation that a patientgenerates through the healthservices (health habits, treatment,symptoms…)

● Tailored educationand lifestyle guidance

● Secure text & voice messagingand videoconferencewith the healthcare teamand alerts:Call/videoconferencing/audio and text messaging;the patient can be in contactwith other people in charge ofhis/her care (family, careassistants, etc.), Sharing/discussing with otherprofessionals about certainsymptoms, treatments, etc.

● Patient access to PersonalHealth Record

Telerehabilitation(physical exercises)

Pilot 8 – France ● Patient access to personal heathfolder and download of personalcontents

● Tailored educationand lifestyle guidance

● To correspond securely with ahealthcare professional, afterhis consent

● To view data and to be informedabout new documents

● Telerehabilitation(physical exercises)

● Data entry: self-monitoringresults, their wishes regardingarrangements at the end oftheir lives, etc.

● To download all or some ofcontents of the personal healthfolder

Pilot 9 – Denmark ● Patient management of personalhealth information

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 9 of 12

patients’ experience after six months of service use.While it is not yet known whether the eHealth function-alities positively affect patient activation, the PALANTEConsortium argues that the solutions implemented forits assessment might per se offer an original, valuableand viable way to overcome the limitations of the trad-itional generation of evidence in EC projects and, moregenerally to publicly-funded inter-national projects. TheEC needs to ground its policy-making on significant―interms of size and robustness―sources of evidence thatcome from both local and pan-European projects.Furthermore, European Regions look to such researchprojects for reliable and pertinent evidence when makingdevelopment decisions for their healthcare systems. Tosupport this, we have formalised three lessons that mightovercome the shortcomings of small-scale, pilot-specificevaluations. These lessons have policy implications inter-esting for the EC and offer new insights to further thedebate about the implementation of evidence-based pol-icies in healthcare.The first lesson learnt is the “relevance of developing

cumulative knowledge”. Any EC-funded project shouldadopt clear terminologies and scales, following the on-going scientific debates in the literature. In PALANTEproject, for instance, patient empowerment was replacedby patient activation as result of a systematic literaturereview. Investments made to stimulate patients taking aproactive role in their health require the development of aunique, consistent story that could cumulate evidencefrom various projects. This would be impossible if differ-ent terminologies, definitions and measurement scales areadopted. In this regard, these concepts should be takenfrom past scientific literature in order to guarantee the ac-cumulation of data and knowledge and the creation of sig-nificant evidence for the EC. The PALANTE Consortiumadopted a consolidated scale―i.e., the PAM scale―that

has already been used across a wide variety of past studies.This means that evidence collected by PALANTE will addnew insights to what is already known. Whether the appli-cation of the same scale to different eHealth solutions andpatient cohorts hamper construct validity will be evidentin the project’s final results.The second lesson learnt addresses the “relevance of

developing large-size knowledge”. We suggest that any EC-(or publicly) funded project involving multiple pilotsshould implement a strategy for a multi-centre researchdesign to pooling local data into a common repository.This strategy is especially critical when pilots are commit-ted to small-size, innovative eHealth solutions ― and sug-gest the involvement of coordinating actors/organisationsthat could mediate pilots’ interests. The informative powerof pan-European projects can be especially increasedthrough the careful identification of cross-pilot policyquestions (e.g., do eHealth-enabled information/educa-tion/communication functionalities positively affect pa-tient activation?) and their subsequent formalisation into aliterature-based, cross-pilot questionnaire data collection.The third lesson learnt is the “relevance of developing

open, freely-accessible sources of knowledge”. We arguethat any EC-funded project should make its full impactassessment dataset available to the scientific communityshortly after the conclusion of the project. This shouldconstitute a formal output of any project. The pros ofthis policy-choice are evident. Opening the dataset tothe scientific community and any key stakeholders (e.g.,healthcare professionals, regional officials, hospital man-agers, patient associations) means they could replicatethe main analyses conducted within the project and toimplement other studies to the same dataset. The consrefer to the protection of confidential informationdefended by national and/or European laws; to the cap-ability of pilots and other partners to disseminate the

Table 2 Deconstruction of pilots’ eHealth service in its main functionalities. Reports the organization of the eHealth solutionsimplemented by the 9 pilots according with the three functionalities identified: information, education and communication(Continued)

● Patient access to Personal Health Record

● Chronic disease managementsupport services

● Online EHRs from hospitals, cross-sectorial personal electronicmedicine profile, access tolab/test results

● List of contacts with publichospitals and publicallysubsidized contacts in PHC

● Organ Donor Registrationand Living Will

● Patient’s audit of any accessto his/her data

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 10 of 12

project findings through publications in high-quality jour-nals or in relevant conferences/workshops; to the possibilitythat non-project participants can profit academicallywithout bearing the huge effort of data collection, and tothe potential misuse of data. Finally, pilots have a vestedinterested in being first on publication based on their data.These issues will be addressed as PALANTE discusses howto provide access to and maintain data sets once the projecthas concluded and may provide a map for guide futureprojects when making their data publicly available.Concluding, the project teams have the responsibility to

design methodologies for their impact assessment exer-cises that are able to meet the recommendations citedabove. The involvement within the project Consortium ofpartners with significant expertise on Health TechnologyAssessment (HTA) and Evidence-Based Policy-makingcan help improve the quality of research designs and ofgathered evidence.

SummaryThese lessons might be valuable for different stakeholdersand could be applied through a number of strategieswhich this manuscript seeks to facilitate.First, the EC officers might lever on the PALANTE

experience to promote the adoption of more sophisticatedmethodologies for the impact assessment exercise in otherfunded projects with the aim of collecting a higher level ofevidence. Upstream, the EC officers might change theirranking system for future proposals by allocating higherscores to proposals that take into account the generationof high-quality evidence in terms of size and coherence tothe literature. Downstream, the EC officers might orientfuture research toward implementing facts-based healthstrategies, rationalizing investments and sifting out un-profitable research fields/directions/initiatives.Second, scientific/project coordinators of EC-funded

projects―as well as professionals which are involved inthese projects―might benefit from these lessons becausethey exemplify real-life situations related to coping withthe high complexity of an impact assessment exercise onheterogeneous eHealth services in different Countries.Third, developers or suppliers of eHealth services might

benefit from the PALANTE story in terms of renewedwillingness and increased ability to gather high-qualityevidence about the capability of their services to actuallyempower patients. In an European context characterised byshrinking financial and human resources, policy-makersmust prioritise which small-size eHealth services shouldgrow to a larger scale and become institutionalised.Fourth, scholars of Health Policy, Healthcare Manage-

ment and Information Science might benefit from thisstudy in terms of new insights to further the ongoingdebate about the development and implementation ofpopulation-based trials. The significant costs―in terms

of money and time―of traditional randomised clinicaltrials obliges the exploration of other pathways to gatherhigh-quality evidence to support policy-making. In thisregard, the development of embedded solutions withineHealth services to collect data could offer an interestingand methodologically sound alternative.

AbbreviationEC: European Commission; PALANTE: Patients Leading and mANaging theirhealThcare through Ehealth; HTA: Health Technology Assessment;PAM: Patient Activation Measure; TAM: Technology Assessment Model;ICT: Information and Communication Technology.

Competing interestsThe authors declare that they have no competing interests.

Authors’ contributionsEL conceived the paper. EL, LPF, GR, PB co-wrote the first draft which wasdiscussed with all co-authors CM, JV, RH, JLL, AC. Further drafts have beendone in close collaboration among all authors. JV did the proofreading. Allauthors provided critical intellectual input, revised and approved the finalversion.

AcknowledgmentThe PALANTE Project has received funding from the European Union’s ICTPolicy Support Programme as part of the Competitiveness and InnovationFramework Programme under GA n° 297260. This publication only reflectsthe author’s views. The European Union is not liable for any use that mightbe made of the information contained in this manuscript.

Author details1Department of Management, Economics and Industrial Engineering,Politecnico di Milano, 4/B Lambruschini Street, Milan 20156, Italy. 2WarwickBusiness, University of Warwick, CV4 7AL Coventry, UK. 3Empirica Gesellschaftfür Kommunikations- und Technologieforschung mbH, 2 Oxfordstr, BonnD-53111, Germany. 4Fundación Pública Andaluza Progreso y Salud (FPS), 13Avda. Américo Vespucio, Sevilla 41071, Spain. 5Andalusian Health Service/Regional Ministry of Equality, Health and Social Policies of Andalusia, 14 Av.Hytasa, Sevilla 41071, Spain.

Received: 30 January 2015 Accepted: 30 July 2015

References1. Fumagalli LP, Radaelli G, Lettieri E, Bertele’ P, Masella C. Patient

empowerment and its neighbours: clarifying theboundaries and theirmutual relationships. Health Policy. 2015;119:384–94.

2. Colombo C, Moja L, Gonzalez-Lorenzo M, Liberati A, Mosconi P. Patientempowerment as a componentof health system reforms: rights, benefitsand vested interests. Intern Emerg Med. 2012;7:183–7.

3. Chatzimarkakis J. Why patients should Be more empowered: a Europeanperspective on lessons learned in the management of diabetes. J DiabetesSci Technol. 2010;4:1570–3.

4. Scambler S, Newton P, Sinclair AJ, Asimakopoulou K. Barriers andopportunities of empowerment as applied in diabetes settings: a focus onhealth care professionals’ experiences. Diabetes Res Clin Pract.2012;97:e18–22.

5. Eysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20.6. Lemire M, Sicotte C, Paré G. Internet use and the logics of personal

empowerment in health. Health Policy. 2008;88(1):130–40.7. Dedding C, van Doorn R, Winkler L, Reis R. How will e-health affect patient

participation in the clinic? A review of e-health studies and the currentevidence for changes in the relationship between medical professionalsand patients. Soc Sci Med. 2011;72:49–53.

8. Bos L. Patient Empowerment: A Two Way Road. In: Wickramasinghe N, Bali R,Suomi R, Kirn S, editors. Critical Issues for the Development of SustainableE-health Solutions. Healthcare Delivery in the Information Age. Springer US;2012.

9. Townsend A, Adam P, Li LC, McDonald M, Backman CL. Exploring eHealthethics and multi-morbidity: protocol for an interview and focus group study

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 11 of 12

of patient and health care provider views and experiences of using digitalmedia for health purposes. JMIR Res Protoc. 2013;2(2), e38.

10. Meijer WJ, Ragetlie P. Empowering the patient with ICT-tools: the unfulfilledpromise. Stud Health Technol Inform. 2007;127:199–218.

11. Lluch M. Empowering Patients through ICT, Organisational Impact onHealthcare Systems in England and Scotland. Commun Strateg.2011;83:37–58.

12. McAllister M, Dunn G, Payne K, Davies L, Todd C. Patient empowerment:The need to consider it as a measurable patient-reported outcome forchronic conditions. BMC Health Serv Res. 2012;12:1570–3.

13. Small N, Bower P, Chew-Graham CA, Whally D, Protheroe J. Patientempowerment in long-term conditions: development and preliminarytesting of a new measure. BMC Health Serv Res. 2013;13:63.

14. Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the PatientActivation Measure (PAM): conceptualizing and measuring activation inpatients and consumers. Health Serv Res. 2004;39:1005–26.

15. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computertechnology: a comparison of two theoretical models. Manage Sci.1989;35:982–1003.

16. Yin RK. Case study research: design and methods. 5th ed. California: SagePublication Inc.; 2013.

17. Oliver K, Innvar S, Lorenc T, Woodman J, Thomas J. Systematic review ofbarriers to and facilitators of the use of evidence by policymakers. BMCHealth Serv Res. 2014;14:2.

18. Peňa JLM, Gil OM. E-Health for patient empowerment in Europe. Madrid (E):Informes, estudios e investigacìon 2007 Ministero de Sanidad Y Consumo,Instituto de Salud Carlos III; 2007. Deposito Legal: M-48076-2007.

19. Agarwal R, Anderson C, Zarate J, Ward C. If we offer it, will they accept?Factors affecting patient use intentions of personal health records andsecure messaging. J Med Internet Res. 2013;15(2), e43.

20. Hage E, Roo JP, van Offenbeek MAG, Boonstra A. Implementation factorsand their effect on e-Health service adoption in rural communities: asystematic literature review. BMC Health Servs Res. 2013;13:19.

21. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of informationtechnology: toward a unified view. MIS Quarterly. 2003;27:425–78.

22. Gallivan MJ, Spitler VK, Koufaris M. Does information technology trainingreally matter? A social information processing analysis of coworkers’influence on IT usage in the workplace. J Manage Inform Syst.2003;22(1):153–92.

23. Ankem K. Factors influencing information needs among cancer patients:a meta-analysis. Libr Inform Sci Res. 2006;28(1):7–23.

24. Gefen D. Reflections on the dimensions of trust and trustworthiness amongonline consumers. ACM SiGMiS Database.2002;33(3):38–53.

25. Cruz J, Brooks D, Marques A. Home telemonitoring effectiveness in COPD:a systematic review. Int J Clin Pract. 2014;68(3):369–78.

26. Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use andimpacts of electronic health records: a review of the research literature.Int J Med Inform. 2008;77(5):291–304.

27. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medicalrecords by physicians from systematic review to taxonomy andinterventions. BMC Health Serv Res. 2010;10(1):231.

28. Sackett DL, Rosenberg W, Gray JA, Haynes RB, Richardson WS. Evidencebased medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2.

29. Brownson RC, Baker EA, Leet TL, Gillespie KN, True WR. Evidence-basedpublic health. 2nd ed. New York: Oxford University Press; 2010.

30. Ferlie E, Crilly T, Jashapara A, Peckham A. Knowledge mobilisation inhealthcare: a critical review of health sector and generic managementliterature. Soc Sci Med. 2012;74(8):1297–304.

31. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translationof research findings. Implement Sci. 2012;7(1):50.

32. Kitson A, Powell K, Hoon E, Newbury J, Wilson A, Beilby J. Knowledgetranslation within a population health study: how do you do it? ImplementSci. 2013;8:54.

Submit your next manuscript to BioMed Centraland take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at www.biomedcentral.com/submit

Lettieri et al. BMC Health Services Research (2015) 15:309 Page 12 of 12