J A C C : C A R D I O V A S C U L A R I M A G I N G V O L . 6 , N O . 1 1 , 2 0 1 3

ª 2 0 1 3 B Y T H E A M E R I C A N C O L L E G E O F C A R D I O L O G Y F O U N D A T I O N I S S N 1 9 3 6 - 8 7 8 X / $ 3 6 . 0 0

P U B L I S H E D B Y E L S E V I E R I N C . h t t p : / / d x . d o i . o r g / 1 0 . 1 0 1 6 / j . j c m g . 2 0 1 3 . 0 9 . 0 0 3

Intelligent Platforms for Disease Assessment

Novel Approaches in Functional Echocardiography

Partho P. Sengupta, MD, DM

New York, New York

Accelerating trends in the dynamic digital era (from 2004 onward) has resulted in the emergence of

novel parametric imaging tools that allow easy and accurate extraction of quantitative information

from cardiac images. This review principally attempts to heighten the awareness of newer emerging

paradigms that may advance acquisition, visualization and interpretation of the large functional data

sets obtained during cardiac ultrasound imaging. Incorporation of innovative cognitive software that

allow advanced pattern recognition and disease forecasting will likely transform the human-machine

interface and interpretation process to achieve a more efficient and effective work environment.

Novel technologies for automation and big data analytics that are already active in other fields need

to be rapidly adapted to the health care environment with new academic-industry collaborations to

enrich and accelerate the delivery of newer decision making tools for enhancing patient

care. (J Am Coll Cardiol Img 2013;6:1206–11) ª 2013 by the American College of Cardiology

Foundation

Echocardiography remains the most versatilecardiac imaging tool in clinical practice, of-fering an ever-increasing array of measure-ments. Improved computer capabilities overthe yearsdas predicted by Moore’s lawdhaveenhanced data extraction and visualization.New opportunities are emerging with interac-tive web-based networks providing easy accessto massive datasets in cardiac imaging. Inaddition to the rise of big data, migration ofhardware functionality to software has madepossible miniaturization of clinical ultrasounddevices. These emerging technologies have thepotential to make clinical imaging smarter andmore cost effective. This review provides abroad perspective on projected developmentsin cardiac ultrasound, highlighting new path-ways to acquire and analyze large datasets us-ing cognitive tools that are particularly relevantfor cardiac function assessment.

From the Zena and Michael A. Wiener Cardiovascular Institute, Ica

New York. Dr. Sengupta has received research funding from Forest La

Intelligence Ltd.

Manuscript received September 4, 2013; revised manuscript received S

The Big Data Revolution in Cardiac Ultrasound

Big data computing is perhaps the most sig-nificant development in the digital world overthe last decade. It has been estimated thatsomewhere between 2003 and 2004, theamount of data created by the digital worldaccelerated exponentially, surpassing the totalamount of data created in the previous 40,000years of human civilization (1). A similargrowth rate can also be seen in the field ofechocardiography, where the number of publi-cations has increased exponentially over the pastfew decades (Fig. 1A). And in addition to theincrease in the number of publications, closerexamination reveals a shift in what researchersare studying as well (Fig. 1B). In the 1980s,publications about Doppler imaging experi-enced the greatest growth in terms of publica-tion volume. However, by the beginning of the

hn Mount Sinai School of Medicine, New York,

boratories; and is on the advisory board for Medical

eptember 9, 2013, accepted September 12, 2013.

Doppler

3D

Parametric

Miscellaneous

No

. o

f P

ub

lica

tio

ns

Year

1970 1980 1990 2000 2010100

1,000

10,000

100,000

Fra

ctio

n o

f To

tal

Pu

bli

cati

on

s (%

)Year

1970 1980 1990 2000 20100

2

4

6

8

10

12

14A B

Figure 1. Growth of Annual Publications in Cardiac Ultrasound

The number of publications in cardiac ultrasound increased exponentially over the last 4 decades (A). The publication trends in differentdisciplines of cardiac ultrasound are shown in the bubble chart (B). The y-axis shows the yearly fraction of manuscripts published in agiven discipline of cardiac ultrasound (the total number of manuscripts cumulated between 1972 to 2012 for the given discipline was usedas the denominator for calculating the annual fraction and expressed as a percentage). The annual publication data and analysis wascompiled from Scopus (Elsevier Ltd.). The size of the bubble represents the absolute number of annual publications. 3D ¼ 3-dimensionalechocardiography.

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 6 , N O . 1 1 , 2 0 1 3 Sengupta

N O V E M B E R 2 0 1 3 : 1 2 0 6 – 1 1 Novel Approaches in Functional Echocardiography

1207

21st century, the number of publications on para-metric imaging had increased, surpassing thenumber of publications from all other echocardi-ography disciplines; coincidently, this occurredsometime between 2003 and 2004.

The number of quantitative parameters that canbe measured during an echocardiogram continues togrow. For example, current guidelines haveemphasized the routine reporting of at least 12measures of cardiac chamber quantification and theuse of 8 variables for grading the severity of dia-stolic dysfunction (2,3). Moreover, with the recentgrowth in parametric imaging techniques such as2-dimensional (2D) speckle tracking echocardiog-raphy, several new quantitative variables have beentested in clinical trials and are awaiting furtherstandardization. It remains unclear, however, howreasonably these variables may be incorporated intoa busy clinical practice.

Notwithstanding an increase in the number ofvariables, the new parameters offer a better under-standing of the relationship between the structureand function of the myocardium. For example,speckles that are tracked to measure myocardialdeformation are not just random acoustical inter-ference patterns, but rather they have their genesisin the underlying counter-directional helical muscle

geometry of the left ventricle (LV) (4). Knowingthis has enabled researchers to comprehend thepeculiarities of heart motion during the phases ofthe cardiac cycle as well as the contributions of3-directional deformation toward maintainingglobal LV pump function. As a consequence, weknow that a disease state that affects LV functionimpacts deformation in all 3 directions. Forexample, multidirectional deformation in heartfailure reveals the presence of unique interactionsthat differentiates the phenotypic presentations (5).Nontransmural disease is associated with reducedlongitudinal mechanics, whereas the mechanics inother directions are relatively preserved. On theother hand, a reduction of LV deformation in all 3directions is associated with transmural disease anda reduction of the LV ejection fraction. Thus, it isoften the cluster of deformation variables, ratherthan a single variable, that identifies a diseasedphenotypic presentation.

The introduction of 3-dimensional (3D) para-metric imaging, pending standardization, mayfurther increase the number of indices.Moreover, theinteraction of myocardial tissue mechanics with LVfluid mechanics may further improve understandingof the hemodynamic presentations associated withdiseased states. For example, the characterization of

Sengupta J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 6 , N O . 1 1 , 2 0 1 3

Novel Approaches in Functional Echocardiography N O V E M B E R 2 0 1 3 : 1 2 0 6 – 1 1

1208

intracavity blood flow in 2 dimensions reveals thepresence of a rotating mass of blood or a vortex for-mation inside the LV (6). The strength of the vortexformationmay be relevant to in-depth understandingof the variables useful for improving the mechanicalefficiency of the LV in heart failure (7,8).As the quantity of parametric data increases, 2

challenges are surfacing. First, how can we store andlater access these large databases? Second, what ifthe number of potential data combinations is soenormous that making sense of the data is practi-cally unmanageable for a busy clinician? Forexample, if one examines the decision tree forgrading the severity of diastolic dysfunction in cur-rent guidelines, the algorithm recognizes patternsbased on outcomes from 8 echocardiographic vari-ables (3). However, the 8 variables can mathemati-cally exist in a staggering 40,320 combinations(8 factorial) with an equivalent number of decisiontrees. And yet we hope that in real life, a singledecision tree, based on human observations andconsensus, would faithfully capture all patterns.How feasible will it be to recognize and handle thesecombinations? One could only imagine that aclinician would be interested in methods that wouldassemble data automatically with decision trees thatscore and categorize any variance appropriately.

Data Efficiency Using Cloud-Based Automation

The Internet has broadened the scope of medicalinformation systems and has led to the develop-ment of distributed and interoperable informationsources and services. Global healthcare systemsoffer a rich information sink that facilitates patientand practitioner mobility. An example of suchconverged framework and shared services, knownas “cloud computing,” was first tested for large-scale echocardiography use in the landmarkASE-REWARD (American Society of Echocar-diography: Remote Echocardiography With Web-Based Assessments for Referrals at a Distance)study (9), conducted at a meditation camp in aremote part of India in January 2012. A total of1,024 patients were imaged with handheld ultra-sound units in <48 h. The scanned data were thenuploaded to a cloud-based platform and distributedto 75 centers worldwide. We learned from thisstudy that clinicians easily recognized structuraldatasets such as valve lesions, but the greatestvariability came from their assessment of functionaldata such as the LV ejection fraction. To stan-dardize the functional assessments, we went backto perform the ASE VISION (American Society of

Echocardiography: Value of Interactive Scanningfor Improving Outcomes of New Learners) study(10). During this study, sonographers on the WestCoast of the United States trained physicians inIndia using a novel Internet-based imaging solution(StatVideo, Andover, Massachusetts) that allowedreal-time interactions as the Indian physiciansscanned the patients. These physicians then scannedover 1,000 patients for cardiac function assessment.Both studies thus established that physicians couldbe educated remotely using a cloud computingenvironment for performing focused echocardio-grams that proved useful in expediting mass triage inremote areas.

Use of Robotics and Automation inCardiac Ultrasound

Quantitative measurements in echocardiographyshow a marked dependence on the skills of theoperator to ensure image quality during repeatedmeasurements. Compared with traditional hand-held cardiac ultrasound examinations, automationin image acquisition and analysis may be useful formore cost-effective and widespread applications ofcardiac ultrasound while maintaining image quality.Currently, robotic arms are able to process complete3D image volumes without human intervention andare being used for laparoscopic procedures as anintraoperative guidance tool (11). The use of roboticarms brings in accuracy, consistency, dexterity, andmaneuverability. Moreover, robotic arms are capableof precisely tracking and generating evenly spacedslices without suffering human challenges such ashand cramps, tremors, and fatigue. The use of arobotic cardiac ultrasound arm may be a usefulcomplement for the emerging field of telemedicine.A robotized tele-echocardiography approach wasfirst developed in the late 1990s and is currentlybeing used for enhancing the quality of health carein medically remote areas (12). This conceptis currently also being explored for performing long-distance intercontinental examinations (Fig. 2).Future developments in robotic arms are expectedto couple with automated image analysis (machinevision) for complete automated acquisition,recognition, and quantitative assessments ofcardiac images without the need for manual in-teractions. This may be helpful for performinghigh-throughput cardiac ultrasound scanning inhospitals and communities, and for collectinglarge-volume datasets in conjunction with cloud-based automation.

Figure 2. The Concept of Long-Distance Tele-Echocardiography

The real-time remote assessment is driven through a satellite or internetconnection. The navigation and visualization of a robotic arm with a probe (A)driven by a remote expert works through a traditional computer using aninterface (B) that allows 2-way audio and video conferencing that enablesdirect navigation and visualization of the robotic arm. The remote operator canswitch the number of views for visualizing the motion of the robotic arm alongwith images from the ultrasound machine, which can also be optimized fromdistance. Figure courtesy of TeleHealthNow LLC and Energid Technologies.

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 6 , N O . 1 1 , 2 0 1 3 Sengupta

N O V E M B E R 2 0 1 3 : 1 2 0 6 – 1 1 Novel Approaches in Functional Echocardiography

1209

Cognitive Computing

As more and more data become available, the chal-lenge will be how to mine the vast datasets to facili-tate and expedite decision making. One of the stepsin meeting this challenge could be to developcognitive tools for automated analysis of big func-tional datasets. Until recently, computer-aided in-telligence was labeled as “artificial” and was thoughtto be potentially useful for clinical decision makingusing artificial intelligence–based techniques suchas a neural network, a support vector machine,a decision tree, or a fuzzy expert system withrules produced by a genetic algorithm. Recentdevelopments in the field of cognitive computinghave been boosted by the development of naturalbrain-like intelligence (13). Our minds do notincorporate the slow, sequential, logical ways ofthinking that is a characteristic of artificial intelli-gence (M. Aparicio, personal communication,September 2013). Instead, our thinking is fast andmostly based on our memory of associations that“connect the dots.” In the past, brain-like thinkingwas dismissed in favor of pure logic. Inaccuracies ofhuman reasoning were reasoned by favoring statis-tical analytics. However, statistics per se originallywas applied to dice and card games, which do notreflect the more general world. When a player trainswith high intensity, a better performance is due tomore than just chance. Just like our brains can makesensible decisions in a complex and ever-changingworld, computer-aided natural intelligence can helpwith anticipatory sense making so that we connectthe dots for predictive decision making. Such esti-mates could be additionally supplemented withstatistical reasoning to enable the selection of thebest connections, especially in nonlinear, high-dimensional situations such as with complex diseases.

Figure 3 shows the application of a natural-intelligence cognitive tool (Saffron Technology, Cary,North Carolina) for assessing speckle-trackingechocardiography-derived functional data. Thistechnology has been used for national security anddefense, and for war zone assessments and aero-space missions (14). We first programmed thesoftware to see a dataset for developing a memorybank of associations. For this model, we used oneof the toughest examples in the clinical world wheremultivariable data are important for arriving at acorrect diagnosisdthe differentiation of deforma-tion patterns seen in constrictive pericarditis versusrestrictive cardiomyopathy. Previous investigationshad suggested a specific constellation of deforma-tion patterns that differentiate the former from

restrictive cardiomyopathy (15). The challenge wasto see whether the cluster of deformation variablesthat differentiate constrictive pericarditis fromrestrictive cardiomyopathy could be recognizedusing a brain-like cognitive tool. The 2D speckletracking data from each patient included 20 timepoints in a cardiac cycle, 9 measurements (velocity,displacement, strain, and strain rate in 2 directions,and area) obtained from 6 segments each in apical4-chamber view and short-axis view of the LV atthe level of papillary muscle. Close to 2 millionassociations were examined in 15 patients withconstriction and 14 patients with restrictive cardio-myopathy. Given the high dimensionality andassumed nonlinearity of echocardiography defor-mation data, a coincidence matrix of associations wasconstructed and the entropy-based information wascomputed for each class. Because the resultingmatrices were extremely large, and although all thecoincident information was exploited for classifica-tion, heat map visualization (Figs. 3 to 5) was limitedto the most informative interactions. By differenti-ating these heat maps, the clear separation of diseasestates was obvious. Dominant associations betweenvariables seen in restrictive cardiomyopathy (Fig. 4,red) were separable from dominant associations seenin constrictive pericarditis (Fig. 3, green), althoughsome overlap was seen to exist (Fig. 5, yellow).

Figure 4. Heat Map of Associations in RestrictiveCardiomyopathy

The red rectangular cells represent the presence of significantinteractions seen in 14 patients with restrictive cardiomyopathy.Analysis courtesy of Saffron Technologies, Inc., and AcceleratedVision LLC.

Figure 5. Comparison of Associations in RestrictiveCardiomyopathy and Constrictive Pericarditis

The 2 heat maps shown in Figures 3 and 4 are overlapped. Thematrix of associations seen in constrictive pericarditis (green) andrestrictive cardiomyopathy (red) are primarily nonoverlappingwith only minimal overlap (yellow).

Figure 3. Heat Map Showing a Matrix of Associations inConstrictive Pericarditis

A coincidence matrix of associations between variables of leftventricular mechanics was constructed, and the entropy-basedinformation was computed for each class and limited to the mostinformative interactions. The green rectangular cells representthe presence of significant interactions seen in 15 patients withconstrictive pericarditis. Analysis courtesy of Saffron Technologies,Inc., and Accelerated Vision LLC.

Sengupta J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 6 , N O . 1 1 , 2 0 1 3

Novel Approaches in Functional Echocardiography N O V E M B E R 2 0 1 3 : 1 2 0 6 – 1 1

1210

Wearable Computers

The aforementioned strategies of cloud automation,robotics, data analytics, and cognitive tools canbe projected to one of the many ways intelligentand efficient platforms could develop in the fieldof cardiac imaging. Advances in digital sensors,communications, computation, and storage willcontinue to create huge collections of data. Forexample, search engine companies such as Google,Yahoo, and Microsoft have created an entirely newindustry by capturing information that is freelyavailable on the World Wide Web and providing itto people in useful ways such as satellite images,driving directions, and image retrieval. Just as searchengines have transformed how we access informa-tion, other forms of big data computing will trans-form the activities of scientific researchers andmedical practitioners. The National Science Foun-dation has enthusiastically embraced data-intensivecomputing, with several new funding initiatives forcreating the cyber infrastructure of 21st century(16). The White House administration has alsoannounced support in 2012 for big data initiativesand for building the road map for a trustworthycyberspace (17,18).

For those who fear that this growth of computeruse in cardiac ultrasound imaging will remove usfrom our patients, the answer is that the merger will

J A C C : C A R D I O V A S C U L A R I M A G I N G , V O L . 6 , N O . 1 1 , 2 0 1 3 Sengupta

N O V E M B E R 2 0 1 3 : 1 2 0 6 – 1 1 Novel Approaches in Functional Echocardiography

1211

be seamless. Cardiac images and analytics will beavailable in real time on wearable computers.Wearable computers on glasses, wristbands, andprofessional clothing are not a futuristic dreamanymore (19,20). Their applicability is currentlybeing tested at point-of-care locations. Due to theearly stage of development of this technology, thedevices are still complex. Once they reach a maturestage, cardiac ultrasound imaging and advancedanalytics will be easily integrated into the profes-sional workflow. The potential of such augmentedreality is exciting, offering us many new challenges.As a community, we have a responsibility to

embrace these changes, building the future for ourpatients’ lives and ourselves.

AcknowledgmentsPresented as the 2013 Feigenbaum Lecture at the49th Annual Scientific Session of the AmericanSociety of Echocardiography, Minneapolis, Min-nesota, July 1, 2013.

Reprint requests and correspondence: Dr. Partho P.Sengupta, Icahn Mount Sinai School of Medicine, OneGustave L. Levy Place, New York, New York 10029.E-mail: partho.sengupta@mountsinai.org.

R E F E R E N C E S

1. Goldin I. Global shocks, globalsolutions: meeting 21st century chal-lenges. In: Held D, Fane-Hervey A,Theros M, editors. The Governance ofClimate Change: Science, Economics,Politics, and Ethics. Cambridge, UK:Polity Press, 2011:49–67.

2. Lang RM, Bierig M, Devereux RB,et al., for the Chamber QuantificationWriting Group, American Society ofEchocardiography’s Guidelines andStandards Committee, EuropeanAssociation of Echocardiography.Recommendations for chamber quan-tification: a report from the AmericanSociety of Echocardiography’s Guide-lines and Standards Committeeand the Chamber QuantificationWriting Group, developed inconjunction with the European Asso-ciation of Echocardiography, a branchof the European Society of Cardiology.J Am Soc Echocardiogr 2005;18:1440–63.

3. Nagueh SF, Appleton CP,Gillebert TC, et al. Recommendationsfor the evaluation of left ventriculardiastolic function by echocardiogra-phy. J Am Soc Echocardiogr 2009;22:107–33.

4. Sengupta PP, Korinek J, BelohlavekM,et al. Left ventricular structure andfunction: basic science for cardiac im-aging. J Am Coll Cardiol 2006;48:1988–2001.

5. Sengupta PP, Narula J. Reclassifyingheart failure: predominantly sub-endocardial, subepicardial, and trans-mural. Heart Fail Clin 2008;4:379–82.

6. Sengupta PP, Pedrizzetti G, Kilner PJ,et al. Emerging trends in CV flowvisualization. J Am Coll Cardiol Img2012;5:305–16.

7. Goliasch G, Goscinska-Bis K,Caracciolo G, et al. CRT improves LVfilling dynamics: insights from echo-cardiographic particle imaging veloc-imetry. J Am Coll Cardiol Img 2013;6:704–13.

8. Abe H, Caracciolo G, Kheradvar A,et al. Contrast echocardiographyfor assessing left ventricular vortexstrength in heart failure: a prospectivecohort study. Eur Heart J CardiovascImaging 2013 Apr 14 [E-pub ahead ofprint].

9. Singh S, Bansal M, Maheshwari P,et al., for the ASE REWARDStudy Investigators. American Societyof Echocardiography: RemoteEchocardiography with Web-BasedAssessments for Referrals at aDistance (ASE-REWARD) Study.J Am Soc Echocardiogr 2013;26:221–33.

10. Pellikka PA. ASE-global health andour foundation. J Am Soc Echo-cardiogr 2013;26:A35–6.

11. Boman K, Olofsson M, Forsberg J,Boström SA. Remote-controlled ro-botic arm for real-time echocardiog-raphy: the diagnostic future forpatients in rural areas? Telemed J EHealth 2009;15:142–7.

12. Priester AM, Natarajan S, CuljatMO. Robotic ultrasound systems inmedicine. IEEE Trans UltrasonFerroelectr Freq Control 2013;60:507–23.

13. deBruyn J. Making a Computer‘Think’. Triangle Business Journal.December 2, 2011. Available at:http://www.bizjournals.com/triangle/print-edition/2011/12/02/making-a-computer-think.html?page¼2. AccessedSeptember 4, 2013.

14. Shane H. The Watchers: The Rise ofthe Surveillance State. London, UK:Penguin Group, 2010:281.

15. Sengupta PP, Krishnamoorthy VK,Abhayaratna WP, et al. Disparatepatterns of left ventricular mechanicsdifferentiate constrictive pericarditisfrom restrictive cardiomyopathy. J AmColl Cardiol Img 2008;1:29–38.

16. Kolker E, Stewart E, Ozdemir V.Opportunities and challenges for thelife sciences community. OMICS2012;16:138–47.

17. Kalil T. Big data is big deal. TheWhite House, Office of Scienceand Technology Policy. Available at:http://www.whitehouse.gov/blog/2012/03/29/big-data-big-deal. AccessedSeptember 4, 2013.

18. Obama B. “Remarks by the Presidenton securing our nation’s cyber infra-structure” [transcript]. May 29, 2009.The White House Office of the PressSecretary. Available at: http://www.whitehouse.gov/the-press-office/remarks-president-securing-our-nations-cyber-infrastructure. Accessed September 4, 2013.

19. Kumar S, Nilsen WJ, Abernethy A,et al. Mobile health technology evalua-tion: the mHealth Evidence Work-shop. Am J PrevMed 2013;45:228–36.

20. First US Surgery Transmitted Live viaGoogle Glass (w/Video). MedicalXpress. August 27, 2013. Available at:http://medicalxpress.com/news/2013-08-surgery-transmitted-google-glass-video.html. Accessed August 29, 2013.

Key Words: big data - cognitivecomputing - echocardiography -

parametric imaging - robotics.