A Report to the Health Resources and Services Administration

Office for the Advancement of Telehealth

_____________________

Delivering Comprehensive, On-Demand Clinical Educationfor Rural Practitioners Using Satellite-Based, Broadband Internet

A Demonstration Project

_____________________

A project funded bythe Health Resources and Services Administration

Office for the Advancement of Telehealthin an award to the New Hampshire Area Health Education Center

and via Subaward to Dartmouth CollegeProject #6N76PE00583-03

February 26, 2001

Interactive Media Laboratory

National Center for Public Health PreparednessCollaboratory for Applied Communication Technology

Dartmouth Medical SchoolHanover, NH 03755

Delivering Comprehensive, On-Demand Clinical Educationfor Rural Practitioners Using Satellite-Based, Broadband Internet

A Demonstration Project

I. Introduction and Overview

The Interactive Media Laboratory (IML) at Dartmouth Medical School has been simultaneously developing new models for technology-based clinical education1 and investigating the application of emerging computer/communication networks2

to deliver educational programs to health professionals. The present project extends, and gains leverage from, that experience, through use of existing IML resources: development methods and tools, e-learning delivery technologies, educational models, and an educational program that applies IML’s virtual practicum model (Primary Care of the HIV/AIDS Patient: A Virtual Mini-fellowship with John Bartlett, MD). The HIV virtual practicum allows us to exercise current and emerging Internet technologies applicable to asynchronous distance learning.

IML has recently been named a National Center for Public Health Preparedness, serving as a resource to provide research, development, and training in technology-based learning and the application of communication technologies for health professional education. This report is the first produced by the new Center. We gratefully acknowledge the support of the Office for the Advancement of Telehealth, Health Resources and Services Administration in this research.

A demonstration of this project is available to those with broadband Internet access (at least 200-300 Kbits/sec, guaranteed bandwidth). The url is http://iml.dartmouth.edu/HRSA. Running the demonstration requires the download and installation of a “plug-in” consisting of IML-developed programs, media resources, and Apple QuickTime. Please note that, though the website may be used by students for continuing education, it is intended mainly as a companion to this report.

A. Goals and Objectives

The goal of this project has been to improve our understanding, via development and demonstration, of methods for delivering high-quality, individualized, interactive multimedia programs on-demand via the Internet, for the continuing education of health care professionals who live and work in rural settings.

To accomplish this we

1. converted an existing CD-ROM interactive multimedia program3 so that it can be delivered via the Internet; the program is an excellent example of a

2

clinical education program that employs a robust educational model developed by IML, the “virtual practicum”

2. made use of advanced Internet technologies (landline and satellite-based wireless) that permit delivery of real-time, motion video of sufficient quality to support this type of advanced training application

B. Results

This project has contributed to our understanding of

1. how to employ technology-based learning to provide interactive, individualized, continuing education to health care providers, at any time or place

2. how emerging broadband networks could be used for that purpose

3. the use of satellite-based Internet access for delivery of on-demand educational services to individuals in rural settings

4. the methods, performance, acceptability, benefits, and costs of such systems

C. Potential Benefits

The chief benefit of this knowledge, if applied, will be to improve the delivery of continuing education services to rural health care providers. Currently, the predominant model for delivery of distance learning is videoconferencing. This method has the distinct advantage of delivering good information to a potentially large audience at a distance. However, videoconferencing has a rigid format: it cannot be tailored to the interests and learning styles of the individual learner and it lacks interactivity (other than limited opportunities to ask questions). Further, active participation in a videoconference, is limited to a relatively few, and it requires that the individual be at a particular place at a particular time; i.e., it must be done synchronously. The alternative is to view the videoconference passively at a later time, as one views any instructional videotape.

The current project expands on videoconferencing models and methods for distance learning in important ways:

the format of the virtual practicum model is highly flexible

it offers a variety of topics and learning modes, allowing the learner to match content to his or her interests and learning style preferences

3

it is interactive, allowing the learner to participate in a variety of learning experiences including simulated cases, lectures, computer-based activities, interviews with patients and clinicians, and case discussions, all under the “virtual mentorship” of a world-class practitioner and teacher

delivery via broadband Internet permits asynchronous learning by individuals, allowing them to access educational content according to their schedules

delivery via satellite service such as DirecPC™ permits shifting from an economic model in which a provider of educational services pays a high, per-event rate to one where numerous individual service recipients pay for general Internet access on a per-month basis; advantages are that any information provider can provide information at a fixed, relatively low cost for Internet access, while recipients gain access to a variety of Internet-based services at a fixed, relatively low cost

II. The Virtual Practicum Model

A. Background: Schön’s Reflective Practicum

The virtual practicum model has theoretical underpinnings in Edelman’s Theory of Neuronal Group Selection, Boisot’s Epistemological Space, Kolb’s Learning Cycle, and Schön’s Reflective Practicum. These are discussed in depth in a paper already cited.Error: Reference source not found The elements of the virtual practicum model are derived from Donald Schön’s reflective practicum.4 Schön criticizes most professional education as focusing on the high ground of “manageable problems [that] lend themselves to solution through the application of research-based theory and technique” and not preparing students to work in the swamp of “messy, confusing problems [that] defy technical solution.”

“The irony of this situation is that the problems of the high ground tend to be relatively unimportant to individuals or society at large, while in the swamp lie the problems of greatest human concern. The practitioner must choose. Shall he remain on the high ground where he can solve relatively unimportant problems according to prevailing standards of rigor, or shall he descend into the swamp of important problems and non rigorous inquiry?

The dilemma has two sources: first, the prevailing idea of rigorous professional knowledge, based on technical rationality, and second, awareness of indeterminate, swampy zones of practice that lie beyond its canons.”

4

Outstanding practitioners, who deal well with the swamp, aren’t generally said to have more knowledge than others (though scientific and technical knowledge are essential); instead, they’re described as having more “wisdom,” “talent,” “intuition,” or “artistry.” But these are commonly regarded as phenomena that are not amenable to “scientific" examination; as a result, education in medicine and public health tends to believe that it cannot adequately deal with them.

Another way of expressing Schön’s concern is that educators in medicine and public health tend to ignore the transactional* nature of practice, i.e., the psychosocial aspects in which the highly variable nature of human behavior and human situations plays a significant role. Management of HIV/AIDS is such an area. Scientific understanding of HIV disease is exceptional, and technical methods for its management are proliferating rapidly. However, care is often compromised by behavioral factors—ranging from prevention of transmission to persistence in taking complicated drug regimens—that require knowledge and skills that “lie beyond the canons” of technical rationality. It is in these indeterminate zones of practice that we find well-developed scientific and technical knowledge, balanced with empathy, intuition, and artistry that mark the exceptional practitioner. The application of emerging communication technologies provides a new opportunity to consider how we might better prepare students to develop these various forms of knowledge, and to work effectively in “the swamp.”

The virtual practicum is an example of a model that seeks to do so, in part by applying Schön’s idea of a “reflective practicum,” designed to overcome the “high-ground/swamp” dilemma. Schön describes the reflective practicum as

“... a setting designed for the task of learning a practice. In a context that approximates a practice world, students learn by doing, although their doing usually falls short of real-world work. They learn by undertaking projects that simulate and simplify practice ... The practicum is a virtual world, relatively free of the pressures, distractions, and risks of the real one, to which, nevertheless, it refers ... It is also a collective world in its own right, with its own mix of materials, tools, languages, and appreciations. It embodies particular ways of seeing, thinking, and doing that tend, over time … to assert themselves with increasing authority…

Students practice in a double sense. In simulated, partial, or protected form, they engage in the practice they wish to learn. But they also practice, as one practices the piano, the analogues in their fields of the pianist’s scales and arpeggios. They do these things

* Contrasted with more procedure-oriented practice. As Schön points out, post-Flexnerian medical schools, seeking respectability within the academy, have chosen to emphasize rigor over relevance, and heavily emphasize the technical-scientific aspects of practice.

5

under the guidance of a senior practitioner… From time to time, these individuals may teach in the conventional sense, communicating information, advocating theories, describing examples of practice. Mainly, however, they function as coaches whose main activities are demonstrating, advising, questioning, and criticizing.”

B. The Virtual Practicum Model

The virtual practicum takes Schön’s description literally, using technology to create a computer-generated, immersive environment that has all of these elements, as follows:

It provides a technology-based “Virtual Clinic” or “Virtual Mini-fellowship” that approximates the world of clinical practice, represented as media elements (graphics, video, sound, text) within which the learner can move, work, and learn

Students learn through simulated clinical practice, particularly simulated teaching cases which compress time and space, giving the experience of dealing with “swampy” problems that can evolve over a virtual time span ranging from days to years. There are also documentary-style “interviews” with genuine patients/clients and practitioners, providing narrative impetus and context for considering the practice of medicine or public health from the patient/clients perspective or to hear practitioners’ “war stories”

It provides a virtual world sufficiently immersive and intrinsically enjoyable to allow even busy professionals to ignore, for a time, the pressures and distractions of the real world. The virtual practicum may also reduce the risks of real-world practicing as students develop and apply new knowledge and skills, since these are done in a technology-generated environment before applying them in the real world

It is a collective world in its own right, providing an inviting, strong sense of place that one can visit repeatedly to learn, containing language, materials, and tools which have analogs in the real world of practice and which borrow from the esthetics of best-practices in computer game design; a key feature is use of narrative and case-based reasoning to increase engagement, enhance reflection, and improve learning.

It embodies particular ways of seeing, thinking, and doing via cycles of experience, reflection, abstraction, and experimentation in the tradition of Dewey, Schön, and Kolb; “story-telling" used in the case presentations provides an underlying structure and context for discussions and reflection that “assert themselves with increasing authority” and intensity.

6

Activities include clinically realistic patient encounters, documentary-style interviews with real patients and practitioners, and computer-generated exercises that allow for heuristic learning of facts and rules (Schön’s “scales and arpeggios”).

All this is done under the guidance of senior practitioner (in the best case a master teacher and master clinician) who may

teach in a conventional sense, “communicating information, advocating theories, describing practice examples” via mini-lectures and case discussions

function as a coach, “demonstrating, advising, questioning, and criticizing” via case discussions and guided (with feedback) reflection and experimentation.

C. Experience with the Model

Reaction of students, practitioners, and educators to the virtual practicum model has been uniformly positive and enthusiastic. Two programs have been produced for the education of primary care providers, Primary Care of the HIV/AIDS Patient and Management of Cancer-related Pain, funded by Pfizer Pharmaceuticals/Simon and Schuster and the National Cancer Institute, respectively. Two more, dealing with topics in medicine and public health (clinical genetics and HIV counseling) are in development, with funding from the Centers for Disease Control and Prevention. The model is also being use for patient education, in a program for cancer patients about to undergo treatment, funded by NCI, that program is now in clinical trials. Finally, the virtual practicum model is being applied in a program for practitioners on smoking cessation in pregnancy, funded by the Robert Wood Johnson Foundation.

It is likely that wide adoption of the model and its attendant methods will positively affect professional education in medicine and public health and, at the same time, stimulate the development of new concepts, models, and methods for technology-based learning.

In addition to the current project, a second HRSA-funded initiative, via the New York State/Virgin Islands AIDS Education and Training Center, is exploring use of the HIV virtual practicum for educating primary care providers.

D. The Virtual Practicum and Broadband Internet

The present project has permitted extension of the model and its underlying technologies for use via broadband Internet. This was accomplished by adding a form of video “streaming” to the software infrastructure underlying the model and by experimenting with various forms of video compression/decompression. We

7

conclude, and have demonstrated, that the quality of performance is equivalent to that of CD-ROM when landline access is used and bandwidth is at least 200-300 Kbits/sec. Performance via satellite-mediated Internet access produces similar results, with a single difference: learners can experience latencies (delays) as high as 3.5-4 seconds between making a choice that results in a video stream starting and actually beginning to see the video. We have shown that these latencies can be considerably decreased by using caching schemes. This work is described in generally in the next section and in a Technical Appendix to this report.

III. Work Accomplished

A. Exploration of Internet-based Motion Video Delivery Methods

Good quality motion video is essential to the success of the virtual practicum model. Our previous research has shown that good quality video must have the following characteristics:

Minimal interruption of service while the video is playing (no jerkiness, very few drop-outs or “glitches”)

Adequate frame size and pixel resolution (clear image, large enough to discern clinical details and personal affect, etc.)

Good synchronization between video and audio (e.g., speaker’s voice and mouth movements move together naturally)

Minimal latency between selecting a video sequence and seeing it

The three variables most affecting perceived video quality are: the speed of the microprocessor on the end-user’s (learner’s) computer (client hardware), the compression-decompression algorithm (codec) used to encode the media, and the bandwidth of the network over which the video is being delivered. Thus, we endeavored to find an optimal combination of these variables given the constraints of likely end-user equipment, current codec technology, and available bandwidth (given a broadband environment.) Note that a governing context for this effort was that our demonstration to be “real world:” it would work with “off the shelf” products and services that are likely to be available to rural practitioners.

1. Defining minimum client hardware configurations

We based our decisions regarding minimum client hardware configuration on our previous work with computer-based education of health practitioners, conversations with MedScape (a leading Internet information provider to health professionals) and the American College of Physicians, and a survey of hardware requirements published on the boxes of commercially available computer games. We chose a minimum specification believed to represent the majority of machines

8

and far short of the performance requirements for high-end computer games surveyed at the time, some of which required 650 MHz Pentium III processors and specialized 3-D accelerator cards. Most computers sold today also have sound cards, speakers, and a CD-ROM player. The last could be used to deliver virtual practicum programs when Internet bandwidth is constrained, e.g., when dial-up modems are used.

Thus, the minimum client hardware specification is

Pentium II, 266 MHz microprocessor

Windows 98 operating system

Video card and monitor capable of 640x480, 24-bit color

Sound card with speakers or ear phones

Keyboard and mouse (or similar input device)

CD-ROM player (when Internet bandwidth is constrained)

2. Selecting and implementing a video delivery method

Selecting a video delivery method depended on four criteria (in descending order of importance): the quality of the video and audio delivered to the end-user, (with determinants described previously); the ease of incorporating the delivery method into the authoring environment of the program (the computer code that underlies the HIV virtual practicum); the method’s performance characteristics, including both server delivery capacities and client-side decoding requirements; and the efficiency of actually doing the video compression during the production process.

We evaluated four methods: QuickTime, MPEG-1, RealNetworks G2, and Microsoft Media Player. We determined that Apple’s QuickTime, using the Sorensen codec for video and QDesign for audio, offered the best match with the specified criteria. Video quality was acceptable to excellent (relative to other codecs that could be used in this application); it was easily incorporated into the QuickTime-based video delivery methods of our existing computer code; it could be streamed through standard server protocols and performed acceptably with our minimum hardware specification; and compression, while slow, was straightforward, using commercial, off-the-shelf software. Additional details can be found in the Technical Appendix.

We experimented with numerous combinations of settings, settling on two main combinations: one with smaller video frame size (more constrained broadband – ca. 175 Kbits/sec) and one with larger frame size (less constrained broadband – ca. 560 Kbits/sec). Frame rate was set at the U. S. standard (NTSC) 30/sec, since

9

higher frame rates did not significantly increase bandwidth requirements (when using the Sorensen codec). The lower setting was used for delivery of the program via DirecPC and the higher setting was used for landline-based delivery (see Results, below). Having selected these two levels of quality and bandwidth requirement, we then compressed all video in the program (approximately 2 hours total in over 300 separate files) and audio (approximately 3 hours total in over 600 separate files) two times, once at 175 Mbits/sec and again at 560 Mbits/sec.

Finally, we knew that under some circumstances there might be latencies between selecting a video sequence to play and actually having it start. This could happen if the requesting signal had to be routed through several nodes within the Internet and/or if wireless transmission distances were high, e.g., from a geosynchronous satellite (see description of signal routing with DirecPC in section III. B, below). Previous IML efforts explored the use of caching schemes in which predictions are made regarding which video segment will be required next as the end-user uses an educational program; these video segments would then be “pre-buffered” as a background operation while the end-user is engaged in other activities.5 Those explorations were theoretical in nature. In the present project we modified our software code to implement a simple caching scheme in which we pre-buffer video segments where the choice of segment was unambiguous. That is, if branching occurred and several video segments might reasonably be selected, then no pre-buffering was done and latencies remained at 3.5-4 seconds. In the unambiguous case, the caching method was successful in reducing latencies effectively to zero.

3. Identifying Internet services for rural providers

Given our determination that a minimum of 200-300 Kbits/sec bandwidth is required for Internet-based delivery of the HIV virtual practicum, conventional, dial-up modems and ISDN services were not considered feasible. There are currently five remaining options: existing broadband connections via institutional (e.g., hospital or clinic) services, digital subscriber line (DSL), cable modems, wireless services via satellite, and sending CD-ROMs via the mail.

The first option is available in many urban areas and in some rural settings. Landline-based institutional access to the Internet is steadily gaining bandwidth, with T1 connections (1.5 Mbits/sec) becoming common. It is likely that during off-hours 3 or 4 simultaneous learners using the virtual practicum could be supported by a single T1 connection, without significantly affecting overall traffic flow. We note that other institutions within a community may also provide access for rural provider training, including schools, community colleges, universities, and National Guard units. We explored use of this type of connection during this project, using the 10 Mbit/sec Internet connection at Dartmouth-Hitchcock Medical Center (a major, rural medical center) and commodity Internet connections between our server at Dartmouth Medical School (on Internet II) and the Centers for Disease Control, the HRSA offices in Rockville, MD, the National

10

Cancer Institute, and the medical school at the University of Utah (see, Results, below).

DSL and cable modems provide broadband access, geared primarily to the home market. They are not well-suited to areas with low population density, since they require an extensive infrastructure and cannot be efficiently delivered over wide areas. Thus, Internet service providers have been investing in infrastructure for high-population areas first; with a few exceptions, rural health-care providers will not have these services available for some time. As a result, we did not explore the use of these technologies during this project.

Wireless Internet access via satellite does offer great promise for education and training of rural citizens. Ultimately, low-Earth orbit satellite systems may provide universal, global, broadband Internet access. Teledesic6 is such a system, currently scheduled to come on line in 2005. It will offer bi-directional Internet connections with bandwidths as high as 64 Mbits/sec. Because LEO transmission distances are relatively short, there is low latency between requesting information and receiving it. Whether the service will be affordable has, obviously, yet to be determined.

Meanwhile, Hughes Network Services currently offers a wireless, satellite-based system that provides downstream (satellite to receiver) bandwidths of up to 400 Kbits/sec, theoretically sufficient to deliver the HIV virtual practicum. Hughes DirecPC system uses a geosynchronous satellite which, at a distance of over 23,000 miles has performance characteristics quite different from LEO approaches (see next section for details). DirecPC complements Hughes DirecTV system, which is very widely available to rural citizens in the continental U. S. The same dealer network that installs and supports DirecTV also supports DirecPC (there are even dual purpose dishes available). According to DirecPC and our own experience, the system itself (receiver dish, PC card, software) costs $299, and an additional $50-150 to install. There are various price plans, the least expensive of which is $19.99 per month for 25 hours of access, including landline connection if the ISP is DirecPC. The most expensive service is for businesses, $129.99 per month for 200 hours of access time. There is no cost in using the system for the educational content provider;* they may gain access to the Internet by any means available to them.

The final option for program delivery, sending the HIV virtual practicum on CD-ROM via the mail, is cost-effective and acceptable to end-users. Most client hardware is already equipped with CD-ROM players and much software is already delivered this way. Disadvantages include the need to publish and distribute new CD-ROMs when a program is updated to accommodate changes in our state of knowledge (HIV care is notorious in this regard). Assuring that end-users have the most recent version of a CD-ROM is difficult or impossible. One

* Compared with conventional satellite teleconferencing, in which the content provider pays a per event cost.

11

solution is to provide updates via a dial-up Internet connection; this is feasible when the changes do not require the download of much data. Another advantage of having the Internet connection (even if low-bandwidth) is being able to track end-user progress and to provide other Internet services (such as access to topical websites and email consultation services). IML is developing methods and software tools to provide for updates using this approach, as part of a separate project. Early experience shows this to be a useful alternative.

B. DirecPC and the HIV Virtual Practicum

As already noted, DirecPC provides Internet access via a geosynchronous satellite that serves the continental United States. As shown in the figure below, the client computer gains upstream access to the Internet via dial-up modem (or direct access, if his or her institution has a connection). Any Internet Service Provider (ISP) can be used, or DirecPC can be the ISP; data rates can range up to 56Kbits/sec, depending on quality of telephone lines available to the rural user. On the downstream side, the Internet is accessed via the DirecPC Network Operations Center (NOC) and the DirecPC satellite; data rates up to 400Kbps are achievable, permitting real-time streaming of video of sufficient quality to support advanced, virtual practicum-type programs.

Suppose an individual user is using the HIV virtual practicum program. The virtual mentor, Dr. Bartlett, has introduced the simulated patient, Laurie Matthews, and the learner can now begin to take a patient history. Questions are asked of the simulated patient by pointing and clicking on a menu of choices on the learner’s computer screen. Assume the learner wishes to ask about past

12

medical history. After clicking on the appropriate menu choice, that action is transmitted via modem and telephone line to the Internet via the ISP. The learner’s DirecPC software has put a label on the request that causes the request to “tunnel” rapidly to an Internet server system at the DirecPC NOC. The NOC has very high bandwidth connections to the commodity Internet. The request is passed at high speed to the Internet server that has the video scene of Laurie Matthews giving her past medical history. This server can be located anywhere on the Internet with sufficiently high bandwidth to stream video; it might be a server at Dartmouth or one at HRSA. The server starts streaming the video. The very high bandwidth connection between the Internet and the DirecPC NOC carries the video to the DirecPC NOC, which then transmits the signal to the DirecPC satellite, thence to the receiving dish of the individual using the program.

To our knowledge there exist no previous data regarding the delivery of high-quality, interactive multimedia programming for clinical education via this method. The architecture of the system, particularly the location of the content server on the Internet, affects system performance. Latencies between the learner making a choice and seeing a video stream were estimated at approximately 3-5 seconds, including network transmission times and buffering of video to ensure an uninterrupted start. This project has investigated latencies and other performance parameters with a real program that places severe demands on the system. We also considered the issue of how far to the periphery of the network that content must be placed; i.e., whether the program’s content needs to be downloaded to a local server and accessed via a Local Area Network to achieve adequate performance, or whether it can be served from any server with broadband Internet access. We also investigated the acceptability of the system and its performance to end-users (see Results).

Regarding availability of Internet Service Providers (dial-up access) for rural health care professionals:

Some form of local dial-in access is available in nearly every local calling area in the continental US. A brief check of only two national ISPs (AOL and Mindspring) shows 55 different dial-in numbers serving at least 20 different geographic areas in New Hampshire alone. This doesn’t take into account any local, smaller ISPs

DirecPC has their own ISP service with about 43 dial-in locations listed in New Hampshire alone

Most telecommunications carriers (such as Bell Atlantic) have decided that bundling all services over TCP may be more cost effective in the long run, and plan to offer all consumer services over TCP/IP in the near to medium future.

13

Hughes Satellite / DirecPC is partnering with AOL for delivery of AOL TV content and other integrated media services, with new satellite systems permitting bi-directional transmission via satellite

Additional information about DirecPC can be obtained on the Web at http://www.direcpc.com (note the absence of a “t” in “direcpc”). There are numerous local vendors listed on the Website, and online purchases can be made via the Web from such vendors as PC Connection. America Online is also able to bundle DirecPC services for its customers. In our case, we purchased the system from a local satellite TV retailer in White River Junction, VT. We installed the system ourselves by purchasing an installation hardware kit (hardware and sealant, etc.) for $50. In all, the experience was precisely equivalent to installing a satellite television system, an experience with which many rural citizens are very familiar.

C. Results

1. Landline-based, institutional Internet access

We tested landline-based delivery of the modified HIV virtual practicum in several settings, all delivered via the Internet, with Linux-based and Windows NT-based microcomputers acting as servers. Video quality was set at higher levels (less-constrained broadband – 560 Kbits/sec). Testing included office and clinic computers at a nearby, rural medical center (Dartmouth-Hitchcock), offices at three government agencies (Health Resources and Services Administration, Centers for Disease Control and Prevention, and National Cancer Institute), and a classroom at the University of Utah medical school. In all but the last setting, we were able to deliver the HIV virtual practicum flawlessly, with performance (in reduced latencies) actually better than that achievable from a CD-ROM mounted in the end-user’s computer. In these cases total available bandwidth was sufficiently high to accommodate fluctuations in traffic while maintaining service for the HIV virtual practicum. In the Utah instance, total bandwidth was lower and fluctuations in local network traffic caused bandwidth available for the program to drop below that required; this resulted in periodic halting of video as it was being delivered. We have tested the HIV virtual practicum with as many as 20 computers simultaneously accessing the same media from a single HTTP server without noticeable degradation in performance.

We tested the acceptability of landline-based delivery of the HIV virtual practicum with 16 3rd year medical students at Dartmouth Medical School and with 10 individual practitioners. All found the performance of the system to be completely acceptable, and indistinguishable from performance obtained when using a CD-ROM instead of the Internet.

14

2. DirecPC

We quickly learned that DirecPC’s advertised bandwidth, “up to” 400 Kbits/sec, really meant that the delivered bandwidth was highly variable. As a result, we had to seek an optimal combination of video quality parameters that would be likely to play without interruption under most circumstances. As noted previously, we were able to deliver acceptable video primarily by reducing the size of the video frame, allowing us to deliver video in a “more constrained bandwidth” mode requiring only 175 Kbits/sec. We tested in three locations: at IML, at HRSA, and during a public demonstration at a distance learning conference at a New Orleans hotel. At the first and last locations, and during much testing at HRSA, video played well with rare and unpredictable halting of video as bandwidth became momentarily constrained. Those aside, the main difference between delivery via DirecPC and delivery via landline or CD-ROM was a 3.5 to 4 second latency between selecting a video to play and actually seeing it. However, in a demonstration of the system conducted at HRSA, we experienced intermittent interruptions in video flow. During some of our testing here at IML, we experienced similar difficulties, which were ultimately identified as a bottleneck at the Hughes ground station. Apparently, each DirecPC system is assigned a “gateway” machine at Hughes when it is initially set up. At the time of our testing, some of these gateways were oversubscribed or performing poorly, resulting in a system that would perform poorly until the DirecPC software was reinstalled and another (hopefully different) gateway was assigned to that system. We often had to make several calls to Hughes during the process of setting up a system in order to assure that bandwidth provided for our account would be at least 200 Kbits/sec.

As described previously, we implemented video caching strategies that would allow video to be “pre-buffered” while the end-user was making choices. In many cases, we knew which video segment would be played next. By pre-buffering in this way we were able to eliminate the latency between selecting a video and seeing it. However, a complete implementation of this strategy would require more sophisticated strategies that would allow for predictive or multiple-video-stream caching, to deal with branching situations in which more than one media stream must be handled.

We evaluated user reactions to DirecPC delivery of the HIV virtual practicum using 8 individual practitioners and a focus group. None of these were given a point of comparison in advance of seeing the program; i.e., they had not seen the CD-ROM or landline-delivered version of the program. Again, the performance of the program was found to be acceptable, in spite of occasional reductions in bandwidth that cause video service interruption. It is significant that test subjects did not find the 3.4-4 second latencies objectionable. This might not be the case for programs dealing with topics that require decision-making in a more dynamic setting, such as managing acute trauma.

15

D. Conclusions

We conclude that asynchronous distance learning programs, employing multimedia including motion video of adequate quality, can be delivered on demand via broadband Internet, either landline-based or wireless via satellite. We also conclude that broadband Internet access to rural practitioners is feasible today, using either landline-based, institutional Internet connections (some practitioners) or satellite-based wireless systems connected from the home or workplace (any practitioner in the continental U. S.). However, we note that the DirecPC service may be unreliable and that special arrangements with Hughes may be required to assure adequate bandwidth to accounts of end-users who are using programs such as the HIV virtual practicum.

16

Technical Appendix

I. Video Delivery Methods

We considered the following multimedia formats and architectures for delivering streamed media over the Internet:

1. Sorenson under QuickTime with HTTP progressive download

2. MPEG-1

3. Real Networks’ G2

4. Microsoft’s MPEG-4 under Windows Media Player

Our evaluation of the above 4 formats took into account the following: 1) server system issues, 2) client system issues, 3) media production issues, 4) courseware authoring issues. Details are provided in section III, below.

A. Server system

Server system issues are those in which hardware and software configurations affect the delivery protocol. We considered: 1) latency, i.e., the time between when media is asked for to the time playback begins without further interruption; 2) the ease of setting up and maintaining the video server; 3) scalability, the number of simultaneous streams a given architecture could serve; 4) fault-tolerance, the ability to handle dropped frames occurring with an overly busy network; and 5) the availability and depth of information returned about the performance characteristics of the media stream.

We served our QuickTime from a standard web (HTTP) server, which is very easy to set up and maintain. Our servers run the Linux operating system and the Apache web server, both of which are free and open source, although any web server on just about any platform—including Macintosh, Unix and Windows—could be used with this system. Streaming media via HTTP has the additional advantage of using the same network protocols that other web pages do, bypassing potential firewall issues and other institutional security problems that come up when using other network protocols, such as the Real Time Streaming Protocol, or RTSP.* Instead of “true streaming”, we employed QuickTime’s Fast-start progressive download mode, in which just enough data is downloaded to the client machine to begin playing the media without further interruption. However, as there is no true real-time bandwidth sensing, fluctuating network conditions

* There are several streaming servers freely available for this purpose, most notably from Apple and Entera. We tested the Entera ELSA server for the Linux operating system, and found it to perform well, although buffer times were somewhat longer than those encountered when using progressive download.

17

can cause QuickTime to “starve”, in effect stopping playback, until there is enough data to resume. HTTP is an error-correcting protocol, so lost packets will be re-sent, resulting in better fault-tolerance and improved overall video quality, provided that there is enough bandwidth and the download is started soon enough.

Streaming media via HTTP has the additional advantage of using the same network protocols that other web pages do, making unnecessary, in most cases, any adjustments of institutional firewalls and other security systems to allow these educational programs through.

B. Client system

Client system issues are those primarily affecting the user’s experience. How good does the video need to look to teach, inspire, and emotionally affect? While the effectiveness of video and audio is not completely defined by objective characteristics, in general, “high-quality” video means 1) large size, 2) sharp picture; 3) smooth motion; 4) absence of distracting compression artifacts. High quality audio is marked by accurate and flat reproduction across a wide frequency range, with an absence of clipping, distortion, or other artifacts.

In addition to the actual quality of the media, several factors influence a user’s overall experience, most notably the latency between user-interaction, say a mouse-click that results in a new video sequence being played, and beginning of that media. Any scheme to pre-buffer or intelligently cache media is therefore desirable. A user’s experience is also affected by things that may happen prior to or immediately following media playback, such as the presence of a company logo or splash screen. What happens when the network becomes saturated? Are frames dropped, or does the transmission halt until the stream can resume? For the user, anything which draws the user’s attention away from the experience of the program itself is a disruption.

C. Media Production

From a production standpoint, the questions we investigated were: 1) how well does a given media architecture integrate with one’s production facilities; 2) how easy is it to maintain and change files once they’ve been produced; 3) how good are the software tools available for encoding media for that format; 4) are there branding/licensing issues?

QuickTime was by far the easiest architecture for us to work with on the production side, as our Mac-based Media100 system natively exports to that format, and Terran Interactive’s Media Cleaner Pro 4 compression software allowed us to experiment with a wide variety of encoding parameters to meet our data-rate target. By comparison, Real Network’s G2 format did not provide nearly

18

as many options, and creating MPEG-4 files for the Windows Media Player was difficult considering the Macintosh encoder lagged behind the one available under Windows by a version or two.

D. Courseware Authoring

After the media files have been created, they are played (delivered) inside the running application. The two main questions we asked were 1) how well is the format supported (if at all) by the authoring environment on a given platform; 2) how much control does the architecture allow over the presentation of media?

The HIV virtual practicum program was developed (“authored”) in a multimedia scripting language proprietary to IML called 5L. QuickTime was already being used in 5L for its media delivery in a CD-ROM environment and was flexible enough to allow important improvements to be made under mid-to-low bandwidth conditions. In particular, a “preload” command was developed whereby media could be requested from the server in anticipation of being needed, dramatically reducing the delay experienced by the user when playing video over the network. The running program could also check the validity of a URL to be sure that the network is available and the desired media clip is actually present before trying to play it.

II. Compression Settings Media compression is considered an art by many, with few hard-and-fast rules and many variables. Having settled on a particular method of delivering digital video via the Internet (Sorensen codec and QuickTime), we provide the following settings, with clarifications and rationale, to others who may wish to use the same approach.

A. QuickTime, Fast-start, with compressed headers.

Explanation: Using the Fast-start option enables QuickTime to begin playback sooner, decreasing the possibility of distracting the learner with program delays. Compressed headers also speeds up the process of the server polling to see whether a clip exists and helps to start playback more quickly.

B: Data rates (for DirecPC)

Video: 125 kbpsAudio: 48 kbpsAverage total: 175 kbps

Explanation: Data rate is the single most important objective factor determining the overall quality of video. The DirecPC system advertises a maximum throughput of 400kbps, but this proved too optimistic in our experience. After

19

extensive testing, we determined that accommodating the very real and common fact of network spiking and saturation obliged us to compress our 284,000 Kbps source material down to a total bit rate of 175Kbps (a 160-fold reduction). Even here we were prone to encounter occasional “starvation” in which the playback would stop as more data is buffered, but in the absence of any bandwidth or Quality of Service (QoS) guarantees, this bit rate offered the best compromise between video quality and the occasional playback glitch for the DirecPC system. In lower bandwidth environments, it is important not to sacrifice audio quality for small gains in video performance, since users respond more severely to audio problems than to video problems. Thus, we kept our audio rate fairly high, at 48 kbps.

C. Audio CODEC

We used QDesign’s Professional Music 2 codec, which has compression capabilities very similar to the popular MP3 format for downloading music files, including high-fidelity reproduction across a wide range of frequencies and similar compression ratios. We encoded with 44.1 kHz 16-bit mono settings, with a data rate of 48 kbps and a low pass filter with corner frequency at 14 kHz.

D. Video CODEC

Sorenson Developer codec v.2 in millions of colors, 30fps, Keyframes every 300 frames, Natural:50,Size:100; video data rate: Variable bit rate with 175 kbps limit.

E. DirecPC Frame Size

240 x 180 pixels

Explanation: The 1/8 screen size maintains the aspect ratio (4:3) of the original source (640 x 480). Frame size and frame rate are some of the most decisive factors in the processor’s ability to decode fast enough to play back video without problems, and the data rate necessary for acceptable video quality. After testing many combinations, we decided that this provided the best quality video given our bandwidth limitations and our desire to have the video play back acceptably on lower-end systems. (Our recommended minimum specification for this program is a Pentium II 266 MHz system)

F. Frame rate

30 frames per second (fps)

Explanation: Maintaining a full-motion frame rate (usually 30 frames per second) is optimal because fidelity to the natural movements of people is the best way to convey important elements of facial and emotional expression. With our choice of video codecs and frame size, we were able to use full motion video with this project.

20

G. Keyframes

1 per 300 frames, or 10 seconds

Explanation: We elected to have a keyframe – a complete video image that is the basis for the following delta frames – once every 300 frames, that is, every10 seconds. This relatively high keyframe rate was appropriate because the video clips used in our application always played from start to finish, and because we needed to conserve video bandwidth as much as possible (and more frequent key frames require more data).

H. Sorenson-specific attributes We used the most computationally-demanding 2-pass variable bit rate encoding scheme for our video, as this processing time only affects the developer and allows the encoding to be maximally intelligent about its allocation of the data rate. Again, this is a result of our need to get the maximum video quality possible into a relatively low overall data rate to accommodate the delivery system’s bandwidth.

III. Other Media Delivery and Server Systems Considered

Although we selected QuickTime as our video and audio architecture for this project, there are viable alternatives that might be used.

A. MPEG-1

MPEG-1 is a robust, well-established standard file format that only accommodates specific frame sizes and full-motion frame rates (25 or 30 frames per second). As a result, the data rates necessary for MPEG-1 video (with frame sizes large enough for our application) start at around 450 kbps, while the theoretical maximum bit rate for the DirecPC system is 400 kbps. This is the primary reason we did not pursue MPEG-1 for this project.

Playback in our authoring environment was also an issue, since our 5L engine is cross-platform between Macintosh and Windows. While both Real and Microsoft’s player can play MPEG-1 files on Windows systems, Apple’s QuickTime 4 could not. (QuickTime 5 has this capability, but was not available at the time of our tests.) On the other hand, QuickTime could play MPEG-1 files on the Macintosh, but at the time of our testing, there was no MPEG-1 playback capability for the Macintosh from either Real or Microsoft. Newer versions of the Microsoft Media Player for Macintosh and the Real Player using an MPEG-1 plug-in from Digital Bitcasting Corporation have theoretically filled this gap, although we have not tested these at the time of this report.

MPEG-1 can be served effectively from a standard web server, and from several proprietary streaming servers, including the RealServer. On Windows systems,

21

the Microsoft Media player used in conjunction with a standard web (http) server such as apache worked very well for progressive download, while the RealServer with extensions from Digital Bitcasting Corporation worked very well as a true streaming server.

B. RealNetworks G2

RealNetworks’ G2 compression algorithms produced good video, but placed frustrating limitations on the developer. First, latencies between selecting a video stream and seeing it were unacceptably long. Even under good network conditions, 10-20 second latency were common, contrasted with latencies of 3-5 seconds with other methods, under the same network conditions. Second, RealNetworks defaults to playing an advertisement “splash” screen before every clip, which will detract significantly from the educational and esthetic impact of a program; that we, as experienced developers, could find no easy way to disable this “feature” indicates to us that the parent company is less concerned with education than with marketing. Third, in order to stream a large number of Real-encoded media simultaneously, the content provider must pay for expensive server licenses (the RealServer Pro edition can cost as much as $8K at the time of this writing). Finally, Real has recently been criticized for its usage tracking and privacy policies, which could be pose significant concerns regarding individual privacy or, for some kinds of training applications (e.g., counter-terrorism), security concerns.

C. Microsoft Media Player 6

Microsoft is rapidly gaining ground as a significant “player” in Internet based media services. Typically, they are ignoring efforts already underway to create an open standard. Microsoft’s proprietary MPEG-4 codec and its Media Player 6 streaming methods provide excellent video quality (though audio quality is sometimes marginal). The codec compressed quickly, looked sharp, and the Media Player 6 provides a large amount of information about the data stream, (very useful in analyses of network traffic and courseware delivery). There is no charge for using the tools needed to encode MPEG-4 but, to serve media properly, developers and content providers must use Microsoft server software products.

Here are the main factors that caused us to prefer QuickTime over Media Player: As with Real, there was no simple path to hosting the Windows player in the software engine that underlies the virtual practicum. Microsoft’s lack of flexibility regarding supported server platforms effectively commits developers and content providers to Microsoft products. Finally, Microsoft’s historic tendency to avoid common standards (unless they invent them) and toward proprietary solutions seemed to place unnecessary restrictions regarding evolutionary paths for Internet-based media delivery.

That said, recent announcements of the feature set for Windows Media Player 7 (unavailable at the time of our testing) claims VHS-quality at 400 Kbps, along

22

with a server that delivers secure content with options to embed multiple streams. They provide excellent support to the developer community in the form of training and voluminous on-line documentation. Finally, Microsoft is also a leader in digital rights management, something which the other companies have only weak support; this is obviously an important issue for content developers who wish to secure rights and attribution for their creations. So, in the end, practical advantages may outweigh philosophical concerns. We recommend paying close attention to these developments.

IV. Media Pre-buffering and Caching Schemes

For the foreseeable future, available network bandwidth will continue to be the biggest obstacle in delivering high-quality video to end-users of multimedia educational programs. Furthermore, the creation of seamless, immersive environments for such programs requires that they be able to respond in a fraction of a second. Delays longer than about a second may, for some learners and some topics, detract from the learning experience.

Latency reduction can, as already addressed by this project, be reduced by caching schemes in which large blocks of data (typically, a video stream) are pre-buffered. The cited technical reports describe these in some detail. In the simple implementation done for this project, we modified our authoring environment to take advantage of a feature in Apple’s QuickTime called “HTTP progressive download.” With this method, just enough data of an individual media element is downloaded to the host computer so that playback can continue without interruption throughout the duration of the clip. While the end-user is engaged in studying screen layouts and making decisions, the program caches enough of the

1 Henderson, J. Comprehensive, Technology-Based Clinical Education: The “virtual practicum.” Int’l J. Psychiatry in Medicine, 1998; 28:41-79. Available online at http://iml.dartmouth.edu/~joe/vpract.html.2 Henderson, J., Carter, J., Campbell, D., Noel, M., and Roberts, B. Next-Generation Distance Learning System for Professional Education and Training, final report on project funded by the Defense Advanced Research Project Agency and the National Science Foundation via subaward #CCS600658001C from the Massachusetts Institute of Technology, August, 1999. Available online at http://iml.dartmouth.edu/resources/library/reports/darpaproject/final_report.doc3 Henderson, J. and the Interactive Media Laboratory. Primary Care of the HIV/AIDS Patient: A Virtual Mini-fellowship with John G. Bartlett, MD. Originally published by Appleton and Lange: Stamford, CT, 1997. Extensively updated in 1999 to include May, 1999 PHS guidelines on antiretroviral therapy. This program is intended for physicians, nurse practitioners, and physician assistants. It is currently undergoing its 3rd major revision to include new confidentiality and partner notification laws, resistance testing, and Hepatitis C, as well as epidemiologic and antiretroviral updates.5 Henderson, et al. Op. cit. Also available online at http://iml.dartmouth.edu are other documents relevant to caching:: A Performance Estimation Model and Content Sourcing Algorithm for the Seamless Integration of Hybrid Multimedia Applications. Amol M. Joshi, IML Report, January 1997; Predictive Prefetching Algorithm for Real-Time; Delivery of Digital Video in Interactive Applications. Vladimir Ristanovic, IML Report, May 1997; Protocols for Real-time Media Transmission. Vladimir Ristanovic, IML Report, May 1997; and A Video Streaming Performance Model and Content Sourcing Algorithm for Optimizing the Quality of Service of a Multimedia Server. Amol M. Joshi, IML Report, April 1997.

6 See http://www.teledesic.com.

23

next video clip to be available for near-immediate playback when the user makes a choice that activates the video. In this way, a small size cache stays one step ahead of the user’s needs, allowing for rapid response to user interaction and uninterrupted media play without having any serious impact on a user’s hard drive storage.

That said, other strategies are possible that were not explored in this project. They are mentioned here to point to areas for further study.

1. Increase the bandwidth at all points between the server and the client (user)

2. Use network protocols that allow for fixed Quality of Service at all points between the server and the client (such as IPv6).

3. Have multiple servers and let the client pull data from the one that is closest (in terms of network topology)

The first two options will occur over the next few years, as Internet infrastructure is upgraded and new protocols (i.e., IPv6) are instituted. (IPv6 will support advanced capabilities such as reserving point-to-point bandwidth for uninterrupted video/audio data transfer.) Upgrading the Internet infrastructure is expensive and will take time, and while IPv6 is a well-established standard, there is currently little incentive to upgrade existing Internet hardware to newer, more expensive hardware that supports IPv6 while the existing hardware is still working with the current protocols. Thus, changeover to IPv6 is moving slowly, and will continue to move slowly until a “critical mass” of IPv6 hardware is accumulated on the commodity Internet. Which leads to the remaining option.

The third option is commonly referred to as a “mirroring” scheme, in that the content on one, authoritative server is copied to other servers across the network, and these servers are available to stream media on-demand to end users. This presents several challenging issues regarding methods for distributing media securely, reliably, and efficiently to several mirror sites, and how clients can automatically choose the best mirror site to use at any given time.

V. Future of Streaming Media:

The following trends seem clear: 1) streaming media is becoming increasingly popular; 2) the tools to capture, compress and deliver streaming media are becoming more sophisticated and easier to use; 3) the quality of the video is improving on every dimension: larger size, clearer image, smaller file size; 4) the infrastructure to support streaming is becoming more robust.

The streaming media developer community has been extremely active and competitive since our initial exploration in March 2000. Real Networks (now

24

called Real.com) released its Real 8 for Macintosh and Windows, claiming even better video and audio compression, while Microsoft released its Windows Media Player 7 for Windows, and made a beta of version 8 available. Apple has released three public betas of its latest, QuickTime 5, slated for official release early in 2001, which includes a new version of the Sorenson’s codec which, in our early tests, seems to regain parity with its competitors.

Given great foment and flux in this field, it is difficult for developers to adhere to any particular product or strategy for long.

From our perspective, Apple’s philosophy and actions are worth noting, as developers (and the Federal government) choose among several alternatives. They have never charged for their basic QuickTime player, and have made the source code for their QuickTime Streamer server free and available for anyone to use or port to other operating systems. Publishing their complete API is an act of developer good will that indicates a willingness to adhere to standards (they support the standard RTSP protocol), which stands to benefit the developer community generally. Indeed, Network World reported on December 18, 2000, that “Sun, Apple, Philips Electronics, Kasenna, and Cisco have announced the foundation of the Internet Streaming Media Alliance (ISMA), formed to promote open standards for developing end-to-end streaming media products over IP.”

4 Schön DA. Educating the Reflective Practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass, 1987.

25