Page 1 of 13

Enterprise-OCTAVE Blended Workshop for Latin American Young and Early Career Investigators: Statistical Methods in HIV Vaccine Trial Design and Evaluation

17-18 March 2011, São Paulo, Brazil

Blended Workshop Evaluation

Activity Summary: The Global HIV Vaccine Enterprise and the Online Collaborative Training for AIDS Vaccine Evaluation (OCTAVE Project) are hosting a series of six capacity-building workshops aimed at providing young and early-career investigators (YECIs) with the tools they need to further their careers in HIV vaccine research and development. Upon invitation from Dr. Cristina Possas from the Brazilian Ministry of Health, the OCTAVE Project and Global HIV Vaccine Enterprise conducted a 2-day blended-learning workshop in partnership with the Brazilian Ministry of Health (Department of STD, AIDS and Viral Hepatitis, Oswaldo Cruz Foundation - FIOCRUZ) and the University of São Paulo- USP. This was the first workshop held in Latin America blending online and face-to-face learning for YECIs from this region and focused on key statistical and epidemiologic methods used to design and evaluate early and late stage HIV vaccine trials. Workshop organizers solicited applications from investigators who were within 10 years of training or degree completion and had experience with HIV vaccine pre-clinical and clinical trial design and development. Sixty-eight people applied to attend the workshop and of those, 26 scholars were selected from 6 countries. Trainees hailed from institutions across Brazil, Argentina, Peru, Dominican Republic, Paris and the US and led by faculty recruited from the US (Drs. Zoe Moodie, Yunda Huang, Don Stablein, Jonathan Fuchs, and Amapola Manrique), Brazil (Drs. Artur Kalichman and Esper Kallas), and Switzerland (Dr. Jean-Pierre Kraehenbuhl).

Prior to the workshop, the Health Sciences and e-Training Foundation (HSeT) created a workshop-specific e-learning portal (at http://enterprise.bio-med.ch/). The portal housed curricula developed collaboratively by the HSeT team and workshop faculty. These included background materials on descriptive and inferential statistics, survival analysis and clinical trials as well as a 20-minute self-assessment quiz, articles, and four pre-workshop activities. Trainees also participated on a conference call and webinar on 22 February 2011 led by Dr. Jonathan Fuchs, OCTAVE Project PI. The goal of the webinar was to help orient trainees to the e-learning portal, curricula, and format of the face-to-face workshop. Trainees had 4 weeks before the workshop to review these materials and complete the pre-workshop exercises prior to the face-to-face workshop. This two-day highly interactive workshop combined focused lectures and small group exercises. Simultaneous translation from English to Portuguese was offered to trainees throughout the two day workshop.

The following report will describe in further detail the trainees, organizational support for this activity, and training program itself. In summary, the evaluation confirmed this workshop successfully met its objectives and calls for future regional capacity-building efforts for Latin American YECIs committed to HIV vaccine discovery.

Trainees: The target audience for this training was highly qualified YECIs directly involved in biomedical HIV vaccine clinical trials design, clinical investigators, statisticians and epidemiologists. All were from Latin America and were within 10 years of completing their terminal degree and training program. A call for applications was sent out in early January, 2011 and over 68 applications were received from across Latin America and from Latin

Page 2 of 13

American YECIs living in the US and Europe. Twenty-six trainees were selected by the organizing leadership group based on the ability of the applicants to directly apply the statistical principles in current or planned HIV vaccine trial work, and effort was made to ensure geographic diversity. In addition to the YECIs, three special guests of the Brazilian Ministry of Health were invited to participate to offer their unique perspectives in mathematics and data management and engaged in all face-to-face workshop activities.

Workshop Sponsors: The Public Health Agency of Canada, the National Institute of Health Office of AIDS Research, The Global HIV Vaccine Enterprise, the Brazilian Ministry of Health (Department of STD, AIDS and Viral Hepatitis and the Oswaldo Cruz Foundation - FIOCRUZ), the University of São Paulo – USP, and the International AIDS Vaccine Initiative (IAVI).

Organizational Support: Global HIV Vaccine Enterprise, Health Sciences e-Training (HSeT) Foundation, The OCTAVE Project, the HIV Research Section at the San Francisco Department of Public Health, the HIV Vaccine Trials Network at the Fred Hutchinson Cancer Center, and the Brazilian Ministry of Health-FIOCRUZ.

Workshop Overview: The workshop was held in the Blue Tree Towers-Paulista hotel in São Paulo on 17-18 March with guests arriving on 16 March. The workshop program, a list of participants and images from the workshop can be found at the end of this report (Appendices 1-3).

Day 1 started with welcoming remarks from Dr. Cristina Possas representing the Brazilian Ministry of Health, Dr. Akira Homma, Chair of the National HIV Vaccine Committee, followed by Dr. Amapola Manrique from the Global HIV Vaccine Enterprise. Dr. Jonathan Fuchs, Principal Investigator of the OCTAVE Project led the agenda overview and scholar introductions. Dr. Artur Kalichman offered the first lecture focused on The Latin American Epidemic: the need for new HIV Prevention Strategies, followed by Dr. Zoe Moodie’s overview of Statistical Concepts in Vaccine Design and Evaluation. After Dr. Moodie’s presentation the scholars were divided into four small groups to complete the exercise entitled Final Efficacy Analysis (Appendix 4). Following lunch, Dr. Don Stablein, chief statistician of the RV144 vaccine trial, discussed key lessons learned from the analysis of that trial—his talk was entitled RV144: From Design to Interpretations. This was followed by the second small group exercise (Appendix 4 and Appendix 5, Scenarios 1 & 2), concluding the first day’s workshop activities. A group reception dinner was organized by the Brazilian colleagues to promote collaboration and networking between faculty and students at a local Brazilian churrascaria or steakhouse. Day 2 was opened by Dr. Manrique who offered an overview of the day (Appendix 1), followed by Dr. Moodie’s talk on Strategies to Search for a Correlate of Vaccine Protection. Dr. Esper Kallas, one of the Brazilian iPrEX study investigators summarized the results from the trial in his talk entitled iPrEX: Advances in HIV Prevention: What does it mean for Vaccine Trials? This was followed by Dr. Yunda Huang’s work Evaluating Vaccine Efficacy: The Way Forward. Most of the second afternoon was dedicated to a small group exercise that explored the Implications of iPrEx results on an on-going HIV vaccine clinical trial (Appendix 4). Dr. Fuchs wrapped up and Scholars completed the final evaluation.

Evaluation Methods: A 26 question paper-based survey was administered to Scholars at the end of the workshop. Questions assessed background information, the value of the introductory webinar and pre-workshop e-learning activities, the quality and format of face-to-face workshop instruction and activities, the

Page 3 of 13

extent to which participants felt the learning objectives were met, and potential future application in their settings. All 26 participants completed the evaluation (100% response rate). Responses were collated and are summarized here.

Results

Table 1: Reported fields of expertise

Field of expertise

Answer Options Response Percent

Clinical Research 63.0% Statistics 14.8% Laboratory Sciences 33.3% Vaccines 40.7% Microbicides 0.0% PrEP 14.8% Immunology/Virology 44.4% Non-Human Primates/Pre-clinical research 7.4% Other 29.6%

As seen in Table 1, of the 26 participants, 63% reported working directly in clinical research followed by Immunology/Virology (44%) and vaccines (41%). Approximately 30% had fields of expertise outside of HIV clinical or basic science domains, suggesting this activity was effective at drawing interest from young talent in other health-related disciplines. Overall, scholars reported an average of 5.8 years of experience in research. They were also asked about the reasons they wished to participate in the workshop (Table 2); 81.5% wanted to share or increase their knowledge on statistical methods needed to design HIV vaccine trials as well as network and collaborate. Foremost, 92.6% of participants wanted to learn information that could be applied directly to their current or future research.

Table 2: What were your main reasons or goals for participating in this workshop?

Answer Options Response Percent

Share or increase knowledge on statistical methods needed to design HIV vaccine trials

81.5%

Share or increase knowledge on interpreting the statistical methods used to analyze the results in HIV vaccine trials

77.8%

Network/establish contacts for further collaboration 81.5% Learn information to be applied to current or future research projects

92.6%

Other 0.0%

Page 4 of 13

Nearly 75% of participants strongly agreed that their expectations for the pre-workshop eLearning activities were met, spending an average of 11.8 hours on these accessed through the portal (Range 2-30 hours). The activities are detailed in Figure 1. Fifty-eight percent of participants found the four weeks prior to the face-to-face meeting to complete the eLearning assignments “just right,” as compared to 42% who found the time to be too short, wanting at least one additional week to complete activities given their average weekly workload.

Participants found the eLearning portal easy to navigate with good access and very few technical issues. Also, 68% of participants agreed or strongly agreed that the introductory webinar was helpful. Figure 1 demonstrates that the selected articles and self-assessment quiz were excellent or very good, with best features such as “easy website navigation,” “flexibility,” and “diverse, clear, good examples,” which could be easily utilized. Over 85% of participants especially liked protocol-based learning and many would prefer to have longer access than a year to the online eLearning materials with additional statistical methodologies and examples to be added in the future.

Figure 1. Perceived value of eLearning training activities

During the face-to-face meeting with simultaneous Portuguese translation, 93% of participants felt that the YECI workshop was a good use of their time and 90% strongly agreed that they learned valuable information with a better understanding of statistical methods, which would not have been easily found elsewhere (82%). Group exercises were favored by nearly all participants, receiving the highest ratings (4.67-4.81 on a 5 point scale). Participants especially liked having ready access to faculty and expert opinion and lauded the excellent presentations. They also commented that the interactive exercises engaged both faculty and participants and were critical to the success of the face-to-face workshop. Participants also liked the interactive questions during the presentations with the Audience Response System/iClicker and felt that it created a positive dynamic between the faculty and participants; they recommended having even more interactive questions for future

Page 5 of 13

sessions. Participants felt the workshop was well organized and exceeded the expectations of many. Some participants did feel that one way to improve the workshop would be to add one day to the workshop to account for the amount of information presented.

Figure 2 illustrates that nearly all participants agreed or strongly agreed that the four workshop objectives were met during the face-to-face meeting and they plan to utilize their new skills in clinical trial design, their ability to critically evaluate statistical methods presented in research articles featuring clinical trials. The four objectives evaluated included:

• Review fundamental statistical and clinical trial design concepts relevant to vaccine evaluation

• Understand the activities of Data Safety and Monitoring Boards (DSMBs)

• Describe finding from RV144 and explore the approach to define correlates of vaccine protection

• Consider new ways to design studies that aim to evaluate combination HIV prevention strategies

Eighty-one percent of participants said they will definitely return to the Enterprise portal to review learning materials and 85% state they are likely to contact faculty with questions or concerns. Also, 96% of participants probably or definitely will contact another trainee to foster collaboration and networking, which was one of the fundamental ideas for the workshop. One hundred percent of participants would definitely apply for another Global HIV Vaccine Enterprise workshop and would also recommend the workshop to a colleague. Participants were extremely thankful for this opportunity and would further like to acquire skills in grant writing; immunological correlates or surrogates of protection; a focused workshop on combination prevention including PrEP and HIV vaccines; and study designs and statistical approaches to HIV vaccine development.

Figure 2: Rating of training experience meeting workshop objectives

Conclusion: The first Latin American OCTAVE blended learning workshop in São Paulo was an important advance for fostering and training YECIs across Latin America in Statistical Methods in HIV Vaccine Trial Design and

Page 6 of 13

Evaluation. It also represented the third highly successful workshop of the Enterprise-OCTAVE Capacity Building Series and marked the group’s first collaboration with the Brazilian Ministry of Health-Fiocruz and the University of São Paulo. The evaluation confirmed this method of training was well received and encouraged a high level of interaction between faculty and trainees as well as between the trainees themselves. There is a large interest from Latin America to continue capacity-building workshops such as this one in the future addressing scientific topics relevant to vaccine discovery as well as areas critical to professional development (e.g., manuscript and grant development).

Page 7 of 13

Appendix 1: Workshop Program

Day 1: Thursday, March 17, 2011

Time Lectures Face to Face Facilitator

8:00-8:30 Registration & Breakfast

8:30-8:40 Welcoming Remarks Cristina Possas and Dirceu Greco, Brazil Ministry of Health;

Akira Homma, National HIV Vaccine Committee, Brazil;

Amapola Manrique, Global HIV Vaccine Enterprise

8:40-9:00 Workshop overview and Introductions

Jonathan Fuchs

9:00-9:30 The Latin American Epidemic: the need for new HIV Prevention Strategies

Artur Kalichman

9:30-10:30 Statistical Concepts in Vaccine Design and Evaluation: An Overview

Zoe Moodie

10:30-10:50 BREAK

10:50-12:30 RV144: From Design to Interpretation—Lessons Learned

Don Stablein

12:30-1:30 LUNCH

1:30 – 2:40 Group Exercise: mid-study DSMB meeting

Faculty and Participants

2:40- 3:00 BREAK

3:00-4:30 Group Exercise: Final analysis presentation and discussion

Faculty & Participants

4:30-5:00 Wrap up and check in Jonathan Fuchs

7:30 Reception & Dinner

Page 8 of 13

Day 2: Friday, March 18, 2011

Time Lectures Face to Face Facilitator

8:00-8:30 Breakfast

8:30-8:45 Review of Agenda Amapola Manrique

8:45-9:30 Strategies to Search for a Correlate of Vaccine Protection

Zoe Moodie

9:30-10:15 iPrEX: Advances in HIV Prevention: What does it mean for Vaccine Trials?

Esper Kallas

10:15-10:45 BREAK

10:45-11:45 Evaluating Vaccine Efficacy: The Way Forward

Yunda Huang

11:45-12:45 LUNCH

12:45-3:00 Small Group Exercise Faculty and Participants

3:00-3:30 Summary Day 2/Evaluation Jonathan Fuchs

Brazilian Host Planning Committee

• Jorge Casseb, University of São Paulo

• Alberto Duarte, Brazilian Ministry of Health (did not attend workshop)

• Bruna Fanis, Ministry of Health-FIOCRUZ

• Beatriz Grinsztein, Institute of Clinical Research Evandro Chagas

• Akira Homma, National HIV Vaccine Committee

• Alex Menezes, IAVI

• Jesus Peinado, IMPACTA, Peru (did not attend workshop)

• Christina Possas, National STD‐AIDS Program at the Ministry of Health, FIOCRUZ

• Valdilea Veloso, Institute of Clinical Research Evandro Chagas

Workshop Faculty

• Jonathan Fuchs, San Francisco Department of Public Health, San Francisco, USA

• Peter Gilbert , SCHARP, Seattle, USA

• Yunda Huang, SCHARP, Seattle, USA

Page 9 of 13

• Artur Kalichman, Centro de Referencia e Treinamento em DST/AIDS, São Paulo, Brazil

• Esper Kallas, Universidade de São Paulo, Laboratorio de Investigação Medica 60, São Paulo, Brazil

• Jean-Pierre Kraehenbuhl, HSeT Foundation, Epalinges, Switzerland

• Zoe Moodie, SCHARP, Seattle, USA

• Amapola Manrique, Science Officer, Global HIV Vaccine Enterprise

• Don Stablein, The EMMES Corporation, Rockville, USA

Coordinating Staff

• Krisztina Emodi, University of California, San Francisco (UCSF)

• Kate Porter, Conference Solutions

• Emily Schaeffer, San Francisco Department of Public Health

• Jaclyn Skrivseth, Conference Solutions

Page 10 of 13

Appendix 2: Photographs

Participants of the Enterprise-OCTAVE Workshop on Statistical Methods in HIV Vaccine Trial Design and Evaluation, São Paolo, Brazil, 17-18 March 2011

Jonathan Fuchs (left), Yunda Huang, Cristina Possas, Zoe Moodie, and Don Stablein

Page 11 of 13

Dr. Esper Kallas leads a discussion of the iPrEx trial and potential implications for HIV vaccine studies

(Left) Dr. Zoe Moodie reviews key advantages of randomized trial designs

(Below) Drs. Manrique and Huang confer during one of the small group break-out sessions

Page 12 of 13

Appendix 3: Workshop Participants

Name Organization Country

Vivian Avelino da Silva Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo

São Paulo, Brazil

Alexandre Barbosa Unesp - Univ. Estadual Paulista - Hospital Dia Domingos Alves Meira

Botucatu, Brazil

Pedro Brasil FIOCRUZ Rio de Janeiro, Brazil

Elizabeth de Albuquerque Bio-Manguinhos / Fiocruz Rio de Janeiro, Brazil

Marise Fonseca School of Medicine, Federal University of Minas Gerais

Belo Horizonte, Brazil

Pedro Gonzales Asociacion Civil Impacta Salud y Educacion San Mateo, United States

Rejane Maria Grotto UNESP Botucatu, Brazil

Ruben Iglesias Asociacion Civil Impacta Salud y Educacion Peru

Flavia Lantigua Unidad de Vacunas IDCP-COIN-DIGECITSS Santo Domingo Norte,

Dominican Republic

Fabio Leal Universidade de Sao Paulo São Paulo, Brazil

Mauricio Martins University of Wisconsin-Madison Oconomowoc, WI, United States

Eliane Matos dos Santos Bio-Manguinhos/Fiocruz Rio de Janeiro, Brazil

Tais Mazzola State University of Campinas Jundiaí, Brazil

Vinicius Menezes Oswaldo Cruz Foundation Rio de Janeiro, Brazil

João Miraglia Fundação Butantan São Paulo, Brazil

Raquel Oliveira (did not attend due to illness)

IPEC/Fundação Oswaldo Cruz Rio de Janeiro, Brazil

Telma Oshiro Sumida School of Medicine - University of SãoPaulo São Paulo, Brazil

Antonio Pacheco Fiocruz Rio de Janeiro, Brazil

Maria de los Angeles Pando Centro Nacional de Referencia para el SIDA, Facultad de Medicina, Universidad de Buenos Aires

Buenos Aires, Argentina

Ludimila Paula Vaz Cardoso Tropical Pathology and Public Health Institute, Federal University of Goiás

Goiânia, Brazil

Susan Ribeiro University of São Paulo School of Medicine São Paulo, Brazil

Page 13 of 13

Paula Rigato Comissasriat de l'Energie Atomique Paris, France

Daniela Rosa Federal University of São Paulo- UNIFESP São Paulo, Brazil

Karina Salmazi University of São Paulo São Paulo, Brazil

Daniela Silva HCFMUSP São Paulo, Brazil

Leandro Tarosso University of São Paulo São Paulo, Brazil