A Pilot Study

The Local Impact of Cultural Exchanges

February 10, 2018

Improving and innovating cultural exchanges requires understanding the experiences and impact for international visitors as well as host communities

Premise

To date, such systematic evaluation has been conspicuously missing

Hosting international exchanges could be highly desirable for local communities to thrive in a global society

Therefore, the local impact is an indispensable aspect of the larger goal of creating mutual understanding through cultural diplomacy

Develop tools to capture and estimate how exchange programs impact local communities – NOT international visitors

Objective

The Type of People-to-People Exchanges We are Interested in …

In person

Immersive

Interactive

Case in Point: IVLP

A network of communities across the country

Diverse range of source countries, representing countries without significant exposure among the broader American public, while important for U.S.’s global engagement

Participants representing influencers/“elites” of source countries

Generally limited travels and professional exposure to these source countries

Authentic interactions created for both professional and general public audiences in local host communities

“Unique” Aspects As Compared with Other Exchanges

Phase 1Develop and pilot test the instrument

Phase 2Refine the instrument and process for partial roll-out

Phase 3Capacity-building and full implementation throughout the network

2016-17 2017-18 2018-19

Our Approach

Focus on Direct, Immediate Impact

Indirect

Direct

Immediate Longer-term

Pilot Study Goals

Develop a framework to guide the analysis

Design a survey instrument to capture local impact

Test validity of the instrument

Refine instrument and process for partial roll-out

Design Test Refine

Developing the Framework

Extensive search and review of related research literature in summer 2016

Six field visits as preparation for the pilot during fall 2016

Met with 25 CBM staff, 30+ partner organizations, 40+ volunteers, totaling 45 hours of interaction

On-going consultations and discussions with CBMs and Global Ties, including a workshop in Washington, DC with practice leaders and academics

Online survey of Volunteers and Resources from six CBMs during spring 2017

First Step

Estimate local aggregate “exposure” of the community to IVLP participants

Collect aggregate data on volunteer and resource interaction

Presence Exposure Impact

Aspects of “Local Community”

Youth (K-12 / higher education)

Community Leaders

Organizations/Professional Community

Volunteers

Diaspora/Immigrant/Refugee

Knowledge Capital — Resources that enable one to be better informed about other countries and international affairs as well as professionally

Social Capital — Resources that enable one (organization or individual) to connect to larger social networks, international as well as local connectivity

Cultural Capital — Resources that enable one to develop Cultural Literacy, Empathy, Effective Cross-Cultural Communication Skills, Appreciation of Diversity, Global Mindedness

Civic Capital — Resources that enable one to develop civic spirit: Volunteering, contribution to community, community belonging

Economic Capital — Resources that enable one to develop business connections and opportunities and to improve local workforce through professional capability-building

Local Impact Drivers

Culture and Knowledge are the primary impact drivers. An expanded global outlook, along with increased knowledge (both knowledge about the “other” and self-knowledge about the U.S.), appears to be a key benefit for both Volunteers and Resources

Summary Findings – Impact Dimensions

Improving local connectivity, strengthening local pride,and enhancing local workforce development are worth noticing as secondary impact

International connectivity and the standard economic impact indicators are less than apparent or compelling

Volunteer Characteristics

163 volunteer respondents (15% response rate)

55% Female / 45% Male

68% with post graduate degree

Median Age is 60yrs old, 25% under 50yrs old

83% white, 5% black/ 8% Hispanic

Median time as volunteer – 8yrs

67% employed full or part time, 29% retired 24

PresenceMore or less represent the general volunteer profile: older, white, educated, and more volunteering experience

75% of respondents either hosted or attended at least one dinner/home visit during the last 12 months

Statement Once a Year

Few Times a Year

Once a Month

SeveralTimes a Month

Once a Week

Several Times a Week

During the last 12 months, how often have you attended, participated, or volunteered in international visitor programs and activities?

19 54 19 6 1 1

Statement Never 1 2 3 4 5+# of times hosted an international visitor(s) for dinner as part of an home visit during the last 12 months?

38 19 15 12 8 8

# of times attended home dinners/visits with international visitor(s) that were hosted by someone else during the last 12 months?

61 17 10 4 2 6

Program Participation

Measures

Cultural Capital Index

Knowledge Capital Index

Civic Capital Index

- Civic Participation Index

Economic Capital Index

Five indices constructed from survey measures

Indicators of Social Capital

Cultural Competency Statement (Agree/Disagree)

MeanScore(1-5)

FactorOne

Loading(51%)

FactorTwo

Loading (17%)

Drop?

I feel more comfortable explaining my personal values to people from

other countries4.2 .730 X

I have a better understanding about reasons and causes of conflict

among nations of different cultures.4.1 .725 X

I am better able to discuss cultural differences from an informed perspective

4.3 .815

I am more likely to consider different cultural perspectives when evaluating global problems 4.3 .811

I am better able to describe how different cultural values and beliefs

explain behaviors from around the world4.2 .819

I am more aware of how my own culture influences how I see people

from other countries4.2 .812

In situations where I must deal with cultural differences my comfort

level has increased4.2 .819

My ability to differentiate between similar cultures has not improved

(reverse coded)4.0 .852

My understanding of how various culture of this world interact socially

has not increased (reverse coded) 4.0 .841

Index Reliability Score (Cronbach’s Alpha)=.87; Two factors, due to reverse coding

Cultural Capital Index

Knowledge Statement (Agree/Disagree)Mean Score

(1-5)Factor

Loading(57%)

Drop?

I have learned a great deal about international affairs and issues generally 4.2 .822My sources of information about international topics have been substantially expanded 4.1 .779 X

I have a deeper understanding of the United States’ place in the world 4.2 .803My friends and family have learned a great deal about other countries and cultures through my involvement in international visitor programs 4.2 .715

I learned a great deal about the country and culture of international visitors whom I have met 4.4 .663

I have learned a great deal about my local community 3.6 .758 XI have gained knowledge that helps me professionally 3.8 .690I have learned new skills that allow me to do my job better 3.5 .768My co-workers and colleagues have learned a great deal about other countries and cultures through my involvement in international visitor programs

3.6 .765

Knowledge Capital Index

Index Reliability Score (Cronbach’s Alpha)=.90; One factor

Knowledge Statements (Professional/Job Related) Mean Score(1-5)

EmployedFull-Time

OtherStatus

I have gained knowledge that helps me professionally* 4.0 3.6

I have learned new skills that allow me to do my job better* 3.7 3.2

My co-workers and colleagues have learned a great deal about other countries and cultures through my involvement in international visitor programs

3.7 3.6

*Significant difference between employed full time and other at p<.05 or less

Professional Knowledge Capital

Civic Capital Statement (Agree/Disagree) Mean Score(1-5)

Factor Loading(68%)

Drop?

I feel a stronger connection to other members of my community3.9 .879 X

I have a greater appreciation for other members of my local community

3.8 .878

I have a better understanding of my local community’s needs 3.6 .841

I have deeper appreciation for my community’s shared values 3.9 .827 X

I have greater pride in my local community 4.1 .828

I am more willing to work with other members of my community on local issues or problems

3.8 .845

I have gained skills or experiences that make me better able to contribute to my local community

3.9 .758

My attachment to my local community has been strengthened 3.8 .904

Through my involvement and participation, I have met and been exposed to people in our community that I didn’t know previously

4.2 .669 X

Index Reliability Score (Cronbach’s Alpha)=.94; One Factor

Civic Capital Index

Civic Activity (Less/More Likely)MeanScore(1-5)

Factor Loading(69%)

Drop?

Participate in events or activities sponsored by groups reflecting a national heritage different from my own 4.0 .849

Participate in community service activities 3.8 .854

Have a lengthy conversation with a non-native English speaker 4.1 .813

Pay attention to international news about issues and happenings in other countries 4.3 .744 X

Attend a lecture/workshop/ campus discussion on international/ global issues 4.2 .815

Volunteer in my local community 3.8 .866

Participate in a local community cultural or social event 4.0 .864

Donate to a local charity or organization 3.7 .795 X

Become more engaged with a local or community issue 3.8 .872

Index Reliability Score (Cronbach’s Alpha)=.94; One Factor

Civic Participation Index

Economic Impact

Mean

Score

(1-5)

Factor

One

Loading

(47%)

Factor

Two

Loading

(15%)

Factor

Three

Loading

(11%)

Drop?

Improve the ability of the local community to attract international investment 4.0 .804

Enhance the ability of local businesses to export goods and services overseas 3.8 .819

Create opportunities for local businesses to partner with international companies or organizations 4.1 .596 X

Help create jobs in the local community 3.5 .822

Provide me unique professional development opportunities that I wouldn't have had otherwise 3.8 .897

Provide me unique business opportunities that I wouldn't have had otherwise 3.4 .856

Boost the local travel and tourism industry 4.0 .648 X

Enhance the image of the community as a good place to

live4.4 .928

Index Reliability Score (Cronbach’s Alpha)=.89; Three Factors

Economic Capital Index

Statement (agree/Disagree)Mean Score

(1-5)Factor

Loading(66%)

EmployedFull-Time

OtherStatus

Provide me unique professional development opportunities that I wouldn't have had otherwise* 4.0 3.6 .821

Provide me unique business opportunities that I wouldn't have had otherwise*

3.6 3.2 .821

I have gained knowledge that helps me professionally* 4.0 3.6 .780

I have learned new skills that allow me to do my job better* 3.7 3.2 .862

My co-workers and colleagues have learned a great deal about other countries and cultures through my involvement in international visitor programs

3.7 3.6 .768

Total Summative Workforce Development Index* 3.8 3.4

*Significant difference between employed and others at p<.05 or less

Index Reliability Score (Cronbach’s Alpha)=.87; One Factor

Local Workforce Development?

Question % PercentageOver the last 12 months, have you developed any close personal friendships or relationships with any international visitors you have met through these international visitor programs and activities?

45

Travelled internationally to visit any personal friends met through program during last 12 months 12

Likely/Very Likely to travel internationally to visit personal friend within two years 67

Personal friend travelled to U.S. to visit during last 12 months 14

Likely/Very Likely that personal friend will travel to U.S. to visit within two years 44

Social Capital Indicators - Personal

Question % PercentageOver the last 12 months, have you begun a formal business or professional relationship (e.g. import, export, consulting, joint projects, cooperative agreement, etc.) with any of these international visitors you met through these international visitor programs

11

Business or professional contact is “very useful” for organization (out of those with formal relationships) 44

Travelled internationally to visit a business contact during last 12 months 8

Likely/Very Likely to travel internationally to visit contact within two years 29

Business Contact travelled to U.S. to visit during last 12 months 6

Likely/Very Likely that business contact will travel to U.S. to visit within two years 26

Social Capital Indicators - Business

Comparison of Index Scores

*employed respondents only

Resource Characteristics 73 resource respondents (4.5% response rate)

Categories of Respondents27% Community/Non-Profit/Faith-Based22% Higher Education10% Governmental 8% Advocacy7% Business 6% Economic Development/Trade

Yrs volunteering w/international visitor programs16% One Year32% Two to Five Years52% Six Years or More

Categories match the overall “population,” except for representations of Advocacy and Government

Comparison of Index Scores

3.9 3.93.7 3.7

3.6

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

Cultural Capital Index Knowledge Capital Index Civic Capital Index Civic Participation Index Economic Capital Index

Mea

n Sc

ore

Index

The employed measures are generally reliable and consistent with qualitative findings

Additional / reframing measurement of - Workforce development (professional knowledge & economic

impact)- Social/Community Trust- Local Connectivity- Multicultural and integrative civic activities

Increase number of reverse coded items to reduce social desirability bias

Selectively trim scales of duplicative or less reliable items

Summary Findings – Measurement

Phase 1Develop and pilot test the instrument

Phase 2Refine the instrument and process for partial roll-out

Phase 3Capacity-building and full implementation throughout the network

2016-17 2017-18 2018-19

Our Approach

enriching and strengthening the narrative, based on data, about the value of IVLP and cultural exchanges in general at national and local levels

Your participation in Phase II (late spring-early summer) is vital to …

Step 1: Compile IVLP exposure report

Step 2: Field the online survey

How to participate …

We will work with you to adhere to best practices in survey administration, including a) pre-notification of survey to target population, b) multiple channels of recruitment and communication, c) consistent and regular invitation to participate in the survey over a long-period of time

Completion of the 1-page Exposure report is an important, integral part of program management

Things to Note …

Data privacy concerning volunteer and resource contact information is strictly protected. The survey follows Federal human subjects guidelines

For Phase II, there will be no interviews and focus groups as part of the study

Reporting and analysis will be shared with each participating CBM