2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose...

2008 FAEIS Annual Longitudinal Assessment

With a Comparison to the 2007 Survey Results

The purpose of the FAEIS annual evaluation is to develop longitudinal and trend analyses in order to assess progress and develop program components for the future.

Objectives

Describe selected demographic and other variables that have potential association with FAEIS use.

Determine the level of usefulness of FAEIS data resources and applications.

Determine the level of satisfaction with selected components of FAEIS.

Determine levels of adoption of FAEIS and factors that explain the levels of adoption.

Determine the relationship of selected demographic and usage variables (involvement, use, experience, primary job title) with perceived usefulness and satisfaction with FAEIS.

Methodology

2007- Panel of Experts - 7 FAEIS project staff, USDA representatives and chair of the FAEIS Panel

2007 - FAEIS Panel Review and Approval

2007 - Instrument Development 2008 - Population 1070 Users; 285

Respondents 2008 - 2 Follow Up Reminders

2008 Participants

Staff Faculty Department Head College Associate/Assistant Deans College Dean or VP University level administration “Government” added with recoding

Question 1 in 2008 My Involvement with FAEIS Includes: (select as many as apply)

(all possible combinations displayed; N=285)

Question 1 My Involvement with FAEIS Includes: (select as many as apply)

(all possible combinations displayed; N=285)

Involvement Type 2007Percent of Total

2008Percent of Total

Data Entry 54 53.3

User of FAEIS Products 8 9.1

Data Entry & Analytical 7 7.7

Analytical & User of FAEIS products

3 6.0

Analytical 6 5.6

Data entry & User of FAEIS products

5 5.6

None 3 2.8

Data entry, Analytical, & User of FAEIS products

4 2.5

No answer 2 2.5

Analytical, Product developer, & User of FAEIS products

0.003 2.1

Data entry & Product developer

3 1.1

Data entry, Analytical, Product developer, & User of FAEIS products

2 0.7

Data entry, Analytical & Product developer

2 0.4

Analytical & Product developer 0.003 0.4

Product developer 1 0.4

Product developer & User of FAEIS products

0.003 ---

Question 1Involvement with FAEIS of Survey Respondents(“none” and no answer responses eliminated)

2007 Percent of Total

(N=308)


(N=285)

Data Entry 81 75

User of FAEIS Products

21 27

Analytical 25 15

Product Developer

9 5

Analysis – in 2008 there were fewer data entry survey respondents, more users of FAEIS products, but fewer respondents indicated that they are analyzing and developing their own products, compared to 2007 survey respondents.

Question 2 How often have you used FAEIS in the past 12 months?

(N=285)

2008

Question 2 How often have you used FAEIS in the past 12 months?

2007 Percent of Total (N=308)


(N=285)

Not at all 26.3 27.4

1-2 times 47.4 44.9

3-4 times 16.2 16.5

More than 4 times

9.7 11.2

No answer 0.3 --

Analysis – slight increase in the more frequent use categories (3-4 times & >4 times) in 2008

Question 3 My total experience using FAEIS is?

(N=285)

2008

Question 3 My total experience using FAEIS is?



(N=285)

More than 3 years

45 49.5

1 to 3 years 31 23.2

Less than 1 year

22 24.9

No answer 2 2.5

Analysis –greater number of “experienced” users (> 3 yrs) and greater number of “less experienced” users (<1 yr) in 2008 compared to 2007.

Question 4My professional position is best described as?

(N=285)

2008

Question 4My professional position is best described as?



Staff 49 46.7

Faculty 18 7.7

Department Head 6 19.3

College Associate/Assistant Dean

16 15.4

College Dean or VP 5 7.7

University level Administration

5 2.5

Government -- 0.7

No answer 1 --

Analysis – More administrators (non-staff/faculty) participated in the survey in 2008 (44.9%) vs 2007 (32%).

Question 5My level of adoption (use) of FAEIS is best

described as? (N=285)

2008

Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)



1=unaware 7.5 10.2

2=aware of it but do not use it 42.2 40.0

3=Use it but not sure of long term use

26.9 27.0

4=Use it and it is integral to my job 15.0 15.8

5=Integral to my job and expect long term use

6.1 6.7

6=So important as to never be without FAEIS

1.0 0.4

Mean Score 2.73 2.70

Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis)

2007Mean

2008Mean

Data Entry Users Only

2.49(N=159)

2.30(N=152)

All Other Users 3.16(N=117)

3.24(N=125)

Total 2.78(N=276)

2.73(N=277)

1=unaware

2=aware of it but do not use it

3=Use it but not sure of long term use

4=Use it and it is integral to my job

5=Integral to my job and expect long term use

6=So important as to never be without FAEIS

Question 6Please rate the usefulness of the data resources in FAEIS (for

student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.). (1 = not useful, 5 = very useful)

2008

Question 6Please rate the usefulness of the data resources in FAEIS

(for student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.).

Score



1, Not useful 7 7

2 12 13

3 28 28

4 29 31

5, Very useful 24 21

Mean score 3.50 3.47

(“no answer” and “not used” responses eliminated)

Analysis – no significant changes between ‘07 & ‘08

Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment, faculty hiring and

fundraising). (1 = not useful, 5 = very useful)

2008

Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment,

faculty hiring and fundraising). (1 = not useful, 5 = very useful)

Score



1, Not useful 9 8

2 21 20

3 36 34

4 19 26

5, Very useful 15 12


(“no answer” and “not used” responses eliminated)

Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 34%, ’08 = 38%). Lower percentage of “not used” answer in 2008 (’07 = 43%; ’08 = 36%).

Question 8Describe your level of satisfaction with FAEIS components. (i.e., newsletter help desk,

data entry, report builder, instructions) 1=not satisfied; 5=very satisfied

2008

Question 8 Describe your level of satisfaction with FAEIS components. (i.e., newsletter help

desk, data entry, report builder, instructions) 1=not satisfied; 5=very satisfied

Score



1, Not satisfied 6 5

2 9 7

3 33 33

4 32 37

5, Very satisfied 20 18


(“Unknown,” “no answer” and “not used” responses eliminated)

Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 52%, ’08 = 55%). Similar percentage of “not used” answer (’07 = 23%; ’08 = 23.7%).

2008 Annual FAEIS Evaluation Did the type of involvement (Q1) have a significant relationship with ratings of: Frequency of use (Q2) Experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)

Respondents listing “data entry” as their only level of involvement were compared to all other levels of involvement. “Not used” and “no answer” responses for Q6, Q7 and Q8 were removed from the analysis.

There were significant differences for “Frequency of Use (Q2),” “Level of adoption” (Q5) and “Usefulness of data” (Q6), and Usefulness of FAEIS to your institution (Q7) and “Satisfaction with FAEIS components” (Q8) between the two groups. There was not a significant difference between the means of the two groups for Q3. Respondents listing “data entry” as their only level of involvement had significantly lower means for Q2, Q5, Q6, Q7 and Q8 than the other group.

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Adoption of FAEIS (Q5)

Yes, there is a positive correlation between “Frequency of use” (Q2) and “Adoption of FAEIS” (Q5). As the frequency of use increases, so does the rating of the adoption of the FAEIS.

F=45.0 (P<0.05)

Key:How often have you used FAEIS in the past 12 months?

1=Not at all2=1-2 times3=3-4 times4=more than 4 times

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of data (Q6)

Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of data” (Q6). As the frequency of use increases, so does the rating of the usefulness of the data.

F=58.2 (P<0.05)



2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of FAEIS (Q7)

Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of FAEIS” (Q7). As the frequency of use increases, so does the rating of the usefulness of FAEIS.

F=25.2; P<0.05



2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Satisfaction with FAEIS components (Q8)

Yes, there is a positive correlation between “Frequency of use” (Q2) and “Satisfaction with FAEIS Components” (Q8). As the frequency of use increases, so does the rating of the satisfaction with FAEIS components.

F=26.1;P<0.05



2008 Annual FAEIS Evaluation

Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Level of adoption (Q5)

For Level of Adoption (Q5), the “Less than One Year of Experience with FAEIS” group was significantly lower than the other two “level of experience” groups. This seems to reason – the less experience with FAEIS, the less the level of adoption of FAEIS.

2008 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS to your institution (Q7) Satisfaction with FAEIS components (Q8)

There was not a significant difference between the three groups for “Usefulness of data” (Q6), “Usefulness of FAEIS to their Institution” (Q7) and “Satisfaction of FAEIS Components” (Q8) .

2008 Annual FAEIS Evaluation Did the professional position (Q4) have a significant relationship with ratings of : Frequency of use (Q2) Years of experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)

The only significant difference was with the “government” group for Frequency of use (Q2) and Level of adoption (Q5). This group had higher means than the other groups for both questions.

2008 Annual FAEIS Evaluation Did the level of adoption (Q5) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8)

Groups responding “unaware,” & “aware but do not use” to Q5 were compared to the other response groups for Q5 - “Use it, but not sure of long term use;” “Use it and it is integral to my job;” “Integral to my job and expect long term use;” & “So important as to never be without FAEIS.”

There were significant differences for all three of the questions above between the two groups. The means of the “unaware,” & “aware but do not use” groups were always lower for each of the questions when compared to the other group.


Responses by non data entry users only 2007 Mean

2008 Mean

“My level of adoption (use) of FAEIS is best described as:” 3.16 3.24

SCALE:1=unaware2=aware of it but do not use it3=Use it but not sure of long term use4=Use it and it is integral to my job5=Integral to my job and expect long term use6=So important as to never be without FAEIS


Responses by all users 2007 Mean

2008 Mean

Rate the usefulness of the data resources in FAEIS. Scale: 1=Not useful, 5=Very useful.

3.50 3.47

Rate the usefulness of FAEIS at your institution for internal applications. Scale: 1=Not useful, 5=Very useful.

3.09 3.13

Describe your level of satisfaction with FAEIS components. Scale: 1=Not satisfied, 5=Very satisfied.

3.50 3.56

2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose...

Documents

Transcript of 2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose...