Technical College System of Georgia Office of Adult Education November 13, 2013.

Becoming a GALIS Data Detective

Technical College System of GeorgiaOffice of Adult Education

November 13, 2013

2

Technical Housekeeping On the day of the teleconference, call 1-866-590-5055 and

enter access code 8019870#

Please mute your phone line to minimize background

noise.

You can access presentation materials on the GALIS

“Help” page under Technically Speaking

Technical Difficulties? Email [email protected].

When asking questions, please state your name, program,

and location.

Stay tuned at the end for a link to an online evaluation form

and information about the next ESL Technically Speaking.

mailto:[email protected]

3

Teleconference Overview Introduction – Charita Boles, GPS Coordinator Presenters

Karen Kirchler – Vice President of Adult Education, West Georgia Technical College

Stephanie Rooks – Dean of Adult Education, Gwinnett Technical College

Brent Stubbs – Dean of Adult Education, Savannah Technical College

Questions of Presenters Sharing from Others Closing Remarks – Charita Boles

4

GALIS Data Detective

The data is used to monitor compliance with NRS and state requirements

Keep an open mind when reviewing data Use the data to make program decision Engaging staff in data is a powerful motivator

6

Identifying and Providing Support to Low Performing Sites/Instructors

Identifying issues requires that:you have a definition of what is “low

performing”you understand that “low performing” can be

relative to the type of site or students served.you have a baseline for comparison.

7


We look at a handful of measures that we find helpful:Attrition 1 – The number of students lost

between the point of intake (orientation/assessment) and the start of classes.

Attrition 2 – The number of students lost between entering class and achieving enrollment status.

8


We look at a handful of measures that we find helpful:Retention: The number of students retained to

a point of post-test eligibility (30, 40, 60 hours depending on EFL).

Level CompletionsPost-test completion rate

9


Attrition 1- Number of students lost between intake and entering classImportant to look at as a whole, in terms of the

program.Relevant to types of sites or environmentsRelevant to groups (ESL, ABE)Not necessarily helpful at the level of evaluating

individual instructors.

10


Attrition 2 – Students lost between entering class and enrollment.Important to look at as a whole, in terms of the

program.Relevant to types of sites or environmentsRelevant to groups (ESL, ABE)Extremely important when evaluating

instructors!

11


Retention – Students retained to a point of post-test eligibility.Important to look at as a whole, in terms of the

program.Relevant to types of sites or environmentsRelevant to groups (ESL, ABE)Extremely important when evaluating

instructors!

12


Report Cards:

Attrition 1 Attrition 2 Retention LC Overall PT LC Overall Teacha, Ima N/A 14% 91% 86.54% 95.74% SDA 6% 16% 63% 47% 81% Main Campuses 6% 14% 65% 51% 82% Community Sites* 7% 19% 57% 41% 77% Institutional N/A 25% 54% 35% 77% ESL** 5% 12% 51% 31% 65%

We don’t measure this at the individual

instructor level.

This analysis of all instructors is provided to the Lead Teachers and is used to inform their efforts in

program improvement.

On all other measures we look at how the individual instructor

compares to the overall program numbers. We look for outliers.

13


Report Cards:

Attrition 1 Attrition 2 Retention LC Overall PT LC Overall Teacha, Ima N/A 14% 91% 86.54% 95.74% SDA 6% 16% 63% 47% 81% Main Campuses 6% 14% 65% 51% 82% Community Sites* 7% 19% 57% 41% 77% Institutional N/A 25% 54% 35% 77% ESL** 5% 12% 51% 31% 65%

Ima is an outlier. She’s an exceptional instructor. What to do with Ima besides try and clone her? Have her do some staff development

sessions…let new instructors observe her in action…try and identify what she does that works and replicate it!

14


Report Cards:

Attrition 1 Attrition 2 Retention LC Overall PT LC Overall Goodwyn, Nada N/A 24% 42% 47% 66% SDA 6% 16% 63% 47% 81% Main Campuses 6% 14% 65% 51% 82% Community Sites* 7% 19% 57% 41% 77% Institutional N/A 25% 54% 35% 77% ESL** 5% 12% 51% 31% 65%

Nada is also an outlier. We are not enchanted with her performance. However, she is teaching at a community site and there might be some

issues with the environment that could affect attrition and retention. But, her post-test level completion rate is way out of range as well. Nada’s continued

employment with our agency is in question.

15


Report Cards:

Attrition 1 Attrition 2 Retention LC Overall PT LC Overall Middle, Malcolm N/A 31% 82% 74% 85% SDA 6% 16% 63% 47% 81% Main Campuses 6% 14% 65% 51% 82% Community Sites* 7% 19% 57% 41% 77% Institutional N/A 25% 54% 35% 77% ESL** 5% 12% 51% 31% 65%

So what’s up with Malcolm? He’s an outlier in many respects. His performance is really good…with the students he doesn’t lose in the first

week! Why is his Attrition 2 measure so high? What can we do to mediate this issue and keep more students in Malcolm’s class?

16


Things to consider:Rates that seem out of range from the baseline.What other factors are affecting the outcomes you

are seeing?If the post-test completion rate is out of line, it’s

wise to break it out by EFL to try and pinpoint the instructor’s area of weakness.

Report cards with program comparisons are a good way to keep instructors aware that there’s more to the game than just LC rates.

17


Nuts and Bolts: While we may give instructors individual report cards, we look

at the data in a more global format via spreadsheets and charts.

It requires a little time, and compiling all of these measures is best done only about twice per year.

It’s accomplished most easily if you are familiar with Excel and Access, though most of this can be done with just Excel.

It allows us to look at the program in different ways in different breakdowns.

18


How we do it:1. Export AL137 – All Students and Status by Instructor to Excel

(This report is used because we need the last attended date and the CRN for each student’s attendance. Students will be duplicated if they attended more than one class—but it gives a snapshot of teachers’ individual attrition.)

2. Clean up the data – Paste the instructor’s name into the far right column. This allows for sorting. If you have instructors teaching multiple classes (particularly if they are different types of classes or sites) you’ll need to add an identifier so you can distinguish between the classes.

If you have a large program and you have Lead Teachers supporting part-timers, you may find some benefit in creating an Access table with all of the identifying info you wish to attach to the class or CRN (Lead, location, etc.) You can export the Excel sheet into Access and it creates a little more flexibility in manipulating the data.

19


How we do it:3. Sort and filter the spreadsheet:

○ For “Attrition 1” filter out any who have more than 3.5 hours. The group that remains gets divided by the total # of students.

○ For “Attrition 2” eliminate those with less than 3.5 hours. Filter the group that remains to determine the number with less than 12 hours. Divide this by the total # of students.

○ For “Retention” eliminate those with less than 12 hours. Sort by EFL and then filter to appropriate hour level = to post-test eligibility. Divide these numbers by the total # of students.

4. Export and marry the Table 4 and 4b rates to your spreadsheet by instructor. (For instructors teaching at multiple sites or different types of classes, this requires more manipulation.)

20


Example Instructor A:Total number of students in AL 137 = 35 Total number of students in Table 4 = 25

Attrition 1 - Students with less than 3.5 hrs = 4 (Eliminate – they never came to class)

Attrition 2 - Students with less than 12 hours = 6 (6/31 =19%)(We eliminated 4 from the original 35 to get our base of 31 students.)

Retention - Students who make it from enrollment to post-test eligibility = 14 (14/25 = 56%)

(Use your enrollment number as your base of students.)

22

Retention: The number or % of students who remain long enough to be post-tested and

make a gain.

Money

Gains

Retention

Hours

23

Breaking It Down…

Many small programs-Not one large one12 sites15 ABE & ESL programs203 classes

Retention by Lead Instructor Retention by site Retention by class/instructor

The Best Tool?

24

….Classroom Sign-in Sheet

Submitted and entered into GALIS weekly Wednesdays-each sheet reviewed and %

attended/absent recorded Broken down by: Each Lead Instructors’

sites by class Spreadsheets sent to Leads – any class

<70% attendance is flagged

25

Class Title Instructor Site Registered Attended %

3F31E7A2-1E Bruce, JulieGTC ESL Afternoon M/W (31) 8 6 75.0%

3F31E7A5-1E Meheux, QueenGTC ESL Afternoon M/W (31) 36 25 69.4%

3F31E7A3-1E Eaton, LindaGTC ESL Afternoon M/W (31) 13 9 69.2%

3F31E7A4-1E Goode, AnnaGTC ESL Afternoon M/W (31) 17 14 82.4%

3F31E7A6-1E Ortiz, RubenGTC ESL Afternoon M/W (31) 32 24 75.0%

3F31E7A1-1E Patel, SangitaGTC ESL Afternoon M/W (31) 30 20 66.7%

3F32E7M2-1E Nayar, Aarti Catedral De Fe (32) 10 8 80.0%3F32E7M3-1E Houston, Leney Catedral De Fe (32) 21 19 90.5%3F32E7M5-1E Meheux, Queen Catedral De Fe (32) 31 21 67.7%3F32E7M4-1E Goode, Anna Catedral De Fe (32) 15 10 66.7%3F32E7M1-1E Royal, Charlene Catedral De Fe (32) 17 12 70.6%3F32E7M1L1E Holland, Catherine Catedral De Fe (32) 17 12 70.6%3F32E7M1-2E Baker, Charlotte Catedral De Fe (32) 17 11 64.7%3F32E7M4-2E Patel, Sangita Catedral De Fe (32) 15 11 73.3%

3F34E8MML1E Royal, CharlenePinckneyville Park CRC (34) 27 20 74.1%

3F34E8M5-1E Meheux, QueenPinckneyville Park CRC (34) 16 8 50.0%

3F34E8M3-1E Houston, LeneyPinckneyville Park CRC (34) 8 4 50.0%

3F34E8M4-1E Patel, SangitaPinckneyville Park CRC (34) 22 15 68.2%

3F35E8A2-1E Bruce, JulieGTC ESL Afternoon T/R (35) 15 12 80.0%

3F35E8A6-1E Patel, SangitaGTC ESL Afternoon T/R (35) 22 17 77.3%

3F35E8A5-1E Houston, LeneyGTC ESL Afternoon T/R (35) 37 34 91.9%

3F35E8A4-1E Nayar, AartiGTC ESL Afternoon T/R (35) 18 9 50.0%

3F35E8A3-1E Butterworth, CarolGTC ESL Afternoon T/R (35) 10 7 70.0%

3F35E8A1-1E Eaton, LindaGTC ESL Afternoon T/R (35) 19 18 94.7%

26

Instructor

Leads – Should know about every class Confer with instructors

Review attendanceObserve class

○ SEP followed?Students called?Follow-Up Form Completed?

27

Retention OutcomeNumber and Percent Retained

Year # Enrolled# completed

level% completed

level

FY13 4527 2231 49.3%

FY12 4961 1557 31.4%

FY11 4528 842 18.6%

29

Analyzing Your Programwith Data

Start with the end in the mind – create processes that match your goals

Set data benchmarks for attrition and enrollment

If you look at it daily, it must be entered daily Contact Hours Tell the Story

30

Backward Engineering Start with the end in mind Our goals: 2,700 enrolled, 60% completions,

90% GED completion, 10% attrition Do my processes match these goals? Am

enrolling enough new students per month to reach my enrollment goal?

At 80% effectiveness, am I giving enough post-tests per month?

Result: Targeted Recruitment

31

Attrition & Enrollment We set monthly enrollment goals in:

NRS Enrollment (A)State Enrollment (B)“In GALIS” (C)

AL077 report can be manipulated to tell the story of attrition

Attrition = C-B/C and B-A/A

32

Attrition is Important C-B/C could mean

We are entering students into GALIS we should not be (wasting time)

We are not capturing attendance for some students (process or responsibility is undefined)

B/A-A could meanOur success at retaining studentsA breakdown in the linkage between

Orientation and students starting class

33

Is Your Data Good Daily? Attendance is entered daily If you need to make real-time decisions,

you must have real-time information 24 hour completion entry goal How can you make decisions about

enrollment, teacher effectiveness, post-testing, etc., if you do not have real-time information?

34

Being in front of students

Contact hours tells the entire story I budget for contact hours We are on track to increase our contact

hours by 30% this year. Why? Students in class, post tests happen,

more opportunities for student success

35

Being in front of students Over the last 3 years, what was your

average contact hour per student? Ours: 66.8 Our goal: 90 Which teacher had the most contact with

students? Did it correlate to better success? Students vote with their feet.

36

Questions for the Presenters

?Please say your name, program and location before asking your question

37

Sharing of Ideas from other Adult

Education Professionals

38

Contact Information

Charita Boles, GPS CoordinatorTCSG, OAE1800 Century Place NE, Suite 300Atlanta, GA 30345-4304Office: (404) [email protected]

Ms. Karen Kirchler, Vice President of Adult Education West Georgia Technical College401 Adamson Square Carrollton, GA 30117 Office: 678-664-0507 [email protected]

Ms. Stephanie Rooks, Dean of Adult Education Gwinnett Technical College5150 Sugarloaf Parkway Lawrenceville, GA 30043 Office: (678) 226-6662 [email protected]

Mr. Brent Stubbs, Dean of Adult Education Savannah Technical College5717 White Bluff Road Savannah, GA 31405 Office: (912) 443-5446 [email protected]





39

Thank you for your participation!

Please complete an evaluation of this session at http://surveymonkey.com/s/TechnicallySpeakingDetective

Next Program Administration Technically Speaking:

Getting to Know your Staff through

Visitation and Monitoring

January 22, 2014 2:00-3:30 pm

http://surveymonkey.com/s/TechnicallySpeakingDetective

http://surveymonkey.com/s/TechnicallySpeakingDetective

Technical College System of Georgia Office of Adult Education November 13, 2013.

Documents

Transcript of Technical College System of Georgia Office of Adult Education November 13, 2013.