Flynn Evaluation Smart Board

download Flynn Evaluation Smart Board

of 35

Transcript of Flynn Evaluation Smart Board

  • 8/4/2019 Flynn Evaluation Smart Board

    1/35

    Evaluation of the

    Smarboard Program atAISBED Tech 505-4172

    Daniel Flynn

    5/10/2011

  • 8/4/2019 Flynn Evaluation Smart Board

    2/35

    2

    ContentsReflection ........................................................................................................................................ 3

    Executive Summary ........................................................................................................................ 4

    Purpose of Evaluation ..................................................................................................................... 5

    Central Questions ........................................................................................................................ 5Impact of the Results ................................................................................................................... 6

    Background Information ................................................................................................................. 7

    School .......................................................................................................................................... 7

    Smartboard Program .................................................................................................................... 7

    Stakeholders ................................................................................................................................ 9

    Attitudes of Technology and the Smartboard Program ............................................................. 10

    Description of Evaluation Design ................................................................................................. 10

    Data Collection Methods ........................................................................................................... 10

    System of Data Collection ......................................................................................................... 11

    Results ........................................................................................................................................... 12

    Student Results .......................................................................................................................... 12

    Teacher Survey Results ............................................................................................................. 14

    Results from Administration observations ................................................................................ 18

    Administration questionnaire .................................................................................................... 18

    Discussion of Results ................................................................................................................ 19

    Recommendations ......................................................................................................................... 21

    Immediate Conclusions and Recommendations ........................................................................ 22

    Long Range Planning ................................................................................................................ 22

    Evaluation Insights .................................................................................................................... 23

    References ..................................................................................................................................... 24

    Appendixes .................................................................................................................................... 25

    Appendix A ............................................................................................................................... 25

    Appendix B................................................................................................................................ 27

    Appendix C................................................................................................................................ 29

    Appendix D ............................................................................................................................... 32

    Appendix E ................................................................................................................................ 33

    Appendix F ................................................................................................................................ 34

  • 8/4/2019 Flynn Evaluation Smart Board

    3/35

    3

    ReflectionIn this course I have already applied methods and techniques of evaluation that will

    assist me in developing a successful career in teaching. The two most important aspects of thecourse are formative and summative evaluations. Formative assessment has been a greatbenefit for me as I am constantly called upon to evaluate software programs that teachers areasking about or wanting to implement. The formative evaluation project for the communityservice presentation in this course was a good example of what I do every day, however, theproject gave me a better perspective on how to refine my formative skills and methods.Summative assessment on the other hand has been quite a challenge as it requires morethoughtful planning and analysis. Summative assessments are the basis for full evaluations andto understand what a program may need to improve.

    Problem analysis was something that I have used all throughout my professional career.When a program is in doubt or trouble it is a problem and at the very least there needs to be ananalysis to try and solve the issue before the program gets off track. I have used this method

    recently in our one to one laptop program in which students were not effetely using theirbattery life though out the day. I sent out a student survey that pinpointed why the battery wasnot used correctly and had solved the problem based on the line of questioning that was posedto the students.

    In my professional career I have conducted a large amount of criterion referencedevaluations of programs related to technology. For instance, in the recently launched one toone program at our school I conducted a mini evaluation based on students perceptions of laptops and their use in the classroom as a tool. In this evaluation there were criterion basedquestioning thought-out each survey given to students and teachers. The surveys were based

    off of goals and geared to gauge students and teachers opinions of the goals that were set forthin the EPD.

    Evaluation is all about long term planning as they are deigned to gauge where a programis and where it is going. The group summative project was a prime example of long termplanning and continuance of modification of the website for users. The website (portal) usedmany different educational tools for its users to interact with and learn. The portal to that pointhad many sites that were not appropriate for its users as they were distracting users from thefull potential of the portal. The organization can use the summative evaluation as a basis for allfuture potential new websites that can be added to the portal. In my professional life long term

    planning can assist me in developing a proper program based on planning for the final product.The evaluation in the end is not the only aspect to long term planning, as the program must firstgo through a proper research stage, design, planning and creating phase.

    Evaluations are a key aspect to any program and its long term success. Learning fromthe results is where the program can go for an organization; otherwise the program may notsucceed according to plan. The lessons from this course will last me as long as I remain in theprofessional world.

  • 8/4/2019 Flynn Evaluation Smart Board

    4/35

    4

    Executive SummaryThe American International School of Bucharest has had a nearly complete Smartboard

    program for the past two years. The purpose of the Smartboard program was to give teachersan extra tool to use for teaching students. More specifically it was given to teachers to enablethem to have an interactive board that would enhance student learning and student interactionwith technology in the classroom. The Smartboard program started four years ago with theaddition of a new IT Director and an ambitious learning community (the board, parents andteachers). Most teachers were excited by the addition of the Smartboard program, however,the program got off to a rocky start as boards were not distributed to every teacher at once,rather over a gradual phase over three years. This distribution scheme caused somecontroversy amongst the staff from its inception. Since the inception almost every classroomhas been provided with a board and teachers have received training two years ago, however,this is a high turnover school so some teachers have not received adequate training.

    The evaluation occurred because some of the administrators were concerned about

    how the Smartboard was used in the classroom. The administration has spent an enormoussum of capital in the program and expects a return in quality interactive education for theirstudents. Even though there is no way to connect dollar amount with outcomes, theadministration does expect teachers to use the boards regularly. This evaluation was set up todetermine the use of the Smartboard for both teachers and students. Furthermore theadministration wants to know how training and attitudes of teachers are affecting the use of theSmartboard in the classroom.

    The surveys, questionnaires and observations that occurred yielded results thatindicated that there needs to be more training for teachers and the possibility of training

    students. The administrator who observed teachers in action didnt feel t hat teachers wereusing it for more than projection and movies. However, the results from teachers are studentsshowed a bit more than just using the board for projection and movies.

    The following evaluation will cover the history of the board, the evaluation methodsused, the results and recommendations based on the results.

  • 8/4/2019 Flynn Evaluation Smart Board

    5/35

    5

    Purpose of EvaluationThe purpose of the evaluation is to make certain the investment of Smartboards in the

    classroom is being utilized to its fullest capabilities. The Smartboards have been phased in overa period of four years and has met some resistance and some animosity amongst staff members. However, within the last year this has cleared up as most teachers have received aSmartboard. The school has invested in professional development time in conductingSmartboard trainings for teachers, especially in 2008 when the program started distributingSmartboards to teachers classrooms. The entire staff was given Smartboard training; however,the teachers who were part of the first roll out program were required to go to two additionalone hour courses. The school has also experienced a large turnover rate since the inception of the Smartboard program since 2008 (this is somewhat due to the nature of internationalschools). The school would also like to understand what attitudes both teachers and studentshave about the Smartboards.

    Central QuestionsThe following central questions were used to set up at the beginning of the evaluation

    process as it was agreed upon with the administration

    1. Do teachers use the Smartboard for use of educational use in the classroom thatenhances students learning and understanding of course topics?

    The first purpose of this project is to gauge if teachers are using the Smartboards asintended, rather than just use it for watching film clips and so forth.

    2. Do teachers use a new feature that they have not used before in Smartboard?For some teachers they are more adept at using a Smartboard so for them this might bea challenge, however, for the majority of users this will be a challenge and will show astep forward in their progress as Smartboard users.

    3. Do students are to use the Smartboard rather than just the teacher?

    The Principal is concerned about this as his observations in classrooms often shows nostudent use of the Smartboards.

    4. Has professional development for Smartboard instruction two years ago been carried

    out in classroom instruction?

    The principal mentioned that he wants to see if the trainings provided nearly two yearsago have paid off or would we need to have more training in the near future.

    5. How do teachers feel about using the Smartboard and how does it fit in within theirprogram (curriculum)?

  • 8/4/2019 Flynn Evaluation Smart Board

    6/35

    6

    After a discussion with the principal and head of IT Department they are concernedabout the impact of the Smartboard program in the secondary school and if it worth themoney and could it lead to more investment in technology to enhance the Smartboarduse in the classroom

    Impact of the ResultsThe results of the evaluation has impact for almost every group in the American

    International School of Bucharest (here in known as AISB) community. The groups that will beimpacted the most are the teachers and administration. The teachers will be charged with usingthe Smartboard as an interactive tool in the classroom. The administration when presentedwith the results must analyze what will be appropriate steps for the next stage of theSmartboard program. The following chart will help break down the impact implications of theSmartboard program.

    Parents Parents Pay tuition to the school and expecttechnology to be used in the classroom(school climate survey of 2009). Thecurrent parent population will be impactedin knowing that there will be a resolve tothe success of the evaluation orexpectations of how the program willimprove.

    Administration Administrators are dependent on the use of technology in order to sell tuition spots inthe school, pass accreditation and satisfy IB

    World School status. The impact of theevaluation may cause a shift in policychange in Smartboard use.

    IT department The IT department is charged withmaintenance and conducting training withthe boards. The IT department is alsovested in the success of the program as thedepartment was the initial instigators of theboards in each classroom.

    Teachers Teachers are on the frontline in using theboards on a daily basis. The intent is to use

    the board on a regular basis for interactiveuse by teachers and students. The impactof the evaluation and decisions theadministration makes will impact the wayteachers use the board as a part of theirteaching.

    Students Students are the customers served by theuse of the boards in the classroom.

  • 8/4/2019 Flynn Evaluation Smart Board

    7/35

    7

    Students are actively engaged learners andthe use of the boards will assist them in thelearning process as this is the intent of theboards.

    Background Information

    SchoolThe school is an American based educational system, with an International

    Baccalaureate program. The school is located in Bucharest, Romania, with a schoolpopulation of 702 students from early childhood to twelfth grade. There are 53classroom teachers and 15 specialists that serve students and their educational needs. Theschool serves students of 49 countries, including Americans, Canadians, and Romanians

    amongst others the school is tech heavy with an annual technology budget of around$500,000 USD. Each teacher is given a laptop and is equipped with SmartboardNotebook software and the drivers needed to run the Smartboard.

    Smartboard ProgramThe Smartboard program began as a trail program four years ago and now is in

    every classroom but three in the school. Prior to the Smartboard program at the school,there were 10 Cleverboards that were bought off of another school in the region.Apparently the boards were bought for double the retail price, and the deal wascompleted with no authorization of the school board. In 2007 the school hired a new IT

    director and the Cleverboard program was scrapped due to faulty equipment that was notcovered by any warranty. In addition, the Cleverboards were not used regularly due tothe lack of training, and sharing of two projectors for the ten boards. The biggestcomplaint from teachers was not having regular use of projectors, which ended updiscouraging teachers from learning or using the Cleverboard on a regular basis for theirclass lessons. The new IT director at that point wanted to implement a new interactivewhite board program that would encompass a strong warranty, provided local service(given that the school is located in Romania) and give teachers the ability to use theboards with confidence.

    The decision was made in late 2007 with a small committee of teachers,administrators and the boards approval of a trail year for the purchase of Smartboards.The committee and IT director came up with a proposal for the board that includedexpectations and spending allocations throughout the first year and subsequent years if the program was trialed successfully. Originally, the program was approved for 7classrooms, 3 in the elementary school and the rest in the middle and high school. In thesecondary school one board was given to members of the math, science, English and

  • 8/4/2019 Flynn Evaluation Smart Board

    8/35

    8

    humanities department and was open to any member using the board as they saw fit. Thegoals and expectations of the program in its initial set up in the proposal to the boardwere as follows:

    Teachers must try to design lessons to ensure that students have the

    opportunity to use the interactive features of the Smartboard.

    Teachers must attend mandatory initial training sessions (at least two one hoursessions)

    The rest of the original goals and objectives will be listed in Appendix A

    The original rationale and standards for the Smartboard program was relativelysimple, as the intention was to develop a pioneer group to deliver a strong Smartboardprogram for the future of the school. The first set of people who wanted a Smartboard,were required to attend two one hour training sessions to introduce the basics of how theSmartboard worked and what features can be used in the Smartboard (and notebook

    software). Each teacher was then expected to learn, develop lessons, share lessons andcarry on training for subsequent years. The initial program of 10 Smartboard wasdeemed successful as the small group that was chosen abided to most of the exceptions of the Smartboard according to the IT director. However, there were issues that arosethroughout the initial phase of the Smartboard program, mostly the large turnover rate atthe school, due to the nature of international school teachers. Since the initial programstarted only one teacher remains at the school and has carried out the originalexpectations set out in 2007.

    The Smartboard program expanded at the start of school years 2008 and 2009 andwas completed in early 2010. All classrooms but two are outfitted with the inventory listthat listed in graph BI-1. When the program fully expanded in 2009 and early 2010 the

    decision was made by administration to conduct a full training session with each teacher(with Smartboard or not) will get 3 one hour sessions, two were in the pre-service week and one was a follow up session late March of 2010 to complete the training sessions.Teachers in these trainings were put into three different level groups, based on self assessed skill level (overall, not Smartboard knowledge) the three levels are beginner,intermediate and advanced learners. The first two sessions based on what level eachteacher put themselves into were focused on features of the Smartboard, SmartboardNotebook software, and non-Notebook software that can be made interactive, such asWord or a website. The third session which was a follow up session was designed tocreate a session of share and show for features that teachers use on a regular basis. Thegroups were divided into four sessions in order to create smaller groups and to assure that

    teachers got the most out of the session. Out of the final session teachers were requestedto learn at least one new feature in Smartboard every year and record this into therepository. The repository which was originally setup for the initial program is still inuse and is expected to be populated by teachers. The repository can be populated withmore than just features; the repository can include techniques, methods, websites withinformation for Smartboards, and so forth. Training since the last formal session inMarch of 2010 has been limited and revolves around individual training session orteacher led technology trainings throughout the school year.

  • 8/4/2019 Flynn Evaluation Smart Board

    9/35

    9

    Since then the program has grown and is now in 53 classrooms, along with this, eachclassroom had to be fitted with a projector and mounting of the Smartboard, the costs arelisted below:

    Item Units Cost Total CostsSmartboard (Wireless) 53 $2,000 $106,000Ceiling MountedProjector

    55 $1,400 $77,000

    VGA Splitter 55 $130 $7,150Cabling 55 $140 $7.700Ceiling Spider 55 $160 $8,800Mounting(Smartboard)

    53 $200 $10,600

    In house labor* $7,200Total $224,450*The labor costs are based on 15% time allocation of IT Department employee, based on his overall salary, per annual (he is a localRomania employee)

    The costs of the Smartboards have been steady over the past 4 years, so no priceshave changed and the figures listed above are accurate. Most of the cost was based onlabor and the items that surrounded the installation of the Smartboard, such as projectors,splitters and mounting equipment. The Smartboards were funded by school tuition andcash was paid for all materials. Cost for maintenance have not been accumulated as of yet because the Smartboards are under warranty until 2013 (for the first cycle) through2015 (for the last cycle).

    StakeholdersAs mentioned prior there are many different stakeholders that are involved in the

    Smartboard program. The list goes as follows:

    Stakeholder InvolvementIT Director The person who is in charge of funding and

    curriculum development for the Smartbaordprogram at AISB.

    Board Of Education The board makes key educational decisionsbased on the advice from the EducationalPolicy Committee and administration.

    Teachers Are the recipients of the Smartbaord and allthe training that is needed to operate theboards.

    Students Students are the members in which theeducational community serves. TheSmartboards are educational tools that aremeant to enhance and engage studentslearning.

  • 8/4/2019 Flynn Evaluation Smart Board

    10/35

    10

    IT Helpdesk Attendant He is the person in charge of conductingrepairs and maintenance of the boards

    Parents Parents pay for a quality educationalexperience at AISB and expect technology tobe implemented and used at the school.

    Attitudes of Technology and the Smartboard ProgramTeachers were not initially happy in the way the Smartboard program was set up

    by the IT director with the direction of the administration. In a school climate survey of 2009, it was stressed by teachers that the Smartboard program was not fairly distributedbased on teacher expertise of technology and seniority at the school. The survey seemedto blame the IT Department and the IT Director specifically for the misappropriation of the boards in classrooms. However, the fact was the division administration made thedecisions to implement the boards. The attitude carried over until the fall of 2010 when

    the school sent out the yearly climate survey, in which the boards were not evenmentioned on the results.

    Description of Evaluation Design

    The model that will be used with the Smartboard Program will be goal-basedmodel, as the primary goal is to measure the objectives set forth in the evaluation. Therewill be both qualitative and quantitative data that will be analyzed based on theobjectives. The following are the goals that are being evaluated:

    Teachers use the Smartboard for educational use in the classroom that enhancesstudents learning and understanding of course topics.

    Teachers use a new feature that they have not used before in Smartboard Students are to use the Smartboard (student center learning) rather than just the

    teacher Professional development for Smartboard instruction two years ago has been

    carried out in classroom instruction. How do teachers feel about using the Smartboard and how does it fit in with

    Data Collection MethodsThe evaluation was split up into two formats one for quantitative and one for

    qualitative evaluation. The first set of data to be collected is the student survey, whichconsisted of 11 questions that were geared to measure students use of Smartboards in theclassroom for educational purposes. The second set of quantitative data to be collected

  • 8/4/2019 Flynn Evaluation Smart Board

    11/35

    11

    was a faculty survey which consisted of 18 questions. The teacher survey was designed tomeasure teachers attitudes and training that was provided in the past. The secondary

    part of the survey is to gauge teachers needs of training in the future with the Smartboardprogram. This part of the data is cross referenced to prior data that of training comparedwith current attitudes and knowledge of Smartboard use in the classroom. A copy of thestudent survey has been included in Appendix B. A copy of the teacher survey has beenincluded in Appendix C.

    The qualitative part of the data collection consisted of observations of teachers byheads of departments, administration questioners, and one open question for teachers.This data is done by questioners and standardized documentation of observation sheets(see Appendix D) done by administration. Originally observations were supposed to bedone by head of departments, however, this was changed to the administration because of time constraints and the administration was completing classroom observations duringthe evaluation period (during the Smartboard observation period).

    The final data collection piece to assist in evaluating the Smartboard program is aquestionnaire given to the administration of the secondary school and the IT Director.The purpose of the questionnaire is to gauge an administration point of view on theeffectiveness of the Smartboard program. The questions for this questionnaire arelocated in appendix E.

    There are many data sources for this evaluation and can be quite confusing,however, it is important to understand the aspects of all parties involved in theSmartboard program. Gathering data and analyzing this against the goal model will assist

    AISB in creating further advances for training and service for the Smartboard program.

    System of Data CollectionThe student survey consisted of 11 questions and was sent out to select teachers in

    the secondary school. This was done by teachers willing to give up some class time tocomplete the survey. A target goal of 100 was set out and 85 responses were completedby students in the secondary school. The elementary school was not survived as the ITintegration specialist felt that this was not appropriate for his age groups.

    The teacher survey consisted of 19 questions and was sent out to the entire staff,

    which included teachers, specialist and classroom aides (from the elementary side only).The intent was to receive back at least 50% from the staff, however only 31% of teachersand aides responded to the survey. I also sent out the form again on a weekend day inwhich only 3 responses came back; however that brought the total amount of responsesfor the survey.

    The questionnaire was given out to the principal and the IT director, the both of them responded with answers.

  • 8/4/2019 Flynn Evaluation Smart Board

    12/35

  • 8/4/2019 Flynn Evaluation Smart Board

    13/35

    13

    Out of five (being the highest amount of use for a specific subject) Science course scoredquite high compared to the other courses in the core subjects. Science scored the highestbeing almost 1.7 points above the mean (3.12). Math on the other hand scored the lowestbeing nearly seven tenths below the mean in use of the Smartboard in the subject.

    Next question asked students how often they use the Smartboard (as in the student usesthe board for interactive use not just the teacher)

    Chart 3

    Answer Reponses Percent usedNever 10 12%Fewer than 5 Times 36 42%5-10 times 27 32%11 times or more 12 14%

    The Occurrence of data for never using the board is about five times or less is 54% andthe occurrence of the board being used 6 times or more is 46% of the student bodypopulation.

    Students also states that they use the boards themselves in the following classes

    Chart 4

    2.41

    3.88

    3.38

    2.811

    Maths Science History Language

    Classes board is used in most

  • 8/4/2019 Flynn Evaluation Smart Board

    14/35

    14

    Class Responses Percent of studentsMaths 17 20%Languages 10 12%Science 28 33%History 22 26%

    Electives 1 1%None 7 8%The data shows that students use the boards most often in their science and historyclasses.

    Students also ranked their self assessed level of expertise on the use of the Smartboard

    Chart 5

    Teacher Survey ResultsThere are two parts to the teacher survey, teachers use and knowledge of the Smartboard,Student centered learning and part 2 covered teacher attitudes with regard to the

    Smartboard and training.

    Part 1-Teacher use and knowledge of the Smartboard

    How often do teachers use the Smartboard in class?

    Chart 6

    0

    5

    10

    15

    20

    25

    30

    35

    No Knowledgeat all

    SomeKnowledge

    Average StrongKnowledge

    Expert level

    Students Level of Expertise

    Series 1

  • 8/4/2019 Flynn Evaluation Smart Board

    15/35

    15

    Teachers use of the Smartboard while during class lessons:

    Chart 7

    *People may have selected more than one checkbox for this question

    Do students or teachers use the Smartboard more often?

    8%4%

    32%

    8%

    48%

    Use of the Bords in ClassroomOnce in a while On occasion About Average Above Average Almost every Class

    0

    5

    10

    15

    20

    25

    Movies/FilmClips

    Projection SmartboardNotebooksoftware

    Interaction of other

    software,such as

    website, MSOffice

    Interactivetools of theSmartboard

    Nothing isused

    Series 1

    Series 1

  • 8/4/2019 Flynn Evaluation Smart Board

    16/35

    16

    Chart 8

    Part 2 Teacher Attitudes about Smartboard

    Did you receive previous training for Smartboard use?

    Chart 9

    Do you feel there is enough training for Smartboard use at our school?

    Chart 10

    0

    5

    10

    15

    20

    25

    I use the board morethen the students

    The usage time of the board is about

    50/50

    Mostly Student usethe Board

    Studnets never usethe board

    Who uses the Smartboard?

    Series 1

    48%

    40%

    12%

    Smartboard trainingAt our school At another school None

  • 8/4/2019 Flynn Evaluation Smart Board

    17/35

    17

    Was previous training adequate for the staff in your opinion?

    Chart 11

    Currently do you feel there is enough training for teachers in operating the Smartboard?

    Chart 12

    0%

    36%

    64%

    0%

    Continuing training for SmartboardYes, There is more than enough Yes, There is enough

    Not really, there is little to no training Not at all, there is not enough training

    No,notatall4%

    Not really20%

    Was Adequate76%

    Was morethen

    adequate0%

    Pervious Training

  • 8/4/2019 Flynn Evaluation Smart Board

    18/35

    18

    For complete results of the survey please follow the links below:

    Student survey http://bit.ly/ijClkH

    Teacher Survey http://bit.ly/mnjrPK

    Results from Administration observationsThe results from administration observing teachers using Smartboard are as follows:

    Teacher rooms visited: 15 out of this there were five that were using the Smartboards intheir lessons.

    The administration observed the following out of the five boards viewed:

    Three classes were using the board for only projection use of PowerPoints Two Classes used the boards for Smartboard Notebook software. Out of the classes none had students use the board Features that were reported used were nothing new, as features were the same

    from the training 2 years prior.

    Administration questionnaireThe administration was asked a serious of questions about the Smartboard program basedon their years here at AISB and not to anyone incident or specific case.

    The results are as follows:

    Question 1

    Not that I amaware of

    28%

    I would say everyonce in a while

    56%

    There is but I havenot attended

    12%

    There is noneed fortraining

    4%

    Current Training

    http://bit.ly/ijClkHhttp://bit.ly/ijClkHhttp://bit.ly/ijClkHhttp://bit.ly/mnjrPKhttp://bit.ly/mnjrPKhttp://bit.ly/mnjrPKhttp://bit.ly/mnjrPKhttp://bit.ly/ijClkH
  • 8/4/2019 Flynn Evaluation Smart Board

    19/35

    19

    Please describe your observations of the Smartboard for educational use in the classroomthat enhances students learning and understanding of course topics.

    Administrator 1

    Teachers who use the Smartboard effectively have redesigned their instructional deliveryby rethinking their pedagogy. Lesson plans have been reworked to allow and fosterenhanced opportunities for students to interact and explore with alternative forms of media and channels through the Smartboard. Students are engaged and challenged toview concepts from multiple perspectives. Multimedia content from a variety of sourceshave supported the students and teachers alike.

    Administrator 2

    I have primarily seen Smart Boards used for two purposes:

    1. Helping to illustrate and elucidate classroom lectures, discussions, andexplanations.

    2. For online quizzes where students provide answers to the teachers who enter themand students discover the correct answer.

    Question 2

    Have you observed students using the Smartboard in class on a consistent basis?

    Administrator 1

    I would not say in a consistent basis. The interaction depends largely on how the lessonhas been designed by the teachers. In many cases it still seems to be driven by theteachers. However, teachers are exploring ways to use applications like LanSchool toengage students to interact with the smartboard from their own positions .

    Administrator 2

    No, to be honest, I have seen students using the Smartboards on only a very occasionalbasis. I have normally seen the teacher using the Smartboard to illustrate points in alecture or class discussion or to do online quiz activities where the teacher takesresponses from the student and enters them.

    Then remaining answers will be located in Appendix H

    Discussion of ResultsThe end purpose of the Smartboard evaluation was to measure the use of the

    board in the classroom, from teachers to students. In addition, measurements indicate

  • 8/4/2019 Flynn Evaluation Smart Board

    20/35

    20

    teachers attitudes of the Smartboard and perhaps are affecting their use of the board inthe classroom and for instruction.

    The following is a breakdown of each goal/objective and how the results impactedthe goals.

    Teachers use the Smartboard for educational use in the classroom that enhances studentslearning and understanding of course topics

    Results have indicated in the administration questionnaire that teachers use theboards for the purposes other than the interactive purposes. One administrator indicatedthat the board is used for interactive purposes occasionally. This can be backed up withthe student survey in which student said that 35% of teachers use the board for moviesand projection only. This can also be indicated in the questioning of what the boardspurpose is in the classroom as 46% of students believe that board is mostly used for

    projection and movies. However, students also indicated that the boards were also used41% of the time for the Smartboard Notebook software, and the interaction tools of theSmartboard drivers. Teachers have indicated that the used the board mostly for movies,(80%) projection, (72%) Notebook software, (60%) and interaction with other softwareor devices, such as the web, or MS Office (56%) in their teaching practices.

    By indication of the surveys and the questions answered by the administration thisgoal has failed and needs attention.

    Teachers use a new feature that they have not used before in Smartboard

    The indication of the teacher observation report indicates that no teacher has usednew features in the Smartboard since last major training period two years ago.

    This goal has not been achieved and needs attention.

    Students are to use the Smartboard (student center learning) rather than just the teacher

    Student survey indicates that students use the board less than five times at AISB(54% of students indicated this). This can also be correlated with the results of what theboard is used for at school as indicated above students indicate that the board is used forprojection and movies (46%). Teachers have indicated that they use the board primarily

    for their use, such as projection, Notebook software, etc. 80% of teachers indicated thatthey use the board and not students in classroom Smartboard activities.

    The goal has not entirely failed however, it will need some work.

    Professional development for Smartboard instruction two years ago has been carried out in classroom instruction.

  • 8/4/2019 Flynn Evaluation Smart Board

    21/35

    21

    Most teachers feel that their use of the Smartboard is adequate (if not higher). This wasindicated through teacher acknowledgment of training either at AISB (48%) or anotherinstitution (40%) in total. However, teachers indicated that there is a lack of traininggoing on currently at the school with 64% believing that there is little to no trainingcurrently offered (given that two years ago was the last staff training). However, 68% of teachers have indicated that training form two years ago to now (or previous work place)is adequate to fully trained. The administration questionnaire shows that they believetraining has not been adequately provided throughout the Smartboard program. The mainconstraints that they indicate are time and resources to provide adequate training for allteachers and staff that have Smartboards.

    The goal needs attention as there are clear gaps in training and indication of training forteachers in the Smartboard program.

    How do teachers feel about using the Smartboard and how does it fit in with their

    program

    The teacher survey indicates that the teachers are happy with the boards being in theclassroom as 60% of teachers say that the board is a great addition to the classroom. 28%of teachers think that the board is good for their classroom, however, they have doubts.Only three teachers indicated that they rather not have the board in their classroom.Teachers have also indicated that they would like to receive more training during schoolhours. No teacher indicated that they didnt want continuing education with theSmartboard; rather the majority of teachers want training during school hours(professional development, in class coaching etc). As well teachers are somewhat

    satisfied with the assistance of the IT department with regards to aiding in Smartboardprogram if there is a technical difficulty (only 16% of teachers are completely satisfiedwith the assistance).

    The goal has passed as most teachers have a positive attitude for using the Smartboard ona regular basis

    Recommendations

    Based on the results of the surveys, classroom observations and the questioners formadministration the following is recommended to better enhance the Smartboard programat AISB:

  • 8/4/2019 Flynn Evaluation Smart Board

    22/35

    22

    Immediate Conclusions and RecommendationsContradiction of results

    There is a discrepancy in the results, as the administration feels that there is a clear gap inboards being used for more than projection and the teacher and student survey results.Teachers did for the most part admit that they use the boards for projection and movies,however 80% teachers state this is what they used the board for. The administration said thatthey have rarely seen the board for anything else but projection. This can be illustrated in theteachers (60%) said that they use the board for interactive purposes. Chart 8 goes into thebreakdown of the use of the board more in detail. Students said that they only see teachers usethe board for interactive purposes 35% (Chart 1) of the time for things other than movies orprojection. This might want to be cleared up during training or a staff meeting, as the results donot indicate consistent responses.

    More training for teachers

    Teachers have indicated their desire for more training as this was backed up by all fourdata collection pieces in the evaluation. Training should be during the school day andfocus on features of the program such as coaching or mini training sessions. This shouldbe implemented immediately and a long term plan should be conducted to address theissue of training for teachers who are in the Smartboard program.

    IT Smartboard assistant

    Most teachers gave mediocre praise for the assistance received from the IT office inregards to assistance to repair or troubleshoot issues with the Smartboard. The ITdepartment has one dedicated troubleshooter for the Smartboard program and shouldprovide better technical and customer service training for the employee. Fixing thiswould immediately booster teachers attitudes with the Smartboard program and helpsensure a good investment for the school.

    Long Range PlanningTeacher retraining of the Smartboard for the following:

    Once a year have training for teachers who are new to the school (up to 25%turnover rate) that covers expectations, features, and all software used with theSmartboard.

    Once year refresher training for teachers who are returning to school. Thistraining should be focused on expectations and sharing of Smartboard features.As well all updates to features should be discussed during this training period.

  • 8/4/2019 Flynn Evaluation Smart Board

    23/35

    23

    All training that is provided at the beginning of the year should be followed up by amidyear mini session in which expectations and features learned at the beginning of theyear are met. Teachers can have the opportunity to express concerns and progress at thismeeting.

    Student centered learning with the Smartboard

    This is tricky as student centered learning in not solely based on the Smartboard alone.The school should consider bringing in an expert at student centered learning (preferablysomeone with experience with Smartboards). This is an area that will take a while todevelop given that the turnover rate of teachers is high and time will need to be set upwell in advance for proper training.

    Student training of Smartboard use

    Students have a decent base of knowledge of how to use a Smartboard; however, there isa measurable gap between the students who dont know how to use the Smartboard tostudents who rate themselves as experts.

    Evaluation InsightsIf the evaluation was done over, the following would be considered:

    Survey questions would be designed so that the questions would tap specificknowledge of the user rather than just broad open ended questions.

    Less administration based questioning. Even though their insights andquestioning were useful it was not easy to get and cost me some time incompleting the evaluation effectively

    Teachers need more of an incentive to take the teacher survey. Only 25teachers/specialist/aides took the survey. I sent out a second request on aweekend to try and garner more responses however only three completed it.

    Observations of classrooms for the Smartboard program should have been doneby the evaluator of the Smartboard program rather than the administration.

    A teacher self evaluation form for Smartboard use for student centered learning,rather than just a couple of questions on a survey.

    One on one interview with teachers and students about the use of Smartboards inthe classroom as this can avoid the discrepancy of results that came out of theevaluation.

  • 8/4/2019 Flynn Evaluation Smart Board

    24/35

    24

    ReferencesBoulmetis, J., & Dutwin, P. (2005). The abcs of evaluation: Timeless techniques forprogram andproject managers (2nd ed.). Jossey-Bass

  • 8/4/2019 Flynn Evaluation Smart Board

    25/35

    25

    Appendixes

    Appendix AOriginal documentation of the Smartboard Program Expectations and goals

    Building in expectations and accountability:

    Expectations for teachers

    Teachers must attend mandatory initial training sessions (at least two one hoursessions)

    Teachers must attend follow up sessions as needed (planned with tech dept) Teachers must maximize the use of the Smartboard by using it as much as

    possible during instructional delivery. Teachers must maximize the use of the Smartboard by learning how to use as

    many of its features and applying them during instructional delivery. Teachers must try to design lessons around maximizing the application of the

    Smartboard. Teachers must try to design lessons to ensure that students have the opportunity to

    use the interactive features of the Smartboard. Teachers must share lesson resources, ideas and plans for Smartboard use with

    others. Teachers can offer Smartboard workshops to others as appropriate. Teachers must care for the equipment to maximize its life span. Teachers must switch off the projector when not in use. Teachers must ensure that all loose equipment and related peripherals are stored

    safely when not in use.

    Expectations for technology department

    Tech dept will provide support for all Smartboard technical issues. Tech dept will provide training on how to use the Smartboard. Tech dept will coordinate the collection and distribution of Smartboard resources

    and make them available to all. Tech dept will plan for, budget and order Smartboards as needed.

    Expectations for principals

    Principals will plan and provide for Smartboard training time and opportunities. Principals will monitor use by teachers. Principals will make recommendations for training and application. Principals will recommend moving smartboard resources as needed to maximize

    usage.

  • 8/4/2019 Flynn Evaluation Smart Board

    26/35

    26

    Principals will provide feedback on how successfully the Smartboards are beingused.

    Principals will make recommendations as needed to improve the application of Smartboards.

  • 8/4/2019 Flynn Evaluation Smart Board

    27/35

    27

    Appendix BThe following survey was created by: Flynn, Daniel, Student Survey, 2011

    https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0

    Have you been in a class where the teacher uses the Smartboard for movies only?

    Yes

    No

    How often do your teachers in your maths courses use the Smartboard for other than showingmovies?

    Not At all to almost every class

    How often do your teachers in your language courses use the Smartboard for other thanshowing movies?

    Not At all to almost every class

    How often do your teachers in your humanities/history courses use the Smartboard for otherthan showing movies?

    Not At all to almost every class

    How often do your teachers in your science courses use the Smartboard for other than showing

    movies?

    Not At all to almost every class

    The Smartbaod is designed and used for....

    Movies, presentations, and projection

    Movies, video clips

    For teachers to use the notebook software, and interact with the tools of theSmartboard

    For teachers and students to use the notebook software and interact with thetools of the Smartboard

    You are not sure what the Smartboard is used for

    Have you ever used the Smartboard in class?

    Yes

    https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0https://spreadsheets.google.com/viewform?hl=en&formkey=dHJFSmdIQzUzekRRWGF1TVlCQVFGeVE6MQ#gid=0
  • 8/4/2019 Flynn Evaluation Smart Board

    28/35

    28

    No

    Not Sure

    How often have you used the Smartboard?

    Never

    Less than 5 times

    More than 5 times but less than 10 times

    More than 11 times

    What classes have you used the Smartboard the most in?

    None

    Maths

    Languages

    Science

    History

    Electives

    How would you rate your knowledge and use of the Smartboard?

    No Knowledge at all to can use Smartboard all the time

    What grade are you in?

    Middle School

    High School

  • 8/4/2019 Flynn Evaluation Smart Board

    29/35

    29

    Appendix CThe Following survey is created by: Flynn, Daniel, Teacher Survey, 2011

    https://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifq

    How often do you use your Smartboard?

    Once in a while to almost every class

    For what purposes do you use the Smartboard?

    Movies/Film Clips

    Projection

    Smartboard Notebook Software

    Interaction of other software such as websites, MS Office, etc

    Interactive tools of the SmartboardNothing is used

    Please rate your expertise with the Smartboard.

    What do you teach?

    EC3-2nd Grade

    3-5 th Grade

    Mostly Middle School

    Mostly High School

    Specialist

    Do you use the Smartboard more often than students?

    Yes

    I use the Smartboard more than Students

    The usage of time is split between students and teacher 50/50

    Mostly Students use the Smartboard

    Never use the board at all

    https://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifqhttps://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifqhttps://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifqhttps://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifqhttps://spreadsheets0.google.com/spreadsheet/viewform?hl=en&formkey=dDNLLTlFTTZtZi11bDh2Nm94OW0wRVE6MQ&ifq
  • 8/4/2019 Flynn Evaluation Smart Board

    30/35

    30

    Do you feel that there is enough training for Smartboard use at our school?

    Yes, there is more than enough

    Yes, there is enough

    Not really, there is little to no training

    Not at all, there is not training is not applicable

    Have you ever asked a fellow teacher for assistance with using the Smartboard?

    Yes

    No

    Didnt know that others were offering assistance

    How important is the Smartboard to your classroom operations on a daily basis?

    Not important at all to could not live without it

    If we there were to be Smartboard Training, when would you want it to take place?

    If there is training provided would you be willing to come after school hours?

    Currently, do you feel there is enough training for teachers in operating the Smartboard?

    Not at all

    During school hours only

    After school hours

    Professional development days

    Pre-Service week

    In classroom coaching

    I dont think we need training at all

    Should Smartboard training be focused on ?

    Curricular development

    Technical aspects of the Smartboard

    Student centered learning with the Smartboard

  • 8/4/2019 Flynn Evaluation Smart Board

    31/35

    31

    All of the above

    None of the above

    Do you think that the Smartboard is a good addition to your classroom?

    Yes, all the time

    Yes, but I have my doubts

    No, because I am not sure how to use it

    No, I rather not have it

    Did you receive previous training for Smartboard use?

    Yes, at our school

    Yes, at another school

    No, never

    How do you feel the pervious trainings for Smartboard has helped you in your teaching?

    Not well at all to has assisted in using the Smartboard for my teaching

    Was previous training adequate for the staff in your opinion?

    No, not at all

    Not really

    Was adequate

    Was more than adequate

    Do you feel that the IT department assists you when there are technical difficulties with theSmartboard?

    Yes, all the time

    For the most part, yes

    Not really

    Poor service

    Not applicable

  • 8/4/2019 Flynn Evaluation Smart Board

    32/35

    32

    Appendix DHOD/Administrator Check sheet for observation of Smartboards:

    Teacher:Lesson:

    Date:

    Smartboard used: Yes No

    What was the Smartboard used for during the lesson? TV/Video other

    Were students involved in using the Smartboard?

    What new features of the Smartboards were used (that were not used in the past training twoyears ago)?

  • 8/4/2019 Flynn Evaluation Smart Board

    33/35

    33

    Appendix EQuestionnaire for administration:

    1. Please describe your observations of the Smartboard for educational use in theclassroom that enhances students learning and understanding of course topics?

    2. Have you observed students using the Smartboard in class on a consistent basis?3. Have you observed teachers using the Smartboard for projection use only (i.e.

    for movies or presentations only)? If so how frequent? Please share observationsyou have about this question.

    4. Do you believe that there are enough teachers training for Smartboards? Pleaseexplain.

    5. Feel free to express any other thoughts and observations you may haveregarding the Smartboard at our school.

  • 8/4/2019 Flynn Evaluation Smart Board

    34/35

    34

    Appendix FRemaining questionnaire for administration regarding the Smartboard program

    Have you observed teachers using the Smartboard for projection use only (i.e. for moviesor presentations only)? If so how frequent? Please share observations you have about this

    question.Administrator 1

    To be honest, I have probably seen it used more often for projection than for interactiveactivities

    Administrator 2

    Yes there are still a few teachers who continue to use the smartboard as a projectionsurface and also as a whiteboard with minimum functionality. The main reason for this isthat the teacher has not reviewed their teaching methodology and their pedagogy in lightof what is possible. Technology in itself will do little unless the thinking changes in theclassroom as to how the technology can be applied in smart ways to enhance learning.

    Do you believe that there are enough teachers training for Smartboards? Please explain.

    Administrator 1

    Time is always an issue in schools and I know that we have tried offering variousoptional sessions on various tech issues that have not been well attended. The simpleanswer is no, there has not been enough training. The more complicated part is findingmeaningful training that will help teachers at all different levels move forward. I think we should offer a series of required sessions (offered at least two different levels) for all

    teachers who receive Smart Boards. These may not need to be long. In addition, itwould be great to have a Smartboard expert who could individually meet with teachers intheir classrooms to talk to them and show them a few new things that they can do.

    Administrator 2

    We could do with more training and sharing of ideas and sharing of our successes. Themore we share the better the understanding as to what the possibilities are with thesetools. Time of course has been a challenge to schedule training. Teachers also need to bea little open and willing to take risks and explore these new frontiers. Effort is equallyrequired on the part of the teachers to explore new ways of instructional delivery. TheInternet contains thousands of resources and ideas and there is much we can learn and

    adapt from these online resources.Feel free to express any other thoughts and observations you may have regarding theSmartboard at our school.

    Administrator 1

    I believe that Smartboards can very effective teaching tools. At the moment, they areprobably best used by our Science department but we are probably not making the best

  • 8/4/2019 Flynn Evaluation Smart Board

    35/35

    use of them as interactive devices. Using the Smart Board to create interesting multi-media lessons is good but we can do more.

    Administrator 2

    We have made a start but there is till work to be done. The potential and possibilities arehigh and encouraging. So we must forge on to go beyond the obvious and reach beyondthe horizon of tools like smartboards to enhance and foster teaching and learning.