Wk 3: Evaluating Programs & Human Performance Technology

1.  Search for at least two other models used for evaluation and summarize these models.  Describe how you would use them to evaluate your instruction.

An additional evaluation model I studied was the Stenhouse Research Model.  Now, I must admit, I chose to look into this model because of the word research...not that I'm really into research (I find it tedious and overwhelming at times), but because it is a big part of my job as a library media specialist.  I have to teach students how to research, showing students how to narrow the focus of their search, how to identify reliable sources, how to formulate questions they need answered in their search.  I also have to help teachers find reliable sources and to find new materials to use in their classrooms.  Much to my surprise, the Stenhouse model matched my contextualist bent as a teacher and even these jobs I have as an LMS as Stenhouse believed .  Stenhouse believed that instructional research should serve teachers so it could be used to improve educational practices, and this could only be done if teachers were the active researchers looking at individual cases to find better ways of instructing students.  This method is more about self-study and improvement than about objectives and data gathering.  Because teachers are researching how to create better curriculum and how to better use this curriculum in the classroom, and they realize how experience and values affect how learning occurs, they are empowered to vary their approaches to curriculum to meet the needs of their students and are free to make changes in presentation as they see fit. So curriculum is not just a written guide to follow, but an interaction among teachers, students, and knowledge...kind of like problem based instruction.  Evaluation is a key in this process and should be formative and ongoing to maintain the focus on improvement. The teacher is considered the developer who studies others and their testing ideas and offers solutions to any problems that may arise while an evaluator  serves as the voice of practicality who keeps the enthusiasm of the teacher focused on the objectives and learning goals of the school.  I could see myself using this method to help me evaluate how the implementation of technology into teaching units.  By studying other teachers and librarians, collaborating with them on utilizing our new iPads, per se, I could constantly be reworking how we can make these iPads tools students can use regularly to enhance their learning of any subject through collaboration with peers and experts, searching for branches of the subject relevant to them and their interests, and creating presentations that show their understanding of learning. 

Another evaluation model I studied was Tyler's Objectives Model.  I initially was drawn to this model because it seemed to be in direct contrast of the Stenhouse Model, seeming to be more in line with the positivist stance.  And, after last week's blogs, I could see that I utilized some positivist-type instruction in my classroom as well.  What I found is that this is only somewhat true.  Yes, it is based on specific objectives and specific goals with data gathering to prove the effectiveness of the program, but it is also focused on improving learning.  Tyler's model is based on 4 questions: 1)  What educational purposes should the institution seek to attain?  2)What educational experiences are likely to attain these objectives?  3)  How can these educational experiences be organized effectively?  4)  How can we determine whether these purposes are being attained?  These are the evaluative questions to be asked all throughout the learning process.  It is imperative that strong goals and objectives are set at the beginning to provide a guideline all the way through. At the end, data will be collected to compare actual production with the goals to determine what improvements in curriculum should be made.  Since this method more closely matches the state issued TEKS and how we assess those, I could see me using this evaluation method for just that purpose...to assess how well our learning units are meeting the needs of our students in meeting the expectations of the state.  This method allows us to follow the objectives and gather hard data evidence into whether or not our instruction is meeting the demands.

                                                http://www.flickr.com/photos/robinhutton/8301964632

2.  Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods.  Should return on investment and management of resources also be considered in evaluations of instructional programs?  What other measure should/could be considered?

Instructional design evaluation ultimately, in the state of Texas, addresses learning in the traditional method of summative assessments in the form of EOCs and STAAR tests.  What we as educators and even parents are finding is that this is not the best method in evaluating students or programs.  We are not assessing the application of learning or the continuing learning that results from instruction.  We are not assessing how productive students will be at adapting this knowledge to the real world in their high-level education and in their jobs of the future.  Instead, we should also be asking:  Are students engaged in learning?  Is the learning relative to their own interests?  Are they manipulating the information gained in order to solve real-world type problems?  Are they collaborating with others in their questioning of the information and how it applies outside the classroom?  Have they utilized technological tools in such a manner that they know how to question and search for information they do not know?  And yes, Return on Investment definitely should be considered and has to be considered in this new age of technology usage, especially considering the rate of change in the technologies used, and considering the expectation that students do need to go on to pursue higher education and be successful in those institutions.  Parents and schools are spending more and more money on these technological tools to be utilized for deeper learning, so they are expecting this return on their investment.  Too often, this technology is utilized only superficially in the classroom, and so students go on to higher learning institutions and even into the job market not fully understanding how to use these tools to help them solve problems.  Looking at retention levels in colleges of students from these high schools could be another measure looked at, or even looking at employment percentages might be considered.  Surveys could also be given in the short-term to parents to evaluate how they felt about the quality of education their student received, and in the long-term to the student themselves...after they have finished college and have had some time in their career of choice.  I was already thinking along these lines of ROI, when I happened upon a blog on facebook this week about a father's feelings about the ROI of having his daughter in dance at the age of 7.  It was funny reading all the terms related to Return on Investment in relation to dance, and it actually made these terms more understandable to me because they were used in a topic that I could relate to personally as my daughter is in competitive cheer.  In fact, earlier that same day (that I read the blog) another mom and I were talking about how all the ridiculous amounts of money we spent on the cheer was actually well worth it because of what our daughter's were gaining---self-confidence, leadership, self-discipline.  It really made me think of instructional design and the importance of evaluation in a different way.  It really made me think about the needs and expectations of our students and especially their parents.

http://dancedadgourmet.com/?p=671

3.  Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning.  Not all problems in learning and/or performance require an instructional one.  Many times a non-instructional approach is a more appropriate solution.  Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem.  Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem?

I would say a performance problem in my area of work, being a library media specialist, would be getting teachers to effectively utilize technology in the classroom...especially since my school district readily spends money on upgrading technology in our schools and parents support the higher taxes to do so.  For instance, all of our schools in our district (16 elementaries, 3 middle schools, a freshman center, and a high school) just received carts of iPads, this just a year after all classrooms got SmartBoards and new document cameras.  Anyway, while the excitement always has teachers "playing" with the new technology with their students at first---looking up information, playing games, showing pictures and videos---soon the new technology gets left on the wayside in many classrooms.  Teachers state they don't understand how to use the technology and have no time to learn.  Several state they don't even know where to start to try and figure it out.  Yes, our district offers initial classes to show us how the technology can be integrated in the classroom, very seldom is this training hands-on with practice time built it in or even ongoing throughout the year.  Teachers get overwhelmed and frustrated and so very resistant to using the technology.  My school is trying to address this problem in a couple of different formats to try to make the integration of these technologies less intimidating.  One thing we have done is set up a campus tech team.  Teachers and administrators alike are on this team to study different ways these technologies could be implemented in the classroom.  Then every other week, we present findings to the staff in a faculty meeting.  The next weeks, we hold a Technology Tuesday after school where teachers can come and practice with the technology while members of the Tech team are there to help get started and to answer any questions that may arise.  These tech team members also serve as tech mentors and are available for questioning about the technology at all times.  Our district has also set up a Technology Tool Box on Google+ that contains click sheets for different technologies, idea websites, app lists, and such.  Any computer tech, library media specialist, or administrator may add to this.  All teachers may give information to any of these individuals to add to the Tool Box.  This just started this past spring and is proving to be very beneficial to all.

Sources:
Evolution of the Model. (n.d.). Tyler Objective Curriculum Evaluation Model - HOME. Retrieved June 18, 2013, from http://tylerobjectivemodel.weebly.com/index.html       

Hendricks, C. C. (2002). A Review of the Work of Lawrence Stenhouse: Questions, Ambiguities, and Possibilities. Journal of Research in Education, 12(1), 117-121. Retrieved June 19, 2013, from http://www.eeraonline.org/journal/files/2002/JRE_2002_15_Hendricks.pdf

 

Shepherd, J. (n.d.). Models of Curriculum Evaluation. .docstoc. Retrieved June 18, 2013, from www.docstoc.com/docs/2279101/MODELS-OF-CURRICULUM-EVALUATION-1-What-is-evaluation