class: left, bottom, title-slide # Module 3: Methodology ## EME6357: Evaluation of Instruction & Training in HPT ### Dr. Bret Staudt Willet ### October 13, 2021 --- class: inverse, center, middle #
**View the slides:** [bretsw.github.io/eme6357-module3](https://bretsw.github.io/eme6357-module3) --- class: inverse, center, middle # Team Dynamics <img src="img/community.jpg" width="720px" style="display: block; margin: auto;" /> --- class: inverse, center, middle #
<br> Evaluation Plan --- class: inverse, center, middle # Systemic Logic Model <img src="img/3-systemic-logic-model.jpg" width="420px" style="display: block; margin: auto;" /> --- # Evaluation Plan, Part 2 <img src="img/group-project.jpg" width="600px" style="display: block; margin: auto;" /> -- - Again, fantastic work -- - Generally, great job introducing the problem and giving a needs statement -- - Add some citations or hyperlinks to your introductions --- # Evaluation Plan, Part 2 <img src="img/group-project.jpg" width="600px" style="display: block; margin: auto;" /> - For non-traditional training, think about where **expertise** resides -- - Potentially, in peers, not just the facilitator --- # Evaluation Plan, Part 2 <img src="img/group-project.jpg" width="600px" style="display: block; margin: auto;" /> - Focus on the importance of **transfer** (Kirkpatrick's levels 2-4: learning, behavior, results) -- - Remember the distinction between **outputs** and **outcomes** --- # Evaluation Plan, Part 2 <img src="img/group-project.jpg" width="600px" style="display: block; margin: auto;" /> - Try to go a step deeper: offer some **critique**, not just description -- - For example, if you comment on the facilitator's credentials, also try to answer, "So what?" --- class: inverse, center, middle # Systemic Logic Model <img src="img/3-systemic-logic-model.jpg" width="420px" style="display: block; margin: auto;" /> --- class: inverse, center, middle #
<br> Flipgrid Discussions --- # Flipgrid Discussions: Feedback <img src="img/chat.jpg" width="720px" style="display: block; margin: auto;" /> --- class: inverse, center, middle #
<br> Module 2 lecture: Tradeoffs --- class: inverse, center, middle <div class="padlet-embed" style="border:1px solid rgba(0,0,0,0.1);border-radius:2px;box-sizing:border-box;overflow:hidden;position:relative;width:100%;background:#F4F4F4"><p style="padding:0;margin:0"><iframe src="https://fsu.padlet.org/embed/2be7mvchzccn5l4n" frameborder="0" allow="camera;microphone;geolocation" style="width:100%;height:480px;display:block;padding:0;margin:0"></iframe></p><div style="padding:8px;text-align:right;margin:0;"><a href="https://padlet.com?ref=embed" style="padding:0;margin:0;border:none;display:block;line-height:1;height:16px" target="_blank"><img src="https://padlet.net/embeds/made_with_padlet.png" width="86" height="16" style="padding:0;margin:0;background:none;border:none;display:inline;box-shadow:none" alt="Made with Padlet"></a></div></div> --- # Self-Report Research <img src="img/streetlamp.jpg" width="480px" style="display: block; margin: auto;" /> - Humans are bad at estimating -- - (People are also bad at remembering) -- - People change once they know you're observing -- ###
Observer Effect --- class: inverse, center, middle #
<br> Module 3 readings: <br> Evaluation Methodology --- # Brandon et al. (2011) <img src="img/interview.jpg" width="520px" style="display: block; margin: auto;" /> -- - **3 case studies** (pp. 295-297) -- - **interview with a training evaluator** (pp. 297-303) --- # Brandon et al. (2011) <img src="img/interview.jpg" width="520px" style="display: block; margin: auto;" />
**Key idea:** -- - Evaluation methodology is **less** about; -- - research design or methodology --- # Brandon et al. (2011) <img src="img/interview.jpg" width="520px" style="display: block; margin: auto;" />
**Key idea:** - Evaluation methodology is **more** about: -- - flexibility and responsiveness to clients - communication skills - access to subject matter expertise when conducting evaluation studies --- # Kundin (2010) - Framework <img src="img/3-kundin-2010-framework.png" width="720px" style="display: block; margin: auto;" /> --- # Kundin (2010) - 1. Awareness <img src="img/3-kundin-2010-element1.png" width="100%" style="display: block; margin: auto;" /> --- # Kundin (2010) - 2. Reasoning <img src="img/3-kundin-2010-element2.png" width="100%" style="display: block; margin: auto;" /> --- # Kundin (2010) - 3. Reflection <img src="img/3-kundin-2010-element3.png" width="100%" style="display: block; margin: auto;" /> --- # Schwarz & Oyserman (2001) ### Self-Report Research <img src="img/3-schwarz-and-oyserman-2001-table1.png" width="100%" style="display: block; margin: auto;" /> --
**Key idea:** Self-reports can be profoundly influenced by the research instrument --- class: inverse, center, middle # Questions? <img src="img/question.jpg" width="480px" style="display: block; margin: auto;" /> **What questions can I answer for you now?** **How can I support you this week?** <hr>
[bret.staudtwillet@fsu.edu](mailto:bret.staudtwillet@fsu.edu) |
[@bretsw](https://twitter.com/bretsw) |
[bretsw.com](http://bretsw.com)