Visualizing Inter-rater Reliability

Background on Reporting Inter-rater Reliability Qualitative studies often report inter-reliability (IRR) scores as a measure of the trustworthiness of coding, or an assurance to readers that they might follow the researcher’s codebook and expect to find similar results. How these scores get reported varies widely. Often, I see just the range of scores reported, hopefully with Cohen’s kappa calculated in addition to the more straightforward percent agreement. The kappa is important because it takes into account variance in the frequency of a code.

Dissertation Themes

As I’ve wrapped up numerous cycles of qualitative coding of interview data for my dissertation, Into the edu-verse: New teachers seeking induction support on social media, I used geom_tile() in {ggplot2} to create heatmap plots to help me quickly visualize how thematic codes varied by interview participant. Heatmap Comparison of Thematic Codes by Interview Participant Note. Columns have been computationally reordered using principle components analysis (PCA) so that side-by-side columns are more similar than non-adjacent columns.

Comparisons in Twitter #Edchat during COVID-19

I am in the midst of two projects looking at how Twitter #Edchat tweets may have been affected by the COVID-19 pandemic. The first project is primarily comparing tweets in Spring 2020 to those in the same timeframe a year earlier. The second project focuses on Spring 2020 but compares tweets from U.K. and U.S. users. I used geom_tile() in {ggplot2} to create heatmap plots to help me quickly visualize and compare a large number of data points.

Bret's Project Landscape (hosted on Trello)

Projects of Bret Staudt Willet

Dissertation Proposal

Into the Edu-verse: Early-Career Educators Seeking Induction Support on Social Media On November 26, 2019, I successfully defended my dissertation proposal for my PhD in Educational Psychology and Educational Technology through Michigan State University’s College of Education. I have shared the full text of my proposal here: https://bretsw.github.io/dissertation-proposal-text/ You can scroll through my presentation slides for the defense below (use the arrow keys) or view them here. At the end of the defense, I asked my committee for advice on sustaining a project as large as the dissertation, and they offered some great wisdom.