“In God We Trust, All Others Must Bring Data”

The title of my post is from W. Edwards Deming, and while Deming was referring to the business world, we all use data in some form and fashion. I’m sure Deming also used data in education because he was an assistant professor early in his career before moving into lecturing in Japan (deming.org). As educators, data is important for driving student growth, informing teachers where to make adjustments to their lessons, and plenty of other decision-making information. As I looked at ISTE 4.5.C, which says, “[Coaches should] evaluate the impact of professional learning and continually make improvements in order to meet the schoolwide vision for using technology for high-impact teaching and learning (ISTE).” I honed in on the first part, evaluate the impact of professional learning mostly because I always want to be mindful of people’s time and for them to learn something poignant as they participate in PD. To do that, I wonder what to do with all that data collected from the feedback received from PD. 

I found an article written by Dr. Sheila Robinson titled “Professional Development Program Evaluation for the Win: How to Sleep Well at Night Knowing Your Professional Learning is Effective.” Dr. Robinson offers several ways to collect the data: surveys, interviews, focus groups, and observations. I think I have always received surveys about PD sessions the majority of the time, but I believe that observations would be one of the best because it offers the chance to see strategies put into practice. Robinson says, “Observations let you see first-hand how the curriculum is being implemented, how instructional strategies are being used, and how students are responding” (Robinson). Of course, this takes time out of the observer’s schedule, which is probably why we seem to see more surveys completed after the PD has ended. Regardless of the data collection method, what do we do with it? We analyze it!

Some people love looking at data. I suppose that is why there are people whose career is data analysis (Ha!), but it is necessary even if you don’t like analyzing data. “You need actionable knowledge to report out results that inform smart decisions about professional learning” (Robinson). Assuming you offer a survey, one important step, according to Robinson, is finding out the response rate. Take the number of people who completed the survey and divide it by the number in attendance, and you get your response rate. This answer helps because if your response rate is low, it could be that you asked too many questions, didn’t make the responses mandatory, or it was lost in an email. That is only one piece of the puzzle of your data collected.

Many surveys are quantitative (closed-ended questions), and the answers can be seen in graphs and percentages. Qualitative questions (open-ended) are great for surveys, interviews, and focus groups and can help get more specific information–the why’s. The data needs to be categorized, which can be done by coding. For example, if the word “time” continually shows up in responses, all those responses can be grouped together as one category. Later, all the responses related to “time” can then be analyzed closer to see if the responses are positive or negative. It could be that the PD session focused too long on a subject the group felt was not vital for their time of learning (negative) or that they wished they could have spent more time on a particular topic (positive). As you can see (and probably why there are people who analyze data for a living), there is so much information to sort through with the data collected.

I have just given a couple of examples of what to do with all the data, and there are a lot more examples that can be shared, but it is also necessary for me to also say that reporting the data to the right stakeholders is vital. Changes cannot happen unless the right people know the correct information. To that end, Robinson says, “They want to quickly and easily understand key findings. Charts or graphs can be efficient and powerful ways to communicate data” (Robinson). I agree; a picture of something can say a lot versus just reading a summary of someone’s findings. I do enjoy a good pie chart! Whatever the method of collecting data, surveys, interviews, or observations, the goal is to see professional growth for our educators with the ultimate goal to see successful students!

References

ISTE standards: Coaches. ISTE. (n.d.). Retrieved March 7, 2022, from https://www.iste.org/standards/iste-standards-for-coaches

Robinson, S. B. (2021, May 11). Program evaluation for Effective Professional Development. Frontline Education. Retrieved March 7, 2022, from https://www.frontlineeducation.com/program-evaluation/

3 thoughts on ““In God We Trust, All Others Must Bring Data”

  1. J Freeman says:

    I enjoyed reading your perspective on data collection! Regarding response rate, which is so important to the overall research process: Yes, it seems important to limit the number of questions added to a survey, as too many might discourage people from completing it. I also think communicating the amount of time it will take to complete a survey, such as 5-10 minutes is important to state, sending out reminders to complete the survey, and offering an incentive, such as a raffle, can increase the response rate. And as you mentioned, the ultimate goal for PD data collection and analysis (and all that goes into this process) is student success!

    Reply
  2. Nick R says:

    Hi Jeff, I thought you brought up a great point of data collection. Your source encouraged observations because you get to see strategies implemented in real time, but you note that there is very limited time, if any at all, to conduct those observations. It feels like good PD must have institutional support in order to have effective & sticky ongoing PD. I appreciated the different opportunities you presented for data analysis and loved that you included the qualitative coding. That was a memorable class and meaningful strategy that I am excited to try myself. Thank you for sharing!

    Reply
  3. Deanna Bush says:

    Hi Jeff, You make a really good point about the importance of making sure that the data from the findings gets communicated to the right stakeholders. . .and that it is communicated clearly in a way that is easy to understand.

    Reply

Leave a Reply

Your email address will not be published.

css.php