Introduction
A data analysis protocol is an essential tool to analyze data as a group. As I discussed in an earlier post, there is great value in involving all stakeholders in local data analysis and decision making. There are many benefits to this including:
- Increased transparency and stakeholder trust in decision-making
- Enables localized, context-aware decision-making
- Fosters a culture of continuous improvement and shared accountability
When I work with schools, I always advocate for regular data analysis meetings that equally involve teachers, administrators, and instructional coaches. In the right circumstances, there could be great value in holding a meeting like this that involves community members, parents, or students, as well.
You can’t just hand a bunch of data to a group of people and expect magical things to happen. A data protocol is the key to an organized process that will ensure proper analysis, organized thinking, and equity of voice. There are a number of protocols that you can find that might work for your team, but, for me, I’ve facilitated enough data meetings to know what I’m looking for in a protocol. Below, you’ll find a protocol that I built myself, based on my experiences. Here are the key aspects:
- The exercise is 45 minutes in length (though I’ll discuss modifications to this below)
- Conducted with groups of 4-8 people (sitting at their own table, if part of a larger group meeting)
- There is one table leader for each group (whose main job is to keep everyone on track using the protocol; they don’t necessarily need to do any prework)
- There should be at least one person in the room who is intimately familiar with the data and can answer questions (they can hop from table to table if it’s a big group)
I call this the Collective Wisdom Data Analysis Protocol because of its emphasis on the collaborative analysis of the group, which will lead to greater insight. I will write this up below but I have also created a printable PDF version that you can download and use with your teams. You can jump straight to that version by clicking here.
Collective Wisdom Data Analysis Protocol
Prework:
- Ensure the data is individually accessible to every participant (paper copy, computer access, etc.).
- Print out a copy of the protocol for each participant (page 2 of the PDF only).
- Determine the layout of the room. If the group is large, split into breakout groups of 4-8 people each.
- Decide how groups will be determined to maximize diversity of thought. Try to reduce the number of people with similar job titles in the same group.
- Create a method for assigning a table leader and a notetaker for each group. The table leader will facilitate the discussion using the protocol. The notetaker will keep notes from the discussion. These two people can also be participants in the data review.
Notes to table leader:
- Your job is to facilitate the discussion according to this protocol. You do not have to be an expert on the data; you just need to be the referee who keeps everyone on track.
- You might want to consider turning the first few minutes of each section into “quiet time” so that participants can gather their thoughts. This is especially important for Step 2, when they first spend individual time with the data.
- Don’t let anyone jump ahead. If, for example, someone tries to interpret the data before step 3, politely remind them that we will get to interpretation shortly.
Step 1: Get Familiar with the Data (10 minutes)
- Review the data set together.
- Clarify what each part of the data shows (e.g., dates, student groups, assessments).
- Ensure everyone has a shared understanding of what they are looking at.
Step 2: State the Facts (10 minutes)
- Describe what you see without interpretation.
- Focus on observable trends, patterns, and numbers.
Example: “60% of students scored below proficiency on the writing assessment.”
Step 3: Interpret the Data (10 minutes)
- Ask: What might explain these results?
- Discuss possible causes, including instructional, systemic, or environmental factors.
Example: “Writing instruction was interrupted due to snow days.”
Step 4: Identify Gaps in Understanding (5 minutes)
- What don’t we know yet?
- What questions do we still have?
- What additional data or perspectives do we need to fully understand the situation?
Step 5: Decide on Next Steps (10 minutes)
- What actions should be taken based on the data?
- Who will do what, and by when?
- How will we monitor progress or revisit the data?
Postwork:
- If the small groups are part of a bigger group (such as an individual table in a larger meeting), consider having each table share out after they have finished their analysis.
- Collect the notes from the notetaker for future review.
Modifications:
- If the meeting length needs to be adjusted, try to keep roughly the same time ratios for each step. So, ~22% for Steps 1, 2, 3, and 4, and ~11% for Step 4.
- If you want to take this one step further, have the group begin developing SMART goals as Step 6..
Why the Collective Wisdom Data Analysis Protocol is Effective
As I discussed in my post on the subject, following this protocol allows schools to tap into a form of crowdsourcing. By combining crowdsourced insights with structured analysis, schools can access more relevant, actionable intelligence—and ultimately make smarter, more responsive decisions that reflect the realities of the people they serve.
James Surowiecki’s 2004 book, The Wisdom of Crowds, argues that groups can and usually do make better decisions than any single member of the group would be able to. He proposes many reasons for this but a big one is the power of having a diverse set of brains working on the same topic. Surowiecki writes, “Diversity helps because it actually adds perspectives that would otherwise be absent and because it takes away, or at least weakens, some of the destructive characteristics of group decision making [such as herd mentalities].” It’s not enough to include only certain people in data analysis and decision making, even if those people are really smart. Surowiecki writes, “…on the group level, intelligence alone isn’t enough, because intelligence alone cannot guarantee you different perspectives on a problem.” There is incredible value to opening these opportunities up to as many stakeholders as possible.
This is a perfect example of an element within an effective Data-Driven Education System. Participants who are actively engaged in the process will feel a sense of ownership and accountability for the data they are analyzing.
