STORIES FROM THE FIELD
Humanizing the Data
A Path to More Compassionate Systems in Education Starting
from the Inside Out
By Jennifer Yales Ed.D
Introduction by Art Kleiner and Peter Senge
Here is a story about the difference that systems awareness can make in the way we gather and use data about students. Jennifer Yales is Senior Director for the System Improvement Leads (SIL) project out of West San Gabriel Special Education Local Plan Area (SELPA). This project involves thinking the way special education monitoring data is used in the state of California. Working for one of the three SELPAs selected as the grant recipients in Southern California, Dr. Yales was also part of the 2020 Master Practitioner in Compassionate Systems Certification Program at the Center.
As these two experiences converged, Dr. Yales became keenly aware of one of the perennial problems with educational data. Though it reveals broad trends, it tends to obscure or even distort the reasons why those trends exist. And if it uncovers problems, as educational data often does, it cannot be used on its own to address those problems.
When we assign agency to the data – with phrases that typically begin, ”The data says…” – we are usually projecting our own biases and pre-set interpretations. They don’t show up in the official reports because they are obscured by the seemingly technical complexity of the data and the administrative realities of the case at hand.
Educational data must therefore be incorporated in a compassionate approach, with a high level of insight and engagement. The data should bring people together instead of labelling and isolating them. (For an in-depth story of what it takes to accomplish this, see “The Human Early Learning Program at British Columbia“.
In this case, the problem uncovered by the data is the systemic misplacement of underserved children – a problem closely related to issues of equity in school systems. This article describes the onset of a project that is still underway, and we don’t fully know where it will lead. But it articulates a vision for how data could be used to make things better. And the article itself is intended as an awareness-raising vehicle, inviting other educators to participate in using data in better ways, particularly when it comes to special education.
Two years ago, after I started working at a systems level in the field of special education, I became more aware of some issues related to the way we use and value data. I am not a data analyst by background. Like others in the field of special education, I’ve had many roles throughout my career that required me to use and interpret data at various levels. My training as a school psychologist, for example, required that I collect data to understand the performance and needs of students, interpret a variety of assessments and communicate data to staff and parents.
When I became a program specialist then administrator, I worked with site and departmental data, including data monitored for compliance by the state. When it came to compliance monitoring activities, I found myself seeking solutions to surface level data issues without the time (and perhaps in-depth knowledge) to truly investigate the depth of these data issues. I’ve come to realize that many of my responsibilities required that I solve day-to-day issues, legal issues, placement decisions, behavioral crisis’, etc. in real time and sooner rather than later. There was no recognition or reward for contemplating the data in more depth, or thinking about the effect of these decisions on real children and families. And there was such a flood of compliance issues that there was no time to slow down and gain perspective.
But then I became part of the System Improvement Leads (SIL) project (a grant supported by the California Department of Education (CDE) and California Collaborative for Educational Excellence CCEE), which brought about an opportunity to rethink the way we approached how we solve problems of practice and look at special education data. I work on a team that focuses on special education data and we are in the process of creating a new data tools dashboard that allows districts to better monitor their State Performance Plan Indicator (SPPI) data for special education. This system is still in its beginning stages, but it is already a game-changer for special education. For instance, it enables districts to upload their data in real-time, allowing district leaders to see how they are doing without major delays. Understanding a school district’s current reality with more real-time data is an essential component of our project’s continuous improvement journey utilizing the methodologies of improvement science.
At the same time, through the grant, I was able to participate in The Compassionate Systems leadership Master Practitioner Certification at the Center for Systems Awareness. As part of the certification process, I had to complete a project and decided to couch my project within the scope of my work around data. Up until this time, I have been surrounded by data that involved mostly numbers. As a system we were continuously being reinforced by hitting numerical targets. As I reflected on this, I became concerned about the limits of our system. Specifically, what Nora Bateson, filmmaker and research director at the International Bateson Institute, calls “cold data:” purely quantitative statistics that don’t express the underlying relationships and human interactions that matter most. [1] Through the program I began to see a greater need for moving beyond the coldness of focusing purely on numbers and the rich warmth that could help our educational system by incorporating human elements to our approach. Understanding that connection, I realized, started with the need for me to understand myself.
Moreover, the data system had brought to the surface some deep systemic problems that we as a state and country are trying to solve for the betterment of all of our students. For example, when working with data we are not speaking about the biased based beliefs that play a role in the way we select and interpret data. These biased based beliefs are causing us to wrongly classify children as disabled and remove them from access to general education resources and opportunity.
To address this we need to better connect with our own beliefs and have conversations about the data as a part of a system and not just a number. There are compassionate systems tools we can use to start this self-reflection and create a more generative social field that invites us to hear and express our stories to “warm” our data. The end goal is educational institutions that can connect with families and children more completely, giving them classroom and other programs that truly meet their needs.
Data and Accountability
The problem with misplacing students had begun as a quick fix to another problem. In years gone by in California, disproportionate numbers of Black and Brown students had been identified as needing special education and related services. In the districts I had worked for and with, African American students have been over-identified with emotional disturbance. Latinx students have been overidentified with specific learning disabilities. Once identified with labels of this sort, students can carry them as burdens throughout their academic careers and beyond. The number of Black and Latinx students bearing these labels had been 7 to 10 times greater than students from other race/ethnicity backgrounds.
This issue is not isolated to the state of California, of course; it is really a national issue. We use numerical standards to monitor a quantitative issue. Specifically, The Individuals with Disabilities Education Act (IDEA), requires the California Department of Education (CDE) to conduct monitoring activities through submitted data that identifies disproportionate representation in special education based on race and ethnicity. If a school district is found to be out of compliance – that is, they are found to have a risk ratio of 3.0 or higher – then the district must put in place a plan to address this.
I wholeheartedly agree we need accountability in our system. My intent in this article and my project is not about disputing the current metrics. Instead I wonder what would happen if our emphasis was more on understanding our individual contributions (beliefs, identity, etc.) and our interconnectedness to the educational system we make up would then these inequities diminish. If we had more compassion by understanding our own story and the stories of others, would we better understand our students and their needs.
It’s hard to change this system because it’s hard to see. The guidelines and expectations for special education are long-established and deeply rooted in the professional culture. Accountability focuses on compliance with the rules, following the metrics rather than what is happening for the children. And unspoken equity issues still influence educational decisions. As Temple University applied researcher Edward Fergus put it, “Educators can unknowingly convey – with varying degrees of nuances – ‘inadvertent’ or ‘unconscious’ racialized expectations that lead to hierarchical stratifications of students across backgrounds and intersectional differences.” [2]
The SIL project and the Master Certification program invited me to think holistically about these issues. Could we develop and use data standards more compassionately? Are there voices left unheard and experiences that should be connected to mitigate the “coldness” of the data? Could we have outcomes that were less biased – and more useful, realistic, and helpful to children’s future adulthood? What would a system look like that enabled those outcomes to occur, not just when a family or advocate protests, but all the time?
Cold versus Warm Data
One starting point is warm data. Nora Bateson coined that phrase to refer to information that takes into account the interrelationships among people and systems, and that looks for patterns of connection. “Simply using analytic methods focused on parsing statistical (cold) data,” she wrote, “will often point to conclusions that disregard the complexity of the situation at hand.” The data on special education clearly fit that description. And to Bateson, the use of cold data naturally leads to misuse. “Information that does not consider the full scope of interrelation in a system,” she added, “is likely to inspire misguided decision-making, which compounds already “wicked” problems.”[3]
But data about special education could be kept “warm” – that is, resonant with the human relationships and other interrelationships involved. According to Bateson, this can be done in several ways. There can be multiple descriptions of a child’s situation, coming from different observers, not just one educational evaluator. The findings can refer to similar patterns of behavior found with other students, who may have ended up placed in different ways. The data can include changes over time, in the relationships with adults or other children, or in the types of behavior. There could be more discussion of the context, including the community, the economics, and other factors.
Our project has been designing a regular set of sessions for administrators and others concerned with humanizing the data. We want to create a space where we could weave qualitative data along with quantitative data, to create a more complete understanding of each child’s context. Stories and interviews with the children and their families will show personal realities underlying the statistics.
Cold data will still be important. Statistics need to be pulled and combined to reveal patterns affecting each student, and each school system. Equity and culture must be kept in mind when data is pulled. We should orient ourselves to seeing macro-patterns emerging in real time, about students and their relationship to the special education system. Questions like these can be answered by looking at cold data more closely:
- Are all students afforded opportunities to access general education prior to referrals to special education? Have most students in general education been receiving appropriate services?
- What do we notice about the students that are being pulled away from general education for intervention, referrals or services?
- Are we disproportionately labelling children from some racial/ethnic backgrounds as students with disabilities compared to other race/ethnicities?
But we should not stop with cold data. We should warm the data, taking into account our observations about students, and stories told by teachers, parents, and the students themselves. Quantitative and qualitative data need to be looked at together to create a detailed roadmap for the future of each student – and for the school systems.
The Conversational Setting
To make sense of this data, we need more generative conversations, where leaders can learn more skillful and nuanced approaches to data analysis. Once this kind of data analysis is prevalent it will be easier to provide students the support that will fit them and serve them well, no matter what their background or previously assigned label.
The conversational elements used to create this setting would include:
- Using and teaching check-ins so that educators, students and families can connect.
- Helping educators understand how mental models can affect the data we select and how we analyze it.
- Incorporating the ladder of inference to help teams stay grounded in the observable details of the data, rather than leaping to assumptions or conclusions.
- Using the Iceberg diagrams to show how the sources of surface problems are not obvious, and how our attitudes and mental models, below the surface, may lead us to categorize children as emotionally disturbed.
- Introducing data protocols to remind teams of the value of staying grounded.
Designing and constructing a whole new way of interpreting quantitative data and interconnecting it with qualitative realties in a student’s life would require a number of new practices. But the conversations can be a start. Our purpose is to use data correctly and carefully. We want to make sure special education is not becoming a place for children who are misunderstood or misdiagnosed and we want to scrupulously avoid disproportionality. The new approach, if set up with heart and integrity, can help.
My perspective on this is shaped, in part, by my experience as a parent of a child with disabilities. Although I am a credentialed school psychologist and special education administrator myself, I too become someone totally different when I walk into an Individualized Education Program (IEP) meeting where my child’s future will be discussed. My mama bear claws come up as do my own personal biases based on my experiences. This is important for me to understand as I lean into and help shape our educational system.
Compassionate systems has helped me a lot, as a parent and as an educator, by growing my awareness of my actions, biases, emotions and how I manifest them. As a practitioner in the field, I know that no one is intentionally out to ruin a child’s life. As a parent, it is challenging to keep this in mind. As someone who works with data, I know the kind of effect it can have.
Thoughtful use of data about children and schools will force us all to deeply examine our own mental models. As Peter Senge says, mental models are generally tacit, often untested, and unexamined. The new data will shine a harsh light on us all but it will clean up the mess we all are unhappily living in. Most people in education mean well. We want to use these new tools to create harmony, better experiences for children, deeper meaningful learning, and equity.
I want to thank the California System of Support. It is a crucial component of the state’s accountability and continuous improvement system, whose guiding principles include equity and local control. The overarching goal of the System of Support is to address inequities and build the capacity of Local Educational Agencies (LEAs) to improve teaching and learning over time, address achievement gaps, and strengthen outreach and collaboration with their stakeholders.
As part of the system of support, the SELPA Systems Improvement Leads (SIL) Project is connecting with LEAs throughout the state with a common goal of improving outcomes for students with disabilities. We are interested in connecting with school district who want company on their improvement journey as you are not alone. We are making a concerted effort to understand our current realities, avoid finding solutions without understanding the problem, appreciating the complexity of our system and adopting a continuous improvement mindset. We are not alone. We will get through this together and come out on the other side a better, more compassionate school system.
As for my project, please watch and share this short illustrated video on the Ladder of Inference from Vimeo.
It can help us understand how our action(s) depends on how we understand a situation. The Ladder of Inference is a great tool that can help us to build awareness of our own assumptions and challenge our thinking before we take action. This includes actions in selecting and analyzing data in education.
For more information on the SIL Project: https://systemimprovement.org/
Jennifer Yales
Senior Director | System Improvement Leads (SIL) Project | West San Gabriel Valley (WSGV) | Special Education Local Plan Area (SELPA)
California HubUSA
See bio >>
1. Nora Bateson, “Warm Data,” Hackernoon, May 28, 2017, https://hackernoon.com/warm-data-9f0fcd2a828c
2. Edward Fergus, “Confronting Colorblindness,” Phi Delta Kappan, February 2017, DOI:10.1177/0031721717690362
3.“Warm Data,” op. cit.
See list of all Newsletter - Spring 2022