Dr. Mark Garrison, of West Texas A&M, founder of NewEdu, and research partner with the Professional Association of Georgia Educators, highlights what he has discovered after spending time with some great teachers, administrators and students in Georgia’s public schools. The conversation touches on many topics, including the origins of standardized tests, practical steps policy makers can take–immediately–to support educators, the importance of listening to what teachers have to say, and the value of a new approach to professional learning.
Amarillo, Texas, and Buffalo, New York (2:15)
Data, evidence, propositions and their roles in the goals of a case study (4:40)
Focus groups, interviews, documents, email, and student work (7:40)
Standardized test scores and the range of metrics that matter (10:30)
Student input as a vital factor in gauging effectiveness of any approach to teaching and learning (12:20)
The standardized test debate actually started 150 years ago (13:30)
Trust as a pivotal component in “assessing” teachers and schools (14:40)
Technology as a two-sided challenge (15:00)
Standards reflect the values of those who establish them (16:30)
Who should make the decisions? (17:00)
How well do these tests prepare young people for their future? (18:35)
Professional learning that is intentionally transformative (20:30)
Trust surfaces, again – this time as a key to creating the conditions for true engagement (21:30)
The role of a teacher is not simple, one dimensional, or clear to everyone (23:50)
Multiple opportunities for success as an important foundational practice (25:15)
How to “improve things” alongside challenges and constraints (27:35)
Schools do mitigate the negative effects of societal pressures and difficulties, such as poverty (28:15)
The work of teachers has a significant positive impact on students and learning (29:00)
What it really means to offer students choice, and how doing so is a wise move (29:50)
Affirming student performance (31:55)
Quality professional learning uses a framework, not a formulaic, prescriptive, or predetermined set of steps (32:40)
What are these students going through right now? (33:55)
The value of teachers and their work (35:00)
The shifts in the processes used to capture evidence inside PAGE’s research projects (38:00)
Phil Schlechty (the Schlechty Center) – “It’s the activity of the learner that causes learning.” (38:45)
Community and political contexts impact the operational reality of schools (40:00)
Go back to the classroom – the center of teaching and learning (41:25)
One-with-one conversations with teachers – learning from our fellow educators (41:40)
About what do I want to learn more? (45:45)
Suggestions for policy makers (47:05)
Program adoption or a customized approach to an important issue? (47:50)
Organizational health and conflicting initiatives (49:30)
Everybody doesn’t have to do the exact same thing in order to make progress (51:15)
Teaching and learning are all about accountability and responsibility (53:25)
The lack of trust has not improved relationships, interactions, or processes (54:15)
Professional learning can legitimately advance the teaching profession (56:30)
Retirement deferred, career and work revitalized, as a result of thinking about one’s work (59:15)
Skinner’s Ghost and the Smart Machine (1:01:40)
Maybe not measure anything? Analyze patterns instead. (1:03:00)
https://www.amazon.com/Measure-Failure-Political-Origins-Standardized-ebook/dp/B007SJNZPI
How did standardized tests become the measure of performance in our public schools? In this compelling work, Mark J. Garrison attempts to answer this question by analyzing the development of standardized testing, from the days of Horace Mann and Alfred Binet to the current scene. Approaching the issue from a sociohistorical perspective, the author demonstrates the ways standardized testing has been used to serve the interests of the governing class by attaching a performance-based value to people and upholding inequality in American society. The book also discusses the implications that a restructuring of standardized testing would have on the future of education, specifically what it could do to eliminate the measure of individual worth based on performance.
https://eric.ed.gov/?id=ED517705
David Reynolds: What matters most in learning? The challenge, the thrill, the benefits, interacting with other people, or something else entirely? What is the connection between leading and learning? Does change drive learning or does learning drive change? What's more important? Teaching or learning? Is everyone a leader, a learner, a teacher? Want answers? Listen in as we address these intriguing issues through commentary and with guests who share their thinking and tell us their stories. Lead. Learn. Change.
Dr. Mark Garrison: It is important to provide multiple opportunities for success to students in the classroom and at the whole school level. Schools do mitigate the negative effects of other stuff going on in the society, whether it's poverty or other things that are associated with poverty. The level of inequality and hardship that people would face would be way worse if we didn't have public schools. It's not healthy for an organization to have conflicting programs and initiatives. The classrooms where it really happens, and you can't have someone who really excels in the classroom if their hands are tied. I started to get it, I started to see the value of thinking this way about teaching and learning.
David Reynolds: Today, we go deep into research. While our guest spends time on research terminology and design, he also lands on the other side, results. If you want to learn how evidence generated from case studies and learning visits clearly reinforces the idea that teachers matter, and that focusing on engagement can make a huge difference in professional learning, you will want to hear this entire conversation. Teachers matter. So too, does research. Today's guest on Lead. Learn. Change is Dr. Mark Garrison. Mark, thanks for taking your time to speak with me today.
Dr. Mark Garrison: You're welcome David, much appreciate it. Glad to be here.
David Reynolds: We want to start with two very deep and philosophical items. One, what's the weather like there in Amarillo, Texas, and two, have you seen any rattlesnakes yet?
Dr. Mark Garrison: Well, I'm going to answer the second question first, the answer is no.
David Reynolds: Okay.
Dr. Mark Garrison: And the first question is that it has been raining like we're in the tropics here for the last five days and it's cold and wet.
David Reynolds: So speaking of cold and wet, especially jumping in on cold, do you think you're going to enjoy the winter weather at West Texas A&M, where you live and work now or will you be pining for the days of snow and ice in Buffalo, New York.
Dr. Mark Garrison: I'm promised that we'll get a few inches of snow. So I will at least have some aesthetic reminder of home. And generally we have four seasons here. So I'm told, so I'm looking forward to that common experience. But I will be going home over the winter break, and so we'll actually be going skiing with the family. So I think I'm going to get the best of both worlds.
David Reynolds: Sounds great. So for some background for our listeners, Mark is a speaker and occasional blogger, a professor, consultant, award winning author, researcher, presenter has spent over two decades working with educators and is a supporter of teacher leaders, and those who work with them. We should also let our listeners know that Mark's a musician, he works with other musicians has his own sound studio and has produced some albums and some singles.
Now Mark, our working partnership grew out of a conversation that you and I had back in 2012 I think, and I had just started the Impact Project at PAGE and you presented at a conference that I attended. And you were really expanding on the basic themes of your book, A Measure of Failure. And that prompted me to approach you to see if you would be interested in reviewing and giving me some feedback on the basic structure of the Impact Project of the work that had just started. And you said sure, that you'd take a look at it.
So I followed up my phone a little bit later. And that conversation led you to propose a research project. And naturally you asked for multiple types of data and you asked me lots of questions. And then we created I believe jointly with you as lead the design for our initial case study. And that's a great place to start. So if you would begin by telling us about the types of data or evidence, as we often call it, that we used in that first case study, and if you can roll into that response, explaining what propositions are and the basic process for how those components come together to assess the Impact of PAGE's professional learning practices. That'd be great.
Dr. Mark Garrison: Sure. So it's actually good to start with the question of data and evidence. So what we started off thinking about was, what are the goals of the professional learning. And then we thought very carefully about what types of evidence would we see if those goals were being achieved. So then, of course there's data, meaning you go collect information of some kind, whether it's a school improvement plan or interview with the principal or observation of a classroom or whatever.
But the idea was to be clear on what it was that was trying to be achieved, and then to come up with two types of evidence to help us evaluate whether or not those goals were being achieved. And the particular way we did that was by creating propositions, so propositions or statements of what you expect to see if things are working the way you think they work, right?
So as a result of attending PAGE Professional Learning sessions, for example, working on engagement and other themes related to that, we would expect to see this. We would expect to hear teachers talk about their classroom this way, we would expect to hear principals talk about their leadership this way, we would expect to see students doing this kind of work. And each one of those propositions would be stated in the affirmative.
Teachers see their role as collaboratively developing lesson plans with other teachers that are focused on engagement, taking the motives and interests of students into account. Then we would create what's called a counter proposition, which is, we would look for evidence that's not taking place. So you have the propositions of what you expect to see if things are going as planned. And then you have essentially the opposite proposition like what we call the counter proposition, which is the same statement I just made, but in the reverse. We don't see teachers doing this, we don't see evidence of... But we see students doing other things than we would expect.
We hear principals talking about their role or leadership in ways that are not what we would expect after having gone through, and that time it was, I think five years or more of PAGE Professional Learning experiences in school discussions with PAGE staff. That model is called the case study design. And it sets up the standard that you use as many different types of evidence that you can get a hold of, so you're not relying on a single indicator to make a judgment, you're trying to find as many different types of evidence. And when those types of evidence point in similar directions, you have more confidence that what you're finding is actually what exists and not just someone's opinion or your own overly positive or negative bias or interpretation.
We relied heavily on structured focus groups with teachers who had both been through the process, and teachers who were new to the school who had not been attending professional learning. And we asked them similar questions. So the idea there was to compare, if the professional learning was giving rise to the kinds of transformation that was intended, we would expect to see the teachers who participated more speak differently than the newer teachers. If they were speaking in similar ways, then that might be the reflection of some other things, either community culture or past history of the school or concern about the policy context or other changes in the community.
We also interviewed people who were key players, either in terms of the school leadership or in terms of their role in the PAGE professional learning. And that was a very important part of that first study, which was retrospective, meaning the professional experiences and work at the school had already transpired. It wasn't that they had stopped doing it when we got there, but they had been going on for years. So we were asking people to recall. In addition to that, we asked for documents, "Hey, you had a professional learning session on building choice into lesson." Or whatever it is that they had been learning in the PL sessions. And we would say, "Can we have a copy of those documents?"
So we would look at the documents. I don't know if you recall, but there was actually a fair amount of emails between PAGE staff and the two schools that participated in that case study. We could see the process as it was in motion, and you can say that's kind of historic, where we're looking at these documents to see how things unfolded. We also asked both schools for samples of student work, which was often in the case of pictures of things on the wall, or the things students had done and also syllabi or efforts. The teachers had to structure their curriculum, how it looked. And we could look at those lesson plans were like, "Oh, are they building choice, avoiding adverse consequences for students."
We'd look at some of the practical philosophies that people learn in the PL sessions to make the classes more engaging for students. We also looked at school improvement plans. And those are interesting because you can see the link or non-link between what you see in the school, what's in the plan and you can also see, there you can explore the weight of other forces that affect the school. So the school has to respond to different communities, state education departments, federal guidelines and or mandate other types of concerns or questions. Also, maybe they want to focus on their relationship with the community or parents. And so we looked at those documents as well.
David Reynolds: It was quite a comprehensive set of types of evidence or types of data that could really paint a broader picture of what's taking place at a school, in classrooms, interactions between colleagues, interactions between teachers and students, and the type of work or experiences that are being designed for the learners. And in that list of types of evidence or data, you did not mention often during the actual study, or just now, or weigh heavily in standardized test scores as an indicator of successful implementation of practices that are emphasized in the PAGE, Professional Learning sessions.
So given the focus of your book, and what you've learned about public schools in Georgia, as a result of your study here, take just a minute to briefly explain the proper role that standardized test scores can and should play in determining the value of a student's learning experiences. And I read one of your blog posts from a few years back, it was, I believe 2015, and it was titled Metric Morality. You discussed concepts such as ranking versus measurement, and cut scores and concerns with validity and diverting the focus from metrics that truly matter. And I believe you've just listed a bunch of the metrics that truly matter.
So as you now discuss previously, we should really bump that conversation and talk about standardized tests in a different way, not are standardized tests bad, are standardized tests good. But dig into that a little bit more deeply, and show the role that they play. So for now, give us just enough to make us want to learn a little bit more. We just want to clarify that PAGE is not seeking to bash test scores or indicate they have no value. We just want to change the conversation to indicate or articulate the solid reasoning behind, what an appropriate balance is between the measures that feed into and they grow out of effective teaching and learning exchanges.
Dr. Mark Garrison: Well, let me begin by saying that the other piece of data we collected that I didn't mention, was interviews and focus groups with students. There's a great deal of emphasis on testing students and asking them to demonstrate what they've learned through the tests that come from the state, and also teacher made tests. But maybe we don't pay enough attention on hearing what students experiences are.
So it's a general principle, that if you want to understand something as complex as a school, or even let's get smaller, something as complex as a professional learning experience that lasts for five years, and your charge is to, how do I evaluate if this professional learning experiences achieving its goals? What kind of effect is it having on participants? Where is it effective? Where is it not effective? It's unreasonable for any indicator to substitute for the range of evidence that actually exists.
If you think of it as a crime investigation. If you really want to understand what happened, you have to interview multiple people, you have to go back in history, you have to understand people's relationships, you have to know the facts of where people were what time, to understand what was going on, was there really a flight to Venezuela at that time, could the guy really... You need all that evidence to be able to come to a comprehensive understanding.
And so that the general approach we took to evaluating and thinking about, "Hey, what is the value of this professional learning?" So you wanted a range of evidence. So the issue with standardized tests, is that we have a debate. It's really 150 years old actually David. Our standardized test fair, equitable, good, promote equality, or are they harmful to teaching and learning, limit creativity, and so on, and so forth.
So we have that debate, and that debate's been cycling like a broken record for a long time, especially since the invention of intelligence tests. And then since the No Child Left Behind, especially. What we want to do, is we want to start by trying to understand them. Why do we have them? They're a fact of educational life and they play a variety of roles. They do play an educational role. Teachers and principals do look at the items and see which ones students got wrong, and they try and make adjustments in their curriculum and teaching strategies based on that.
But they also play a political role. They're an accountability tool used by federal and state governments, because there's a long standing problem in the relationship between educators at the building level, teachers and principals and the relative trust or lack thereof with the authorities above them. So the whole reason standardized test developed, was to solve a political problem. We don't trust what the teachers and educators are doing in buildings. Is that distrust warranted or not? It's not merely a technical question.
So the other pieces that the reason it's so hard for us to talk about standardized tests, is there actually a technology and our culture is really kind of bad. We have a weird relationship with technology. We adulate it and expect it to solve all of our problems, and we're definitely afraid of it. The robots are going to take over the world. If the handwriting goes away, people are going to become completely illiterate, yet, we want the computer to do everything, and everything has to be in the cloud.
And so we have the same sort of difficult relationship with assessment technologies. Human beings have been assessing their educational efforts in a variety of ways. Doesn't necessarily look like standardized tests. Because they've been doing it for time immemorial. It's a natural human thing to make a judgment about your efforts. How am I doing? Am I doing a good job? Am I progressing in some way? Am I good at hunting? Or do I need more work at hunting? Whatever it is. It's a natural part.
So standardized testing is a very particular form of assessment that emerged at a very particular time for very specific reasons. And until we come to grips and understand that origin, where these things came from, we're going to have a difficult time in figuring out what we should do with them, or what we should not do with them. And I'll just end by saying it's really important for everybody to realize that standardized tests are just a standard. It's a tool for making a judgment, comparison or measurement.
Now, they're not measurements in the way weight or temperatures or measurement, but they certainly are used to make comparisons or assessments or judgments about people. Now, it's the fact that all standards in every field, reflect the values and interests of those who established them. So the beginning point for solving and making productive headway and discussions about standardized tests is to assert that teachers and building principals especially, need to be involved in thinking about what kind of assessments are most useful to be a decision maker in the nature and function of those tests, what they look like how they operate when they're given.
And you'll see that a lot of the tension over testing is really over who is excluded and included from the development of the test in the first place. So it's a technical discussion at one level. What is measurement, what is not measurement? But it's also a cultural discussion, which is who should be involved in setting standards for what students are learning in school and how well they're learning. Parents have used, teachers have used, politicians have used, local business people have used and the fact of the matter is, there's a very small group of people making all those decisions.
And whenever you have that happen, you're going to have some conflict. And so having an educational discussion about assessment, and also realizing that assessment has also gotten itself connected to political and economic concerns. Test scores are related to property values. No one really wants that to happen necessarily, or they weren't invented as a measure of property value, but they become integrated into that system. And so because they're so integrated in these other things that really don't have a lot to do with what goes on in the classroom, they affect what goes on in the classroom.
And so that's why there's consternation, there's a lot of emotion. And we need to start over and sort of say, "Okay, we need a new understanding of what these things are and where they came from. And then we need to have a framework for how teachers and educators can insert themselves in the discussion about assessment in a way that's productive for them as a profession, but also takes into account the voices, motives and interests of the youth." The people in this, the school is there to educate the next generation of young people, prepare them for adulthood.
So what kind of assessment actually serves that aim? We can use that quickly right now as a way to say, "Okay, how well do these tests help prepare young people for their future?" And we could go through and collect data, which would be both the actual data or the test data, we could do comparative analysis to other countries, we could look in the past how other countries have evaluated their schools. And we could look at how well youth are or not prepared for their future activities as parents, citizens and employees.
David Reynolds: I should have given the full title or shared the full title of your book, A Measure of Failure: The Political Origins of Standardized Testing, that would have certainly indicated why you have such a deep understanding of and passionate about the topic. Some of the indicators that some places require to be embedded in professional learning experiences to get quote unquote, credit, like raising test scores, and graduation rates, et cetera, are not the focus of meaningful professional learning that we're talking about today. So when you focus on things instead, over which teachers have direct control, such as quality relationships, structuring a learning environment in a certain way and designing work for learners, those are within the realm of the teachers control.
So when you structure things that way, that seems to have a lot more value for the participant and then have much more transfer to practices in the classroom that benefit learners. So could you talk about that and what you've observed about that type of approach, because people need to also know that you've not just worked in Georgia, you have been working with people in other places. So you have a broad range of experiences from which to draw. So what have you observed about this type of approach?
Dr. Mark Garrison: I'm going to approach it, first of all, what's unique about it, because that also informed how we studied it. The first thing is fairly simple and obvious, but it makes a huge difference is that it's not a one shot approach. It's explicitly developmental, and that people who participate teachers in particular go through a journey of exploring their understandings of students, and designing work for students, and what it means to collaborate with colleagues in a new way and what it means to collaborate with administrators.
Our goal was to really evaluate were the kinds of transformations at that level taking place. So we examined the beliefs, values, norms, and roles of educators. And we had, you and I and our teams, respective teams crafted those propositions about beliefs and norms and values and roles of students and teachers and principals that would be consistent with what we would expect to change or be transformed in a school, if the professional learning was really having the impact that it was designed to have.
And what we found was this, that we didn't set out to do this, it wasn't explicit, but this point about trust. So in the schools, where we found the most evidence that these kinds of practices were being tested out and tried by the folks who participated, and sometimes even shared with people who didn't. Different ways to bring teachers together to evaluate their lessons very quickly, where teachers could have really honest and supportive discussions with each other, "Hey, I tried this, why didn't it work?"
And there's a framework that they learn how to think about protocols, to think about going back and trying the lesson again, but trying different things based on feedback from colleagues. All of those things require trust between your colleagues, your peers, and the administration and the teachers as a more general rule. And in schools where we saw that trust strongest, we also saw more conditions for them to take up these ideas about engagement, about the design qualities like choice or focusing on the actual designing the work for students with their motives, interest and background in mind, so that the student can make a connection between their experiences and world and the values and goals that the society has for what it is that they're supposed to learn.
So that was probably one of the more consistent findings we had across our experience with schools in Georgia. But it's also somewhat generalized with other work that we've seen in other buildings and other school districts. This trust thing, I think we phrase it as creates the conditions for more focus on engagement, both of staff and students. In terms of PAGE, I would frame it as a question of more intrinsic motivation. That is, how do we get it so that students can take their own energies and apply it to what the school is asking them to do in a way where they're like, "Hey, this is interesting and meaningful to me, and I'm putting my effort into it." And there's a sense that the teacher said, "Yes, that's an important thing to work on." And that we really, we saw solid evidence that that occurred.
The more they participated regularly in the PL sessions. So this is still over years, right? It's not, you go one weekend and you're done, and you come back to your school and that's it. This is an ongoing iterative process, the more participation, the more the teachers and administrators started to think differently about, "What do I believe about my role?" And that seems, when you talk to people who aren't necessarily professional educators, what's the role of a teacher? It's what you should know that.
And in fact, it's a very complicated question, because it's how you frame that role changes the dynamic in the classroom and the characteristics of the school. And I think eventually the outcomes, how the students are actually socialized and educated. The third thing that we saw that was really important was the emphasis on multiple opportunities for success. So there's a philosophy that the homework is due, if you didn't turn it in, you get an F or you did the best thing in your project, the grade is the grade. And this is meant to mirror the adult world of your boss tells you the reports due on Friday and the reports due on Friday.
And that has a place and I think a lot of teachers feel that it's their responsibilities to socialize young people to be responsible in the way that I just described. At the same time, you saw transformation saying "Yes, that's true." But when we're learning, when I'm an employee, and I'm going to do the report, it's presumed that I've already reached a certain level of maturity, and intellectual development and emotional poise.
When I'm 14 or eight, I'm still developing. And what I think the value, the third thing that we found was that educators were saying, "You know what? It is important to provide multiple opportunities for success to students in the classroom, and at the whole school level." And we have propositions about that, and we look for that. And we found that the more people participated, the more evidence was. The school and the classroom teachers were saying, "You know what? I'm going to try this out, and provide more opportunities for students to do that."
So the trust, on the one hand, the significance and value of focusing on engagement. And finally, this idea that it's appropriate to provide multiple opportunities for success for our students to "Try it again, give it another shot, maybe you want a different way of presenting what you understand." And so that's a much more developmental and iterative process of teaching and learning as opposed to a more one shot. "You had your chance, that's it, you got to see."
Students need to be evaluated where their work is not appropriate or up to standard, they should be. But that doesn't preclude saying, maybe we can move to the next level and say, "Let's try again, and let's try in a different way." So those are the three things that we observed. And I would say create or limit how much of that stuff happens, has more to do with the policy context, and the way the school district is interacting with that policy context.
David Reynolds: That really is a great segue into barriers or challenges or obstacles to shifting, even incrementally over time ones practices, with students, with lesson design, with colleagues, how you message about what's really going on at school, there's so many things that this could spin off into. And without getting into a lot of policy context, discussion at this point, think about and talk about for a second, some of the factors that are truly beyond the realm of the individual educators influence, what are practical ways that teachers and administrators can do their work in the way that they want to, for the benefit of learning and learners, the best way possible, and the most professional way possible despite, or alongside some of the challenges and factors that are pressing on classrooms and schools.
Dr. Mark Garrison: Well, the first thing, I had a very influential colleague and mentor who told me as just a general life slogan was that, there's always something you can do to improve things. So that's not meant to negate the limitations that exists because of the conditions of poverty, or because of the stranglehold of certain policy mandates over which schools are able to try or do or feel at liberty to do.
And a lot of this turns on how we frame what it is we're saying. So as a general rule, people work very hard, this is a real thing, you're working very hard in these external conditions, create real pressures, and you get to the point where you say, "Ah, there's nothing I can do." What we actually know from research is that schools do mitigate the negative effects of other stuff going on in the society, whether it's poverty or other things that are associated with poverty.
So they don't, they're not capable of solving or eradicating those difficulties, but they do mitigate them such that if we did a thought experiment, if we said, "If the kids are living in poverty, we can't be held accountable for their performance, because the conditions of poverty are so debilitating." That may be true to some extent, especially depending on what kind of extreme or non-extreme situation we're talking about.
But the fact of the matter is, if we removed public schools altogether, things would be way worse. Teachers don't realize the overall effect they have historically, on actually keeping things from becoming more desperate in communities that face a lot of challenges. And so that's why the value of public schools, and the value of the work educators do and teachers in particular is so important, even if the day-to-day experience feels like it's not making a difference.
Actually, if we look at the data over the last... Actually here, we can just use things like test scores and graduation rates and associate them with things like jobs and income. The level of inequality and hardship that people would face would be way worse if we didn't have public schools. Now, it is an illusion to say public schools are going to solve those problems. Now in terms of practical solutions in the classroom, what we've learned from the work with PAGE is that if we take the design qualities. So these are techniques or frameworks for thinking about how to interact with young people, and engage them in the work.
So let's take the simple example of choice because it's really easy to understand and when people get frustrated with as they say, "Well, I have a group of learners in my class, they face some significant challenge at home, they're in fourth grade, but they’re reading at a first grade level. There's a lot of behavior issues, I can't provide those students choice because it's too hard to manage my class." Now the question there is that's right, you can't provide them the same choices that you would provide students who don't have those kinds of experiences.
So the issue with the framework is to figure out what type of choice can I give this particular set of learners such that it will push them forward and enable them to start where they are and moving forward. In some groups of students, especially if they face strong... Their community is under resourced, and there's a lot of needs not being met. And I'm not just talking about healthcare, emotional needs. Some of the framework such as removing adverse consequences, may be more significant to focus on, at a certain part and over the course of a given academic year.
So if the students are particularly feeling beaten down or hopeless, then focusing on getting them to do work and not having adverse consequences is very important. And then building in the, "Okay, let's push them to the next level. Sometimes building choice into that can be scary for people who don't feel competent to activate the choice. It requires a certain level of development and emotional maturity to say, "I can choose what book to read." Maybe that's not an appropriate choice for a set of learners, but a different type of choice. Like, how do I share with the teacher, what I've learned might be something that they feel better about.
So if the aim is to get the kid to read and demonstrate what they learned, maybe making them do the essay isn't the right essay. Making a video or something that the kids are interested in, and then working on the writing part as you move forward. So I think people get frustrated, because a lot of the recommendations seem to be very general. You remove adverse consequences, focus on products. What else? So the other ones are missing, they keep focusing on-
David Reynolds: But there's novelty and variety, affirmation and performance, affiliation, content and substance.
Dr. Mark Garrison: So let's talk about affirmation and performance. So I think one can get frustrated because they see some of the performances wanting. If the students performing at the first grade level in many areas. And in fourth grade, one's going to say, "Well, I can't affirm this, because it's below grade level or it's below expectation." And it's sort of saying, "Okay, but can we find a way for the student to demonstrate some of the learning," however, and then build on that, to raise their levels say, formal writing and English.
So sometimes, if a student gets to talk about what they understand, which is something they're more familiar with doing, and if they're not good at reading and writing yet, then you can build that in. So one of the things that I think we're going to discuss more is, what's different about the professional learning model that PAGE is supporting and facilitating, is precisely the idea that it's a way of thinking about designing lessons that people are being introduced to, it's not a series of steps. First, add the garlic, cook the garlic for five minutes, and then add the onions, right?
It's not like that. It's thinking about understanding classroom dynamics and curriculum development, using a framework to make decisions. And that framework can be applied to any circumstance. So we did find schools that had significant levels of students classified it free and reduced meals, being able to engage in these types of activities. These teachers were able to pull it off in the classrooms, they did the lessons, they made it work in their classrooms, and students didn't behave poorly or weren't able to engage. And in fact, it seemed like it was creating a more positive environment in the school.
Now, of course we also found that the trust was important for that to happen. Trust in supporting relationships, especially with students who are lacking the resources and supports for whatever reason, is very important to them, those relationships are very important to create the condition. So the frame work applied to the concrete condition. So you have to ask yourself, what is it that these students are going through right now? So that's why I think some of the work that PAGE did on the question of poverty was helpful for teachers, because it got them to think about being the students, what are the students needs, and something as practical, as you know, I'm asking them to bring a notebook and they feel bad about that, right?
So then, since that's an adverse consequence that the teacher didn't even intend until the teacher said, "You know, what? I need to reframe my understanding of what these students are living like and what they're living through, and then use the framework to appropriately channel each one of these design qualities at their level, their level emotionally. How much frustration can they handle?" Intellectually, what is their reading level, what's their background knowledge, what's their vocabulary, and behavioral. Do they know how to cooperate? And so if those things aren't all there, then you build those qualities in, practically at the level the student's at, which was of course, the aim of pushing them forward.
Now, is it harder? Yeah, it is. It's probably also, historically speaking, the teachers who do that work, they could never be appreciated enough David, and you and I have witness that together in some of the schools we visited were they're just, that sometimes the teacher is that one positive thing in that child's life or a group of children's lives and the teacher pays the price for that as a professional. And that's why that's why one of the goals we have is to support professional educators and advocate for their professional status, their heightened role in decision making, and that their voices influence policy, because they do really live it day-to-day, they do really have a fairly good set of ideas of what would help improve the situation. And they already have, although in ways we haven't fully documented, helped a lot of people who would otherwise be in way worse situations than they are.
And now we're talking about the negative things. But I think I tried to do two things there. One is explained what was uniqueabout the approach to professional learning that you're not, you know, "Here's how to use the software," if you do this technique with your students in the class, you follow it like a cookbook. So that one thing is that it's different. Because it's different, it's more powerful, but it's also harder to grasp. And it's easier, it's frustrating, right? I have to tell them that the behavior is wrong, they have to have adverse consequences.
I'm not saying there should be consequences for the behavior, what I'm saying is that, when you have a specific learning goal in mind, you want to have it such that the student is able to try and keep trying, and one of the conditions for doing that is that you remove as many of the unnecessary adverse consequences. No, you can't hit other children. No one's saying that. No you can't cheat or whatever happens in that situation that is unacceptable. But you can create using that framework, looking at your specific place based education, that you have to deal with the students in front of you.
That is harder, but we saw examples of where when people use the framework in their classroom with students who faced many of these challenges, you also saw the beauty of humanity. Because kids have a tendency, when they're given a little bit, they can do a lot. And we did find examples where teachers were able to work with students from a variety of backgrounds, they were able to connect with them. Unfortunately, they don't necessarily have the professional authority to make the kinds of decisions that are needed to really push that forward. And I think that's one of the outcomes we both hope from this work is that we can change the conversation a little bit about accountability in public education, such that the proper and dignified place of professional educators and classroom teachers is centered to that discussion, and respectful.
David Reynolds: That reminds me that I should definitely mention that PAGE does not view its work as the silver bullet. First of all, we don't consider anything that we do a programmatic. There's a big difference between frameworks and programs. And that's a topic for another day. But this is not a silver bullet, nor do we think that we've landed on the set of processes. Even inside this research, we've made changes over time. In fact we've moved away from the historical case study approach that you described at the outset of our conversation. And we've shifted that a couple of times. And there are reasons for doing that. So I wonder if you would be willing to talk for a moment about the shifts and the processes that we've used Mark to capture the evidence, historical case study versus real time as it unfolds, network versus, not versus, but it's been a shift to the current emphasis on individual educators through learning visits?
Dr. Mark Garrison: Sure. Well, let me begin by thanking Phil Schlecty for one of the most important things he said in terms of my own thinking about education. And he said, at one point in which book I don't remember, but it is very simple. But it can be used for a teacher or actually anyone working with people to get started in a good way and solving a problem, which is that it's the activity of the learner that causes learning. "Teachers don't cause learning, they create the conditions for learning."
And so that going back to the example that I just gave, and then crediting Phil Schlecty and his team, and the Schlecty Center with the work and efforts that they're doing to elaborate and involve people in that kind of discussion. It's a very important point, what causes learning is the activity of the learner. So then everything is geared around activating the learner, and not in the sometimes silly way that you hear, but activating learning a profound way, the way that I think is emblematic of the kinds of sessions that you folks organize.
Because of that, we did the case studies which were historical by nature, because that's when we connected with you and you wanted to see if we could design something that would explore what the effect has been. If it was making a difference. And we selected two schools that we thought were similar enough in terms of their participation, but unique enough for us to learn something about what are the factors involved that create the conditions for professional learning to work. We ended that report thinking, "You know, it can't just be at the school level, it should be the district."
And we had this initiative to have networks of districts joining together in professional learning and that was really an educational experience. And we designed, because we're dealing with a much larger group of schools, we had surveys. Now surveys are great, but they tend to be global. And so we learned some things in the general sense about network participants and what they were learning and what was going on in their school. And some of that we could attribute to PAGE Professional Learning experiences. We also realize that there's this larger context, again, there's the social context which we can talk about the economic, social and cultural conditions of the community that a school serves. And we can talk about the policy context and legal political and governmental contexts in which a school works, whether it's a school board or policies from the federal government, or whatever.
And that network experience of a district signing up to participate sensitized us to how hard it is for a district to put their weight behind one single initiative. It's not a criticism, that's a larger discussion for us to think about with policymakers of how we can create the conditions for districts to have a more unified focus based on their own needs assessment of where they are, where they've been, and where they need to go.
And that led us to our current initiative, which is to go back to the classroom, to get to the center of the teaching and learning where we survey students in the teacher's classroom, we observe the teacher in the classroom, we take notes, and we have a system for making observations about that. We then meet with the teacher to debrief about that experience, and it's not evaluative, it's sort of like, tell me how you were thinking about your classroom that day, what was going on that day. And in doing that, we learn about how the teacher is thinking, what their needs are, what their concerns are, what strategies they developed at work.
And we can link that to their professional learning opportunity, right. One, we can see where teachers are actually using things that they've learned in their context, not just filling out in a survey, "Yeah, I do this, or I don't do that, or I work with teachers this way. I don't work with teachers that way." Here, we're focused on really getting in the classroom, because one of the findings we had was that while there was a general change in beliefs, and reconsidering roles that we didn't, we didn't have enough evidence about the particulars, how in particular, are teachers working in their classroom? What kind of things would help them? What kind of things are they not finding helpful?
So by going into the classroom and debriefing with them about a real classroom experience, looking at how students responded to a survey with the students voice, and taking all of that, and using it to both evaluate with what the professional learning is providing or not providing, but also enriching our understanding of the professional work of educators in trying to be, I think in and soon we'll be in a position where we can start to present that for both educational purposes, but also policy purposes. These are how professionals are actually working and making decisions. These are the kinds of supports they need, these are the kinds of successes they're having. And that will improve, hopefully, the conversation about the policy context. But hopefully, it will improve the conversation about what kind of professional learning is meaningful and useful to educators, and really puts their voice in the center of the question, what's the value of this professional learning, right?
So we're observing them during their work, we're talking to them about their work, we're asking the students in the class about their experience, maybe we're collecting some artifacts. Sometimes, as you know, we have a follow up interview because of the discussion with the teachers is so rich. Teachers actually, like the ones that we practice this with David, really liked the experience, that experience is not filling out some survey, if you don't, which everyone's fatigued by. It's really a chance for them to discuss and reflect on their work. So that's the quick overview that history of how we started the case study with schools retrospectively and ended up with a focus on the classroom, the teacher and the student and the kinds of nuance and the decisions that are making to inform the professional learning and learn about where it's making a difference.
David Reynolds: The focus on or the focus at the classroom level and on the individual teacher, as the place where our research is now, is definitely appropriate and the right thing to do. I mean, teachers matter in this process of teaching and learning more than anything else. And so when we get to have one on one interaction with them, it really enriches our understanding of what they do and what they need to support their work. And I want to clarify or expand on something you said in that response where you talked about going into a classroom, conducting observations, taking notes. If we leave it there, although you did expand on it some, I just want to add to that. That sounds initially to people very traditional and very evaluative.
Although you did say, it's not evaluative, I want to emphasize that the learning business that we conduct are truly not evaluative. We call them morning visits, because we are learning. And as soon as the visit has ended, then we capture our thinking about that particular visit. And it's really in a very simple format which is, what did I actually observe? What did I see and hear what was happening? What did I learn as a result of being in that space with those people for that time? And, I believe the pivotal one is, what do I want to learn more about?
So when we go back and speak with the teacher about that day, about that lesson, that experience we focus on, here's what I want to learn more about, and the teacher gets an opportunity to share his or her thinking about the development of the lesson, what they might do differently, why they chose to do what they did. And it's so different from any other type of conversation after someone visits your classroom than most people are familiar with that I think it's really refreshing. One person, he made a reference to this. She said to me, "I've been in the classroom for 18 years, and this is the most feedback I have ever received."
And I didn't really view the discussion we were having as me providing feedback, it was simply, "Hey, here's what I learned, tell me more." And it was just seen as extremely, extremely valuable. So I think we're onto something by spending time looking at the classroom level, and the individual teacher, what the students are doing in that classroom and why? And so I think that's really, really valuable. And that our shift and how we're looking at our work is appropriate. And another thing that you mentioned multiple times now is policy context. So let's linger there for just a minute. What would you suggest to policy makers regarding work that they can undertake to support these efforts?
Dr. Mark Garrison: Well, immediately one of the things we noticed by looking at the school improvement plans was the pressure that schools and districts felt to check out various boxes in Georgia, for example, it was the CCRPI right? Like if I do X, Y and Z, I get more points. If we use this language of choice that in some spheres, legislators are favorable. Like they like charter schools and choice there. So the thing would be to, okay, you want to have this accountability metric. I'm not going to argue that right now. But how about we build in choice to how those achievements are completed.
Let me give you one example, that would be very easy for a legislator to approve, and I don't think it's controversial at all. So one of the things we noticed was that schools get bonus points for signing up for PBIS, Positive Behavior Intervention systems that are promoted by the US Department of Education. If schools want to do that, that's fine. If the legislator said, "You don't get five points for PBIS, you get five points for dealing with issues of student motivation in your school."
As long as you can demonstrate to us that you're signed up for some sort of experience, that's to help you foster positive... So PBIS is about fostering positive school climate. Great. There's a range of programs, initiatives and frameworks that have been developed to create positive school climates, PBIS is one. So why not allow choice so that districts can align their particular school climate work with other things that they're doing? If they're doing a reading program, and it's not aligned with the motivational frame of another initiative that they have, or it's not aligned with overall district goals, but they're doing it because it meets some requirement, create an accountability system that allows the school district to demonstrate that they're doing the category, school climate, school safety, academic recovery, STEM, whatever it is that you get points for, and allow them a range of ways to do that. And encourage them to make it coherent.
So one of the things we saw is, schools you felt so bad, because they're struggling, they're signing up for this and signing up for that. But the two things they signed up for are kind of in conflict, pedagogically, they're kind of in conflict. I'm not even saying one is better than the other. It's not healthy for an organization to have conflicting programs and initiatives. So why not create an accountability system that says, "We just want to see evidence that you folks are trying to work in these areas that you're identifying problems and that you're crafting solutions. We think one solution is this PBIS."
Fine, but there's some others. If you can demonstrate that it's real, that it exists, and that you have fidelity to it and you're assessing it to see if it works, then you get credit for trying that. So let's take this idea of choice and build it into these accountability frameworks as an initial step. Now that's not very radical, I don't think it's very controversial. The whole choice idea in policy circles is very big. Districts can opt out of certain mandates, if they can demonstrate they're doing something well.
Why not expand that notion into some of the criteria that are on state accountability systems such that schools can have the opportunity to be more conscious about aligning all of their initiatives that point in a similar direction. Forcing them into one model, and having no other choice, probably isn't going to solve the issues that we face, because the problems that schools face vary by locale, in the abstract way, they're very similar. But in concrete terms, there's different histories. And different solutions are needed at different times for different buildings in different districts. So you can't expect one, there is no silver bullet David, like you said. There's no silver bullet, let the educators have some authority to make some decisions within frameworks that a legislator and policymaker can understand.
You have to be working on your school climate, you have to be , you know you've got an anti bullying initiative, whatever it is, as long as you can demonstrate you're working on it, and it's consistent with the direction your school is headed, that would be helpful to schools, and it doesn't mandate everybody do the same thing. Everyone doesn't have to believe in the same techniques of classroom design or student motivation. But it would create the conditions for those who want to try different approaches, the legitimacy to do that, and encourage them to assess it.
And if it is working, they can share that with the legislature and the policy people in the state ed, and they can share it and it can become, "hey, this is something that's worked in these three districts, why don't you learn more about it." And I think that would be an initial start. Like I said, it's small, but it creates the possibility, it takes the pressure off of districts to check off boxes to meet mandates, and allows them to think a little bit more carefully about their direction, where they're headed. And then the kinds of initiatives that they want to engage in including professional learning that match that.
And so that everything is pointing in common direction, where they've been, where they want to go. And we know that that's part of being an effective organization, I don't think there's too much question about that. If the organization is pulled in 14 directions, it's going to struggle. So the state policymakers, legislature has a responsibility to create the condition, so that schools can focus on the direction that's needed for them based on their careful assessment of where they are. And they should help with that.
David Reynolds: As I hear you respond to that prompt about suggestions to policymakers, the accountability system that you suggest, seems to me to evolve again, around the notion of trust. So current accountability systems seem to avoid an emphasis on trust. There seems to be more of a checkbox type of thing, and it seems sort of counterintuitive or hypocritical almost to be talking about places of learning. And giving students choice and not doing that at the organizational level, just seems to be misaligned, the efforts and the language and the tools and the ways that we assess and measure and hold people accountable.
I don't find that educators are anti accountability at all, I think the entire process of teaching and learning is completely about accountability. Here are my learners, here are my students. And here's where we need to go. And I'm going to get them there. I think that's accountability in a very significant way. So I just see a big disconnect and I think trust is-
Dr. Mark Garrison: Well David any teacher knows you walk into your classroom, and you're not accountable to the students in the appropriate way, you get eaten alive. You have responsibility and when you're not fulfilling it, you sense it in the classroom, it's visceral. If you can't do your job, the students know. Right?
David Reynolds: Sure, sure.
Dr. Mark Garrison: The second thing is that for policymakers, if they say, "Well, we don't trust that your school climate, or school culture work is appropriate. So we're going to make you do this?" My response would be like, Look, there's no evidence that the lack of trust has served anybody. It hasn't improved schools. There's not a lot of evidence that any of these things by themselves really make a big difference, right? Does PBIS work or not? In some cases, there's some evidence that it does this work means that there's less behavior issues, and some places it doesn't.
And there's the building in of the non-trust has not served to improve the system in a personal relationship, or in an organization or in a political government relationship. Once that trust is broken, it's hard to reinstate, but I guess the minimal message was is that I think the time has come where we need to try a different approach and we at least need to give that a chance and at least have conversations about "Okay, well what kind of checkup mechanism would you like?" we want to have choice in this respect. What kind of check up mechanism would make you folks happy given that you don't really trust us, which we're not even going to argue about anymore. It's just the fact, there's a lack of trust, fine. There's a lack of trust on both ends.
Teachers don't trust the publishing companies who make the test and there's reasons for that. And lawmakers, maybe don't trust the teachers, maybe there's reasons for that. So we have to actually take that problem on by, step by step working on. So the instituting more lack of trust isn't going to solve any of the problems that we have. And the fact of the matter is, the system is too big, too complex and too varied, to be managed, like a command and control. Like we said, education takes place, ultimately at that classroom level and no matter how much surveillance and policy pressure and mandate you want, that classroom is where it really happens, and you can't have someone who really excels in the classroom, if their hands are tied, they can't do their job.
The nature of teaching is you have to have your hands untied. Otherwise, you can't be a professional and you can't do your job. So we have to figure out together in a professional way how to solve that problem. But I do think our work has suggested that trust is an important part of the solution. There's other parts too. But policy wise and organizational wise and classroom level wise, I think it's important to advance that discussion.
David Reynolds: What is it that you have learned about teaching and learning as a result of this research effort?
Dr. Mark Garrison: Before meeting and working with you folks, I had a relatively broad, and maybe even dogmatic negative view of professional development, and that change in language was as a result of working with you folks. And it's just the teachers, the experiences that I saw in various professional development activities of either districts, or third party vendors, or the state, were generally, very unappealing, and I thought not only ineffective, but in some ways harmful.
So the first thing that I've learned is that there is an alternative, and then it's a very valuable alternative. And this isn't to say that it's the only alternative. But because it is an alternative, it opens up space to think about professional learning in a different way. And it thinks about the professional learning being, we've tried to have it as an iterative process. We have the research arm, we have the professional learning team, we have worked with the Schlechty Center. We have our personal dialogue things have all contributed to a richer professional development experience, both for the participants, but also the people who read and learn about the work that we're doing together.
So that's the first thing, that there is an alternative and it's, one doesn't have to have this negative idea about professional development, that professional learning is extremely valuable and important, and can be very affirming, not only feel good affirming, but affirming meeting like the professional part of you is growing and advancing with your colleagues.
The second thing is that the value in that professional learning of having broader discussions, so one of the things I know that first, participants are sort of might say, "I don't really understand what's being said, I just want to know how to do this in my classroom." And, of course, that's a positive instinct. Teachers, they have practical demands that they face every day and they want help. But I've seen the value of what some people might call these more philosophical discussions, because when we did those interviews, teacher after teacher after teacher told us, we're the ones that have the most experience with it and really tried.
They said, "After that first year, I started to get it. I started to see the value of thinking this way about teaching and learning." And it made me see that, there's a perception, there's the lack of trust in teachers. And then there's this perception that teachers are just practical people who implement lesson plans every day. That's it. And they're not. They're thinking smart, philosophically, culturally rich people who've been given the chance really flourished. And some of the best education moments I've had are in those focus groups with teachers who participated and explained the transformation that they went through.
And there's that one time where, or a couple times where the teacher has said, "I was about ready to retire, and this is now the third phase of my career. And it's the best because they were renewed because of these ideas, and ways of thinking about teaching and learning." Some people said, "If I had had this 20 years ago, my career would have been very different and probably more productive." And so the intellectually rich, but yet practically significant part of this professional learning, really affirms the value of that work. So that there's a pressure like "Just tell me where to put the sticky notes." And sometimes that's needed. But I think what would it affirm for me is that, thinking about the work we're doing together, and creating conditions for that thinking is a really valuable activity and even if it's frustrating at first, because it seems nebulous, it really is transformative.
And it also provides a sense of hope, because people are in the position to be more active thinkers and active participants in their own routines. We get caught in our own routine. So a lot of this work asks people to reflect on what are those routines, what do parents actually think of your school? What do the students? How do they actually respond to your lesson, and once you become open to that, you start having very rich discussions about teaching and learning. And I saw that emerge for people who were pretty committed to the process and put in a good effort.
But those kinds of anecdotal things were pretty consistent across people who participated a lot and that gave me hope. So it's not all, there's nothing we do or the policy direction is the wrong way. It's like, "Yeah, there's some problems, but there's also some good things going on, and there's a chance for people to participate and really demonstrate the value of the work that they're doing and grow professionally." I think this is not only about public education, it's about affirming and renewing the profession. And I've learned that that's possible, that that's actually taking place, and then that's impart where hope lies.
David Reynolds: Is there anything else you'd like to share? Any burning issue that we haven't already addressed or said we're going to park that for another discussion. Anything else you want to add?
Dr. Mark Garrison: No, I think I'm done. But I appreciate the offer.
David Reynolds: Okay, you know when we asked that question, the focus groups, we often get no,-
Dr. Mark Garrison: The best answers.
David Reynolds: Except, and then they keep going. So here's the last question then, what next? What's Mark's encore? I know you're writing a book right now, and maybe it's too.
Dr. Mark Garrison: Well, I've got two projects, and they're related. One is the book and the book is called Skinner's Ghost in The Smart Machines. And that's a reference to B.F. Skinner, the famous radical behaviorist psychologist of the 1950s and '60s. It explores the motivational psychology that's embedded in emerging educational technologies and social media, learning algorithms, big data learning algorithms, the platforms that run Facebook, Netflix, Amazon, and how those platforms are being imported into education.
And so it looks at how these things are premised on behavioral psychology. And it explores the origin of that. And the implication of that, looking at how these influenced when we talk about 21st century skills, what do we actually mean by that? So I assure that there's a very particular notion of skill, talk about when there's this discussion of social emotional learning, what's actually being promoted by that personalized learning? What does that actually mean in the context of these learning algorithms and digital technologies, development of that thinking going all the way back to Pascal, and examining how that has been mixed with what's called artificial intelligence. Learning and artificial intelligence is what makes learning analytics work, that's premised on what used to be known as cybernetics.
And so the first part of the book traces all of that. Now out of that, comes a methodological critique. And so I'm working to develop an alternative methodological approach with some other people who are far smarter than I am, that solves the problem of measurement in education by not mandating that we measure anything, but creates a very rigorous framework for analyzing patterns and data without making assumptions about that data or inferences that are inappropriate about that data. And it's a philosophically rich, alternative framework that I think shows a lot of promise. So I'm pretty excited. I mean, I could talk forever about those two things. So I better stop.
David Reynolds: Yeah, I was getting ready to say, and I will say it now. That just a minute ago, you said no, you were done talking. And then I asked you that one question, there you go. So we don't even have to buy the book now. That's beyond the cliffnotes. No, that's, that's really interesting. I think the title is a definite hook. And then this whole notion of how you rolled AI and those algorithms and looking at social media into what's actually happening every single day, in a classroom level when you get right down to it with testing and other ways that people are gauging the quality of a school or learners experience I think is really fascinating.
So, Mark, I want to say thank you again, very much for giving us a glimpse into PAGE's research, from your perspective and for providing us with some very thought provoking material on what really matters most in teaching and learning. We greatly appreciate it.
Dr. Mark Garrison: Well, thank you David. A very significant part of my professional career has been my relationship with you, Marta and the PAGE team and the Schlechty Center folks have had a huge effect on me. And I feel honored and lucky to have participated.
David Reynolds: Well, thanks again and have a great day.
Dr. Mark Garrison: All right, take care.
David Reynolds: Thanks for listening today. Find the Lead. Learn. Change podcasts on your search engine, iTunes or other listening app. Leave a rating, write a review, subscribe and share with others. In the meantime, go lead, go learn, go make a change, go.