“It is possible to improve results in Latin America without stigmatizing anyone.” Interview with Rafael de Hoyos, Mexican researcher and education expert.

Interview by Pablo Torche and Catalina Godoy

❝ What our studies show is that, if you have this shared responsibility approach, you not only make everything politically simpler, but you can also change the results in the very short term. ❞

 

 

1️⃣ | In your research you address the issue of delivering learning results to schools and accountability strategies, how important do you consider the delivery of results to schools for improvement?

 

It is fundamental. Nothing else is important, it is fundamental. It is the starting point for the design of all other policies. In fact, if you look at the historical perspective of education systems in Latin America and to some extent around the world, until the year 2000 there was a balance where the three major players in the education system were aligned towards the same objective of increasing coverage. On the one hand, the governments, on the other hand, the teachers’ unions and, finally, the parents. And if you look at what happened with educational coverage in Latin America between the end of World War II and the year 2000, it is impressive, it rose at exponential rates. But this gigantic growth in coverage necessarily implied a sacrifice in quality. Sacrificing quality is mathematical. The objective was ‘let everyone in, and then we’ll see what we can do about quality’. And since nobody measured quality, that was the adjustment factor. It was not until the year 2000, among other things with the PISA test, that this balance was broken because one of the three parties, the parents, began to say “hey, wait, but I take my children to school to learn, and it turns out that they are just warming the bench because they don’t learn anything”. And then a negotiation is generated to find a new balance. The equivalent in Mexico would be the ENLACE test, the first ENLACE 2006 census test. And then, every time the results of the test came out, they reached the front pages of the newspapers in Mexico, and that made the discussion of the quality of education and learning reach the tables of the average Mexican. And then the need to carry out reforms to guarantee learning was generated. If you look at it from this historical perspective, having standardized tests is fundamental. It is an essential part that can trigger a lot of reforms.at least in the context of Latin America, where we had that previous balance that I have just described.

 

2️⃣ | The usefulness of learning tests is therefore key to generate concern for educational quality and trigger reforms. But there is an academic discussion that delivering these learning results per school would be more effective in developed countries. In contrast, in developing countries the impact of standardized assessments for improvement may be less. What is your vision in this regard?

 

There are several things to take into account. The first thing is that the identity and the use given to standardized tests has varied greatly from region to region and from country to country depending on their level of development. For example, the No Child Left Behind from U.S. Department of Education used standardized testing as a mechanism to hold schools and teachers accountable, and if they failed to live up to expectations, in some states they literally closed schools. I believe that The assumption behind this paradigm is that the teachers or the teaching staff together with the principal, that is, the schools, really do know how to improve learning and all they need is for someone to call them to account. And that is enough to improve results. I believe that this paradigm, although there is evidence that shows that it may have worked in the case of Florida or in the case of Texas, is a paradigm that is not only closed but also politically very complex in the reality of Latin America. I think that politically it is very complex to close schools because they are not giving the right amount.

 

3️⃣ | Why do you think this paradigm is not adapted to our region? And how do you think the issue should be addressed?

 

Well, because when you have a great inequality, when you have in regional terms 40% of your population living in poverty, you cannot assume that managers and teachers are going to have better results simply because we demand more from them. In other words, it’s almost like assuming that they are just being lazy, and therefore it’s enough to tell them “Hey, get your act together”, and that’s enough for the children to learn more. Well, I think that this in itself is a paradigm that has an assumption that does not correspond to reality. I believe that what we need are tools to identify what our children are learning, what they are not learning, what are the areas of knowledge in which they are making the most mistakes and then, with this information, tell the school “Look, this is not to make rankings, nor to look for culprits, nor to point fingers, nor to fall into ‘shaming’. No, no, no, no, on the contrary”. It is to say “Look, we know that these are the areas of knowledge where we are failing and the education authority is taking part of that responsibility”. To say how do we transform this into an improvement plan, because it is not about finding the guilty parties, because the paradigm is not that if I demand more from the schools with a little piece of paper that says look, they are failing 50% and next year I want this to be reduced to 40, otherwise I will close the school.

 

“One has to go to the school and tell the principal and the teaching staff here are the results, so that the results reports have an impact on learning, they have to reach the only place where learning is generated within the school, which is in the classroom ❞

 

4️⃣ | Based on the study conducted in Argentina, for example, where it was shown that the delivery of learning outcomes does have an impact on improvement, based on the studies you have conducted, what paradigm do you think would work in Latin America?

 

What our studies show is that, if you have this approach of shared responsibility, not only do you make everything politically easier because you don’t generate that confrontation with the teachers and the manager, but you can also change the results in the very short term. In the case of Argentina, one has to take into account what the context was. When we applied this in La Rioja, the context was that there was a test called ONE (Operativo Nacional de Evaluación), which began as a census applied every one or two years. But then it ceased to be representative at the provincial level and no longer had a psychometric theory that would allow you to compare the results from year to year. Then, some states such as Salta, La Rioja, Córdoba and Mendoza wanted to have a standardized test, and what we did in La Rioja was to manage very well the message of why we want a standardized test. Why? For some reason the use of standardized tests, at least in Latin America, is associated with a political affiliation. In other words, having a standardized test is considered neoliberal and then a lot of narratives emerge, saying that it is there to remove teachers, to close schools, to privatize the educational system and what is in the collective imagination. So what we did in La Rioja was to announce that we were going to design this standardized test together with the Provincial Government, but that we were going to do the design with the teachers of La Rioja. So it is the same teachers from La Rioja who make the items of the standardized test, which are then developed by a psychometrist, etc. So it is not the test that comes from Buenos Aires, or from the United States. No, it is a test of the Riojanos so that the Riojanos can take advantage of this. The other big message is that there was not going to be any ranking. These results were not going to be published in the newspaper, the local press, but we were going to make a report of the results together with the teachers of La Rioja so that they could say what was most useful to know.

 

5️⃣ | According to the research in La Rioja – Argentina, you found that the provision of information improved the performance of language learners and also brought about changes in school management and teaching in the classroom, can you tell us a little more about how this is explained and the importance it has?

 

I believe that the explanation of this impact is related to the asymmetry of information that we are breaking in teachers. Because before we applied the standardized test, the teachers and the principal had no idea what percentage of their students were achieving the minimum expected learning. I remember doing several focus groups with principals where I asked them what they did to identify the most important challenges of the school, and I remember that none of them mentioned learning, none of them mentioned hard data. Therefore, the delivery of results immediately triggered a discussion among teachers and directors that had an empirical basis and also allowed them to set clear, measurable and often attainable objectives.

 

❝ Our research shows that results can be improved in the Latin American context without stigmatizing anyone ❞

 

6️⃣ | What were the main lessons and strategies derived from this study in relation to the effective use of learning results for improvement?

 

What is very clear is that it is important to take the results to the schools, not to assume that, because they are in a portal, the schools will download them and use them. That’s not enough, that’s not going to change things. In other words, one has to go to the school and tell the principal and the teaching staff, “Look, here are the results”. And for results reports to have an impact on learning, they have to reach the only place where learning is generated within the school, which is in the classroom. Second, we have to say how the results can be used, how they can be interpreted. And third, these results can become sufficient input for an improvement plan. If you do that, then you can generate a link between your report, the results and the classroom, and then we can talk about a positive relationship between the standardized test and the results.

 

7️⃣ | In other research they conducted in Colima, Mexico, they also showed that reporting results, followed by a fairly light support strategy, could generate results. Could you tell us a little more about it?

 

That’s right, in Colima the results are very similar. It was a strategy, let’s say a very superficial intervention strategy, where we gave the results to the directors and told them that we were going to transform these results reports into an improvement plan and that’s all. If I remember correctly, it was three hours of sitting down with the principals for a diagnosis of results and the responsibility for generating the improvement plan was the responsibility of the principal with the teaching staff. It must be said that the elaboration of improvement plans had a long tradition in Mexico, and in Colima in particular, starting with a program called Quality Schools Programthe message we sent was that the most important variable to consider in this plan was the learning results.

 

8️⃣ | In both cases then, in La Rioja and in Colima, do the results seem to advocate this model that you have called “shared responsibility” rather than a more punitive accountability?

 

Yes, I definitely believe more in shared responsibility. I believe that this research shows that results can be improved in the Latin American context without stigmatizing anyone,without targeting anyone and also from the point of view of political pragmatism, because why go through conflict if we have had so many conflicts with teachers’ unions. It is not productive to point fingers to one side or the other, nor to try to find someone to blame, I think it is much healthier and at the same time more efficient to simply say this is our reality, it is a very complex reality, a reality where half of our kids are not learning what they should be learning and how can we change this reality.