A few years back I went for a pizza with an old friend. We shared a pretty large pizza but somehow ended up with just one slice left between the two of us which we both desperately wanted. Bearing in mind that we would both happily lie, trick or outright fight each other for the last slice, our ensuing discussion about how to apportion it ended in a stalemate, with neither of us agreeing on a compromise. At this point, my friend suggested a solution: I would cut the slice in half, and he would choose which half to take.
Brilliant. It was in my interests to cut it as equally as possible, guaranteeing that we would both get those last few mouthfuls of now-cold pizza. Because I didn’t know which half I would get, the only logical path was to cut it equally.
Without knowing it, my friend had stumbled across an incredibly important theory from political philosophy: John Rawls’ Veil of Ignorance. The Veil is a way of distributing the goods and resources of society equally. Essentially, you decide to structure society – its rules and its norms – before you know which role you will take within that society. So you wouldn’t set up a society where 20% of citizens are slaves, because you don’t know if you are going to be a slave or free. You wouldn’t set it up under patriarchal lines, because you don’t know if you will be male or female. You would include access for the disabled in infrastructure, equitable toilets in public spaces and social support for those struggling with ill health as who knows – perhaps you will be disabled, will want equitable access to a toilet or will have poor physical or mental health. Not knowing if you were to be gay or straight, would you not legalise gay marriage? This is how Rawls argued we should set the rules and mores of our society.
It’s a great thought experiment for a couple of reasons. First, because it forces you to think of the Other. Normally whenever thinking about such charged topics it’s difficult to divorce our private personal experiences and histories from our public opinions about the structure of society and its laws. Middle class, working class, black, white, gay, straight, atheist, Jew or Christian – it forces you to anticipate everyone’s needs. You don’t know who you are going to be, so you have to think about what it must be like to be absolutely anyone else. If I were to be that person, what would I want? What would I need?
The second reason I think it’s important is that it forces you into a radical re-imagining of what things could be like. If you started from scratch, if you hit the “delete” button on everything you had: every rule, every policy, every law and every structure – what would the world look like?
You can play this game with smaller entities than entire countries too. A school would be a prime candidate. Most schools I know have over the years built up a huge amount of clutter: “new” policies built on the remains of old policies, departmental inconsistencies, cultures that are a messy patchwork of hearsay, experience from other schools and barely remembered CPD. School life and culture is an often incoherent amalgam of historical detritus: the debris left in the wake of any large organisation strapped for time and cash.
Such an environment is ripe for the question to be asked: what would it look like if we could start again? What if we could de-clutter the detritus and start afresh? The Veil is perfect for this.
The Veil is helpful for another reason too: in my admittedly limited experience I’ve found that different people in a school don’t really know what their colleagues do most of the time, and therefore cannot understand or anticipate what they need in order to do their job effectively. Teachers might not understand the needs of the head, the head might not understand the needs of a TA, a TA might not understand the needs of the School Business Manager who in turn may not understand the needs of the parents. Students, parents, governors, heads of department, site staff: all fit into this mess of conflicting needs and agendas. Each of these people fulfills a vital role in a school, and the needs of each must be considered when designing policy.
I’ve been reading a lot of articles about school use of data recently. I don’t consider myself an expert. I know some people who really are, and I’m not one of them. I’ve asked a lot of questions and I’ve read a lot of articles on the topic but I’m still way behind. I do think that data would be an interesting case study for our Veil thought experiment, and as many schools are thinking about their data policies the Veil could prove a useful theoretical framework through which to assess and adapt school data policies. The framework would boil down to two simple questions:
- In terms of data, what are the needs of the different agents in a school environment?
- Starting from scratch, how could we design a system that caters to all their needs?
Below are my tentative steps to addressing these questions. I’m not going to answer them directly, but I am going to point to a number of areas that I think are worth considering. With Ofsted announcing that they will no longer look at internal data, there has never been a better time to radically re-imagine your school’s data policy. I haven’t linked to everything that I have read on the topic, but I have collected a number of articles here.
Do you trust your staff? If you’re the one writing the policy, do you trust what your staff have to say about your students? Maybe, maybe not. But sure as hell if you were a teacher you would want to be trusted. And that’s the point of the exercise – you don’t know which role you will take. So you have to build a system predicated on trusting your staff, and providing them with the means to make judgments which they are confident in.
2. Who needs what?
People need different things from the data. If you decide that your deputy head needs detailed sub-levels colour coded and subtracted from target grades that’s fine (though I wouldn’t recommend it), but it may not be what the parents need. If you were a parent of a child at your school, what do you actually need to know from the school? Different roles need different things.
I managed to get to point 3 before talking about workload. As a Head or Deputy with only a handful of classes, it might feel reasonable to ask for three, four or five data drops a year. But if you were to be a frontline teacher actually implementing this policy with 14 classes…maybe not so much. Equally, frontline staff need to appreciate the need for strategic planning and preparation. If you were going to be a deputy head, you would want to have some kind of information about the performance of students in their various subjects. A compromise – from behind the security of the veil – must be made.
4. Won’t somebody please think of the children??
Flippant, but apt. Remember, you don’t know what role you are going to have. You might be a student who really struggles at school and, four times a year like clockwork, gets told that they are under target. Or that they are on target, but their target is a fail. Not so nice. I know I wouldn’t want to be that student.
All roles in a school need, at some point, to be accountable for their performance. I don’t think many would disagree with that. I want to know that somebody is keeping an eye on me, and will haul me up if I do something wrong or stop performing. But one of the roles in a school is “teacher with tough classes” or “teacher who ends up with bottom sets” or “teacher who has inherited a class who know bugger all.” These, and any number of other types of teacher must surely invalidate the use of data as a performance metric. It’s just too complicated. There are too many variables – too many things outside of the teacher’s control.Even more so, I also might not be a teacher but the person looking at the data, at which point I want to know that it is uncorrupted and gives me information about student performance, not about how teachers are playing it safe so as not to provoke a passive aggressive email from their line manager or a request from on high to put in more interventions for students who routinely mess around in their classes. If accountability is desirable, but based on data will corrupt and become self-defeating, maybe find a different route to accountability?
6. Infernal inferences
Data is just a string of numbers: a test score, an average test score, a residual or whatever. You are using them to infer something about something. So you might use a low mock exam result to infer that a student did not do enough work for that mock. But what if the student’s dog died the day before? Or they worked, but their methods were ineffective? Or their teacher hadn’t adequately prepared them or this or that or any number of things. It can be a dangerous games making inferences, and none of us want to be in a role where we are making bad inferences or, perhaps worse, to be in a role where bad inferences are being made about us.
7. What’s next?
Similar to inferences is the role of action. What is going to be done about this data? Do you want to be a teacher who has to spend hours each year entering data from which nothing will actually be done? Doubt it.
8. Aggregation is the friend of reliability…
A phrase borrowed from Becky Allen, it’s an encouragement to think about how many data points your inferences are based on. In short, inferences from one test are not that reliable. Two tests: more reliable. More students, more tests = more reliable. As Becky points out, you don’t need an actual test for this. Any time a teacher looks at a student’s homework, or listens to their answers in class they are taking mental data on that student. All of that counts.
9. …but more crap tests will just give you crap data
So this is where curriculum and assessment comes in. What are your tests testing? Are they doing it well? In the past we used tests which weren’t well tied to what had been taught, had confusing questions or any number of other technical flaws. You don’t want to be a student sitting a test which doesn’t test what you learned, and you don’t want to be the teacher that wastes their, and their students time, like that. So if you are going to spend time on something, whatever your role, maybe just say “right no data drops this year, but everyone is going to give 15 hours of their time across the year to making sure the assessments they are using make sense.”
10. You’re all bright and sparkly now, but remember you could be me
At a conference a little while ago I saw a head of department presenting their incredibly complex and detailed RAG sheet for student mocks. They were all bright and sparkly and talking about all the wonderful things that they had learned from entering all that data and analysing the ace colour palette. Look, if you think the role you will take is that of a workaholic head of department who will martyr their Sundays on the altar of presenting shiny things at conferences and getting #pedagoo likes on Twitter that’s fine. But remember: you could end up being me. And I would rather shave my eyeballs with a rusty switchblade than fill out a rag sheet. Just something to think about.
That’s probably it for now. If anyone knows of some brilliant practice on this front or really interesting reading please do let me know.