Entrenching inequity in Gillard’s education devolution
|
Take a good dose of robust ideology — a laudable conviction that every child should benefit equally from education. Add some imported rhetoric reflecting notoriously failed school accountability reforms from the UK and US, and a dash of imported expertise (New York’s City schools chancellor Joel Klein). Combine this with a good dollop of political nous and the usual paucity in relevant background and outcomes research for the education ministry portfolio. Then beat it all up within a large budget, spicing with a generous quantity of obtuse statistics. Half bake the final product and glaze it with a colourful presentation — and you have the recipe for Julia Gillard’s MySchool website. In the ongoing furore from many education professionals over Gillard’s responsibility for the publication of NAPLAN test results on the MySchool website, six months on, the basic argument seems to have long been lost in translation. The website displays results of the National Assessment Program in Literacy and Numeracy averaged by school, and was launched on January 28 with great fanfare about transparency in school performance indicators. An information filter of media confusion and disinterest in the rationale behind the publishing of these results has helped ensure that MySchool may now appear a resounding success. Recently, software alterations made it a little less easy for further simplistic school league tables to be generated directly from the website for tabloid publication — a concession that perhaps, at last, showed some acknowledgement from the federal education ministry that there could be a problem. The problem, however, is that MySchool, now “tweaked” and promoted as supposedly comparing apples with apples and oranges with oranges, remains a lemon. The front page of the SMH July 3 Julia Gillard appeals to voters “Judge me on how I do the job” and lists the MySchool website among her achievements when minister. This article has been placed alongside a report highlighting the fact that students from Asian backgrounds are dramatically outperforming students from English-only speaking households in selective high school entry tests, a fact that would be no surprise to most teachers. Although the irony may be easily missed, the assumptions underlying MySchool comparisons stand juxtaposed to many accumulated years of similar evidence that socio/cultural factors account for far more of the variance in averaged student test results than the quality of teaching purportedly highlighted through Gillard’s MySchool website. A hallmark of federal Labour’s performance has been an unprecedented attempt to provide comparative measurements of previously unquantified information. In deference to our public right to informed choice, we were to be provided with comparative grocery prices and comparative petrol prices. It might then have seemed a small leap to provide parents with comparative information on “school performance”, presumably then enabling them to make an informed choice about their children’s schools. Undoubtedly, many parents would have welcomed the display of statistics that superficially may seem to mean something valid and informative. To facilitate lay interpretation of NAPLAN results, all schools supposedly performing above average are distinguished by green highlighted NAPLAN results (green for “GO”?). An Index of Community Socio-Educational Advantage (ICSEA) is claimed to identify “statistically similar schools” using data from the 2006 census, but can at best use only a small sample of potentially relevant variables (such as proportion of indigenous students but not proportion of Asian students). Moreover, the invalid assumptions underlying the ICSEA ranking severely skews NAPLAN comparisons in favour of already advantaged schools. For example, during the Howard years, higher-performing students from higher-income families increasingly “followed the money” in the accelerating drift to private schools, so that now higher-income families choose private schools at double the rate of lower-income families. However, ICSEA ratings assume homogeneity of socio-economic status for all students within each (2006) census area and therefore assume a statistically “random” choice of school for any particular family. By definition, only half the schools on MySchool can achieve above the average; and (surprise, surprise) overwhelmingly these are the socio-economically advantaged schools and those schools better able to select their student intake (such as “coached” and high achieving Asian students). These advantages would include many diverse variables such as maternal nutrition, the access to quality pre-school education and early supervised internet access. Six months on, as most educational researchers predicted, given the contextual vacuum and lack of statistical transparency attendant with their publication, MySchool NAPLAN results have become a catalyst for misinformation and prejudice against those 50% of students from schools with largely red highlighted results. On ABC’s Q&A on August 6 last year, Julia Gillard had stated “let’s just get each school’s results out there for all to see, so that we can see which schools might need more help”. So far we have not seen more help. What has been evidenced is some media beat-up about a few teachers supposedly tweaking class results, school principals reportedly fearing being sacked, tests being opened early and answers supplied, and many children with learning difficulties not sitting the tests. NAPLAN test administration directives are to be tightened, which will do nothing to ease any growing paranoia among those teachers and principals whose students perform, for whatever reasons, below the Australian school average. Moreover, while Gillard’s ideology may hold merit, the assumptions reflected by her statement beg more than a little critical thinking. The assumptions seem to be that a school’s overall performance can be measured by averaged NAPLAN student test results for that school, and under-performing schools can therefore be identified through their lower than average NAPLAN test results. The problem here is so obvious that it seems often overlooked. Gillard understands that it is not whole schools but individual students who “perform” academically in literacy and numeracy tests and that student results reflect a host of complex variables, only one of which is teacher performance. However, apparently it is less obvious that the above assumptions are a quantum leap into a quagmire of potential confusion and misinformation for parents. Essentially meaningless “damn lies and statistics” commonly influence public perception and decision-making. This happens because, in a strange quirk of our collective Western psyche, we pervasively tend to ascribe intrinsic validity to comparative information when mathematically expressed. The trouble with MySchool is that the more meaningful the construct of “school performance”, the more complex and multidimensional, and therefore simplistically unquantifiable. The comparative “performance” of public hospitals could hardly be validly measured through league tables of births and deaths per population demographic, not even for “statistically similar” hospitals. However, given some reductionist terminology, the definition of “school performance” has lost complexity and therefore become statistically measurable. Before the launch of MySchool, Julia Gillard was studiedly careful in attempting to reassure all stake holders that the federal government’s publishing of NAPLAN test results were not to be used by media outlets to feed prejudicially presented simplistic comparisons between schools (aka league tables, which, of course, then ensued). So what, then, are the results to be used for? Considerable paranoia among teachers in disadvantaged schools has been understandable, when nothing has been made clear about just what school averaged test results should be interpreted to mean, what these can be used for, and the assumptions behind their measurement. A problem with simplistic school performance comparisons is the virtual impossibility of truly comparing like with like. Contrary to claims on the MySchool website, it is virtually impossible to include enough relevant variables to ascertain which school groupings are truly “statistically similar”. In fact, even regarding something as ostensibly measurable as comparative prices for groceries, to actually compare “apples with apples” and “oranges with oranges”, far too many variables had to be controlled for to allow the Rudd Government’s grocery choice website to ever have got off the ground. Were we to simply average the price of all individual grocery items in an Aldi store, then compare this average with that similarly obtained from a Woolworths store in an adjacent suburb, the public would hardly have been duped. Variables omitted obviously include the quality and quantity of each type of grocery item. Such a comparison then, would clearly be ludicrously meaningless were it to be portrayed as any sort of measurement of the relative “performance efficiency” of these two supermarkets, and would tell the consumer absolutely nothing about which supermarket might be selling the best priced Granny Smiths. Yet something rather alarmingly akin to this supermarket comparison is exactly what Gillard’s department facilitated for comparing school performance, because the number of variables involved in even one individual student’s test results is far greater than the overlooked variables in this parody of a simplistic supermarket comparison. MySchool, in providing chalk and cheese comparisons, has particularly highlighted socio-economic disparities between Australian schools. So what might provide a valid assessment of “school performance”? The Rudd government’s response to this question betrayed some ideological biases and restricted assumptions about the nature and purpose of education itself. An unfortunate default position, in which schools are perceived as “fact-learning factories” has had a long and unfruitful history. The pendulum of educational ideology has regularly swung back towards the extremes of rational economics and education devolution. It seems that the further the perspective has moved from classroom realities, the more the political lens reflects a myopically constrained view of the education of children as being something adequately expressed through their literacy and numeracy test results. It could instead have been assumed that “school performance” might be better measured by weighted multidimensional factors reflecting the health of a school’s organisational climate, such as older students’ ratings of teachers’ merits, teacher job satisfaction, and truly representative parent participation. Academic performance could be better estimated through assessing “school value added” performance, such as through measuring the degree of improvement in individual student’s results over time. A vast array of other factors might also be measured to provide a better understanding of “school performance”. These might include, for instance the percentage of students learning to play a musical instrument from scratch, or continuing on to a trade or further study, or not attempting suicide before they graduate. Within such a rich conglomeration of achievements, statistics such as averaged individual student improvement over time in literacy and numeracy test results could be given meaningful context. Given enough resources, time and expertise, we can validly measure almost anything, provided it is a clearly defined and quantifiable entity, irrespective of whether or not it is actually worth measuring. However, schools are neither factories nor supermarkets, and little Johnnie brings a vast array of complex variables to his test-taking performance on any particular day. Only one variable of the many is how well his successive (i.e. past and present) teachers have taught him literacy and numeracy skills. Other variables include familial, cultural, societal, psychological, physical, nutritional, sensory and emotional variables impacting upon his academic and test-taking ability, but that prove much more difficult to measure, and are constantly changing. The best-performing schools therefore create and maintain an organisational climate in which teachers consistently work towards the achievement of each individual child’s highest potentials, with awareness of all these above factors informing every aspect of their pedagogy. “School performance” thus defined would require far more complex assessment. Moreover, the best performing schools would inevitably include some that would fall well below the average on the NAPLAN tests. Conversely, by this definition, some of the worst performing schools, where teachers are “coasting” on the school’s reputation or those schools where competitiveness has prompted teachers to start “teaching to the tests”, would likely fall within the top MySchool rankings. Perhaps the Rudd government should have gone to the top of the class for the sheer quantity of measurements pursued in the name of accountability and the rights of consumers to make informed choices. Unfortunately, the failure to listen to and consult with informed educators, combined with the failure of remedial class action by the teaching profession to be heard and understood by most parents, tabloid journalists and voters, has left a loathsome legacy from Gillard’s stint as federal education minister for all those concerned for “our education future”. I rest my case — MySchool remains a lemon. Elizabeth Lyons is a psychologist who works with children, teenagers, teachers and parents in the NSW public school system. |
|
|
|
19 Comments
What a load of pompous crap. While I’m reading this rot I keep hearing a voice that sounds permanently pinch-bummed. “School performance” thus defined…..I mean really.
I had to read at least 5 paragraphs of this swill before I could find out what your problem was and even that was so convoluted that I just decided that your argument was rot and shouldn’t be taken seriously.
How about people were bit4ching about the quality of education, the state governments didn’t know what to do about. The Federal government had the resources and capability of gathering the data and giving it to the public to analyse as they wish, which is what they did. I get to recognise my local public school is better, and meets my families scholastic requirements, then my local catholic school. People are given the numbers so they can work it out for themselves without government, educational or media bodies skewing it to project their own biases. Also I think people would start asking questions if the schools average is so high and their attending child can’t read.
Incomprehensible. I hope this is not representative of school psychologists’ communication skills. No wonder we need NAPLAN and MySchool.
The high profile political football that is Indigenous affairs, that gave us the pre-election “Intervention” and post-Election “Apology” at the last Federal election seems to have faded into oblivion. The dog whistle however is back with a vengance with Tony and Julia blowing to their hearts content. Those pesky boat people and those despicable people smugglers are the source of all evil in this, the land of the fair go.
“NAPLAN test results ….been lost in translation” Indeed lost in translation. Here in remote Aboriginal Australia our third graders are not only tested in a language they are just starting to come to terms with (as a second language), but the tests are contextually and culture completely alien. The “results” are abominable, yet these are bright little smart kids that have no problem in pointing in which direction Alice Springs is, and have lively intricate conversations in Warlpiri. You test grade three children in Box Hill, posing the questions in Warlpiri, and see what their NAPLAN results are. Also ask them where to find honey-ants (yurampi).
No, I agree with Elizabeth Lyons. NAPLAN tests and the Myschool website are a minefield for turning information into misinformation
Delerious when you use phrases such as ‘gathering the data and giving it to the people’ or ‘People are given the numbers so they can work it out for themselves without government’ it seems that you just don’t get it. ‘Without Government’…who put the myschool site up? Your assumption that ‘data’ and ‘numbers’ are neutral facts that ‘speak for themselves’ is naive and ignores how power operates. The site is highly political and designed to mislead. The accuracy of the NAPLAN tests on which myschool is based is now contested (Margaret Wu from Melbourne University). The site unfairly sorts and ranks students and schools and perpetuates the existing inequities.
Brilliant article.
The four big assumptions underlying the MySchool idea are:
1. That NAPLAN results give an accurate indication of children’s ability to read and write.
2. That comparing average NAPLAN results to ABS data about the socioeconomic status of a school’s postcode, you can derive a number that tells you how good that school is at teaching children to read and write.
3. That knowing how good a school is at teaching children to read and write tells you everything you need to know about how good the school is.
4. That, if you had a number that told you how good a school was, it would be a good idea to publish that information.
There’s been plenty of debate about 4, but not enough debate about 1, 2, and 3. Thank you for speaking some sense.
Salamander and Delerious [sic]: The author is not always to blame when you find something difficult to comprehend.
The language of this article is a little unfortunate. Its core message, however, is quite valid, however much you rant and rave. People cannot take the figures and ‘make up their own minds’, because the site is not designed in that way. Have a look at a page of results for a school. Red bad, green good.
My School is misleading because it gives you a number and expects you to believe that it is an adequate measure of a school’s worth. The numbers are the result of one test, and exclude a whole bunch of information about a school. Even the ICSEA scores are misleading,a s they take information about the Collection District of each student instead of their own family’s socio-economic situation. Because advantage and disadvantage are not necessarily concentrated, but wealthier families ‘self-select’ to private schools, the effect is that poor schools look richer than they are and vice-versa.
Basically what My School does, to simplify the author’s hospital analogy, is to judge a dentist by the number of cavities their patients have. Just as that would be ridiculous because different dentists treat different patients, My School is ridiculous because it assumes that all test scores come down to schooling - ignoring that ‘bad’ schools may in fact be doing a fantastic job with a disadvantaged cohort.
The author made all of these points, they just write too ‘academically’.
Prefer Jamie Nutall’s summary to wading through the content. My god. It even reminds me of reading psychology research papers at uni.
@Mark Egan
“Your assumption that ‘data’ and ‘numbers’ are neutral facts that ‘speak for themselves’ is naive and ignores how power operates.”
I read somewhere that Stalin sent all Russian statisticians to Siberia.
I agree with Delerious on the quality of the article. Turgid to read and the points made are all but lost. There is no ‘academic’ or ‘non-academic’ writing, only good and bad. In good writing, the point being made is clear and clearly supported.
I think there is valuable insight to be gained from tracking cohorts of individuals through a number of years of these tests - for example, comparing the improvement/decline of a cohort of individuals in a school over a series of these tests. This would normalise for many of the hidden factors such as socio-economic factors and the exclusion of specific members of the class from being tested. I have heard nothing about this type of useful analysis being undertaken.
The way the data is used currently, I think it is next to useless.
This is an astonishingly long article saying very little.
You note Julia Gillard as saying “let’s just get each school’s results out there for all to see, so that we can see which schools might need more help”, to which you follow up with the throwaway comment “So far we have not seen more help”.
The remainder of the article proceeds to ignore this basic point and pretend that the point of myschool must be interpreted by its value without reference to anything else.
IF there is actually a system such that ‘underperforming’ schools are provided with significant additional resources, surely this makes your whole article redundant. Myschool has only just been set up - provided there is the intention of using it to direct resourcing, which I think validly remains to be seen, it can be a force for good in the education sector.
Note the irony here is that if/when this targetted resourcing occurs, any school trying to ‘game’ the system to make themselves look better are actually causing their school to receive less resources.
Further, parents who send their students to the schools with lots of green, potentially depopulating lower performing schools, will actually be balancing the playing field - provided that these schools receive those additional resources they should be able to do stuff like decrease student-to-teacher ratios, increase the quality and quantity of classroom equipment per student, etc.
The political point of special note is the bugbear that is private school funding, and I think it is forgotten that this is one of the underlying points about setting aside a large chunk of funding to be allocated based on ‘need’ - if there’s an objective (or at least mostly objective) measure of need based on measuring actual student performance, all of those nasty formulas the Howard government came up with to favour private school funding can be thrown out the window.
And yes, this article used about 10 times the number of words required. You get a ‘D’ for quality of writing.
The MySchool data are clearly suited to longitudinal research. This year’s MySchool gives a baseline, enabling tracking of cohorts over time. This should enable comparison of the effects different schools are having on performance independently of social confounders, as each pupil/cohort functions as their own control. Anyone working in the educational research field would be onto it.
I left school when I was 14 years old, needless to say I’m not the best at spelling and didn’t do a lot of reading but I’m streak’s ahead of my Grandson who is in first year at high school.When my daughter went to see the teacher her answer was ” he is disruptive in class” yes he was but the reason being, he couldn’t do the work and no-one would help him. He used to count with the help from his finger’s, disgraceful. He now goes to a different school and is doing great, a teacher took the time to actually teach him
I also found this article somewhat rambling. Following a student’s progress year by year seems a reasonable idea so I don’t see why it’s unreasonable to follow a school’s progress. I do accept the points made byJungarray and others but feel the system in its’ infancy asyet. Give it a go before condemning it.
Fantastic article. I think it’s hilarious that the MySchool website is portrayed by the government as a huge achievement, and in political terms is a positive for Gillard as she “stood up to the unions”. What we have ended up with is a very public website that attempts to rank school quality based on a figure that means absolutely nothing. They may as well have been randomly generated. Perhaps in this case the unions (and non-union teachers, principals, academics, parents) should have been consulted a little more…
‘Welcome’ says the website and offers: ‘Search by school name’ (skipped that bit as I don’t know any, just wanted a look) or ‘Search by postcode, suburb or town’. Put in my postcode and got the red letter ‘you’ve done it wrong’ message ‘Please enter a suburb/ town’. Oh so postcode isn’t good enough though you asked for one. Thanks for that. How user friendly.
Turns out you have to enter just few enough letters for their search engine to give you a menu list (not ‘postcode, suburb, town’, but actually ‘suburb, State/Territory, postcode), but if you go on typing the menu disappears again and you can’t find nothin’.
Great design. Perhaps the site is designed for use by people who can only type v-e-r-y s-l-o-w-l-y, eg parents [joke]. But I’ve been an internet user and webmaster since before most people had mobile phones and I’ve never seen this type of clunky interface before now.
And the result … the school at the end of my street (where I vote at every election) ‘provides high quality learning experiences for all students in a caring school environment emphasising mutual respect and co-operation’ yet scores average or below average on everything to do with numeracy and literacy. I think it would be hard for me, were I a parent, to make a decision based on this information. How important for me would it be to have a school to which my kid/s could walk in three minutes each way? Meet and make friends with kids who live nearby? I am sure there are all sorts of variables, and literacy and numeracy scores are surely the product of multiple influences ranging from child’s innate ability to home environment, etc - not just teacher performance.
And then, there are other facilities and experiences offered - sports, music etc which may be just as important or more important, for some kids who are not naturally gifted in 3Rs. T hey don’t seem to get scored. Nor the general culture of the school which surely affects a child’s well-beings and sense of belonging.
Just my 2 cents, I have no kids, let alone at school.
In my youth, I attended a school which meant two buses each way and arriving home cold and tired, I was not happy there and in the end my parents moved me to a less prestigious establishment within walking distance. I have no idea whether this affected (plus or minus) my ultimately rather moderate A-level results.
These are far from easy decisions. The quality, personality and attitude of the (rather few) teachers that a senior school student has (or had in those days) probably counted for a lot more in relation to an individual student, than overall scores averaged across the whole school.
Yes I am sure it is good to know as much as possible about your kid’s schoo,l and I know from comments of my parent friends that they are often intimidated about approaching teachers, and may be too busy with the hurly burly of combining family and work to do the research. But is this website the answer? or only a step in the right direction?
It seems to me that most of the arguments against the MySchool site stem from some sort of outrage - “how we dare we be measured against a set of irrelevant standards, etc.” Isn’t this ironic when students are forced through the same process?
We expect schools to provide report cards on students’ performance - why shouldn’t schools be similarly measured?
Obviously measurement systems are limited and the NAPLAN is not perfect. But it may give us a rough idea of where schools are at. It at least forms the basis for a system that may be improved to include, in the author’s words, “weighted multidimensional factors reflecting the health of a school’s organisational climate.” (Maybe that’s how K Rudd would describe it.)
There are many factors that go into selecting a school for your kids. Location, cost, diversity, sports program, music program, teacher engagement and yes, test scores.
The website provides more information to help make the choice (which is why it is well received by the punters), but do you honestly think that parents select their schools based solely on the website? I think we are a bit smarter than that.
I don’t know about anybody else, but I just love it when Julia Gillard tells us that the creation of MySchool represents brave and visionary action on her part. Apparently, the brave part of it is her “staring down” the teachers’ unions.
I read the article, waited a week and reread the article. I’m still not entirely sure what it’s about.
Yes, the socioeconomic background of the schools is the primary determining factor in it’s quality. This has been widely known for a long time but it completely misses the point. A parent looking at the website to choose a school isn’t choosing between a poor rural town and north shore Sydney, they are choosing between two schools in north shore Sydney. So the socioeconomics are fairly consistent. Even if there is a slight variation they are going to want to choose the better school with students from the better background.
Yes, the Myschools data only covers one aspect of the school. In theory the description of the school should cover more but they tend to be fairly bland. Again though it misses the point. Parents aren’t going to use the website as the sole determining factor, they are going to look at the school, talk to the staff, talk to friends etc.
No, they haven’t significantly changed the Myschools website to make it harder to pull all the data and create league tables. I know, I supplied the data to several major newspapers when it was released (and confess some bias in these comments).