The release of the Australia Council’s latest research report, More than bums on seats, has provided a bonanza of positive publicity for the agency.

“Aussies love arts as well as sports” read one headline, and many articles picked up on the headline figure that 93% of those surveyed had participated in the arts in some way — equating to some 16 million of us.

It has also highlighted the dire state of arts journalism in this country. Of the journalists who covered the report, only The Australian‘s Michaela Boland picked up (almost in passing) the crucial weakness of the report, which is the way it defines “the arts”.

There’s no doubt that the report is a major piece of new research: the last such exercise was the Saatchi report from 1999, so the new data is welcome. The project conducted 15 focus groups and polled 3500 people, making its findings statistically significant to a fairly low margin of error.

The Australia Council’s agenda is stated clearly enough on its website: “We trust the research will stimulate debate within the community and the arts sector about new directions, new investments and the shape of future cultural policy.”

The survey comes at a useful time to influence Peter Garrett’s ongoing review of Australian cultural policy, and you can be sure that OzCo would like “new investments”. But despite the Australia Council trumpeting the importance of the report, there are major methodological flaws.

The most glaring problem is the way the research defines “the arts”. Instead of asking people what they thought  “the arts” were — which in the Saatchi report led to  some uncomfortable findings about what people think about their elite and snobby nature — this report defines “the arts” in advance, telling those surveyed that “the arts” are music, writing and the visual and performing arts.

This is all well and good, until you realise what that definition leaves out. There’s no cinema, no television, no radio and no games. What gives?

Leaving out screens, radios and games is no minor oversight. According to the Australian Bureau of Statistics’ survey of cultural attendance, going to the movies is far and away the most popular category of cultural attendance, while the ABS’ 2006 Time Use Study found that “the most popular recreation and leisure activity was use of audio/visual media, which includes watching television and DVDs, listening to the radio and CDs.”

In other words, the Australia Council’s survey of “the arts” leaves out the most popular artforms. It’s almost as though OzCo set out to define the arts  as “what the Australia Council funds”.

As the report itself states on page 13:

“The focus of the study is upon the art forms that are supported by the Australia Council (visual arts and crafts, music, dance, theatre, literature). The definitions were agreed after a thorough consultation within the Australia Council and with key stakeholders.”

Not only does the report therefore feature highly skewed statistics, it also ignores some of the Australia Council’s recent efforts to address digital culture, like it’s recent pamphlet about “Arts Content for the Digital Era“. Why have a digital arts strategy, if you’re not going to bother to measure the most important forms of digital culture?

Another interesting outcome of the survey is the importance of reading for Australians. The survey confirms previous data from the Books Alive project that Australians are avid recreational readers: 84% of those surveyed read for pleasure, with a surprising one in five loving poetry. This poses uncomfortable policy implications for the Australia Council: if writing is the most popular part of OzCo’s portfolio, why does OzCo still devote most of its funding to the less popular performing arts? The Literature Board gets just $8 million in a total budget of $175 million.

It’s another piece of evidence of the ostrich-headed policy direction of the Australia Council under Kathy Keele. After abolishing its New Media Arts Board in 2004, the Australia Council continues to turn its back on popular culture and the art forms of the 21st century. It might be More than bums on seats, but its a lot less than the full picture.