While the emerging dispute within the Coalition over manufacturing protectionism might get reported in negative terms, it’s a sign of health that people such as Joe Hockey are prepared to publicly debate whether we should continue to support the automotive industry. It’s a debate Labor should be having as well and hopefully will have when the new funding packages go to cabinet. Labor has long grappled, successfully, with these issues under previous prime ministers. It can do so again.

Bland assertions that “we need to be a country that makes things” or claims that a car industry is the mark of a First World economy shouldn’t be the basis for spending billions of dollars.

One of the problems everyone outside government — and probably many in government — have is the lack of information about the true effectiveness of industry programs. Metrics such as how many jobs are supported across an industry and how much co-investment is generated can be found, but there’s little information that enables a comparison of differing programs on a common basis. This week, Warwick McKibbin and Henry Ergas called for a study of how effective automotive assistance is compared to other forms of assistance. But such comparisons notionally should already be enabled by the Outcome-Output budget framework established by the Howard government and still in use.

The idea is a legal framework for appropriating money (although a rather loose one after the High Court’s Combet v the Commonwealth decision) and a performance evaluation framework — it identifies outcomes designed to be achieved by expenditure, the outputs intended to achieve it, and performance indicators designed to show how effectively it has been achieved after the event.

With such a framework, you theoretically should be able to see how departments and the programs they administer are performing.

In practice, there have been problems. Neither ministers nor bureaucrats like performance indicators that are beyond their control. Just as you should never launch an inquiry when you don’t know the outcome, no one ever likes to commit to performance indicators they don’t know they can meet. Performance indicators thus tend to end up being about processes, not about what impacts expenditure may have had.

So if you go into the Department of Innovation’s budget documents and annual reports, you’ll see lots of “metrics” such as the number of grants made — something more or less in the control of the department — but nothing on what are described in the budget papers as “top level performance indicators” that might give a guide to how effective grants to business are — such as the number of patents or commercialisation agreements or the level of R&D expenditure achieved by business.

In the most recent budget, the department (and this is certainly not a problem confined to Innovation) admitted that its reporting on program effectiveness was a work in progress. “The department is continuing to build capacity to monitor the performance of departmental programs … The top-level key performance indicators for Program 1.2 are supported by metrics which demonstrate how the department may report on performance in future years,” a statement repeated about several programs.

In short, we don’t even know whether automotive assistance is more effective in supporting jobs, R&D investment and innovation than other programs administered by the department. Giving multinationals hundreds of millions of dollars to keep their plants running here could be the most effective possible form of assistance, or we could be doing far more good in other industries with the same or less money — we just don’t know until the Innovation Department and the Finance Department — which oversees the Outcome-output framework — finishes overhauling its performance indicator process.

Of course, given the government has just told departments to find an additional 2.5% in savings, we might be waiting a while. If you’re running a department, which are you more likely to cut or fail to replace — the staff who hand out the grants to industry, or the staff who would monitor and evaluate those grants?

That means the debate over industry assistance will continue to occur in a vacuum.