Pearls and Irritations

A recent submission to a Senate inquiry by the Centre for Public Integrity claimed that in the decade to 2023 companies making political donations “were 2.49 times more likely to win procurement contracts than non-donors”, and that the value of contracts won by donor companies was on average 4.4 times the value of contracts won by non-donors. Centre Director Geoff Watson SC told The Guardian that “the people who are making these donations are not fools and are doing it for a reason. The data demonstrates that the donations achieve their purpose because they’re receiving more government work. It’s as simple as that.” Spoiler alert: it’s not.

When I read the Senate submission from the Centre for Public Integrity, two issues stood out for me: how did the Centre get hold of the data to support the conclusion that donors are 2.49 times more likely to win contracts? and how does the Centre imagine donor status practically affects tender outcomes?

The data issue

The submission stated that the Centre’s research drew on donations data from the Australian Electoral Commission and contract data from Austender. Together this data may support conclusions about the number of contracts awarded to donor companies as compared with non-donors. But it cannot support conclusions about the relative likelihood of winning contracts, because it does not contain data on unsuccessful attempts to win contracts.

To draw a sporting analogy, the South Melbourne/Sydney Swans has won five VFL/AFL premierships, while Port Adelaide has won one. But this doesn’t mean the Swans are five times more likely to win, because it ignores the fact that the Swans have competed for 127 seasons, and had 127 opportunities to win, while Port have only competed for 27 seasons. Adjusting for opportunities to win, the success rate for both teams is a little under four per cent.

A conclusion that donor companies are more likely to win contracts implies that their success rate is better than non-donor companies. Measuring success rates requires knowledge of how many tenders each class of company submitted, as well as how many contracts they won.

However, this data is not publicly available, and the Centre has confirmed with me that it did not have access to it.

When I raised the difficulty of drawing inferences about the impact of donor status on contract success in the absence of this data, the Centre agreed to review its findings. It concluded that it could not state that donor companies were “more likely” to be successful, but that it was valid to state that “contracts were disproportionately awarded to donors in the year they donated by 2.5 times of what would be expected”.

The Centre explained to me that this figure was derived from a database of all political donations and associated contracts over the period 2013-2023 by:

1) Calculating the number of parties who received a procurement in the year they donated, divided by the number of entries in the dataset in which somebody did donate, which results in a 5.96% probability of a donor winning a procurement contract in the year they donated.
2) Calculating the number of parties that received a procurement given that they didn’t donate in a given year, divided by the number of people that didn’t donate in a given year. This results in a 2.40% probability of someone in the dataset receiving a procurement given they didn’t donate.
3) Dividing the results from step 1 by the results from step 2.

This analysis still ignores the impact on success of the number of tenders submitted.

The tender process
The Centre’s submission to the Senate observes that “there is no causative relationship able to be established” between donor status and tender success. That is because there isn’t one.

The tender evaluation process focusses on getting value for money for the Commonwealth. Tender evaluation committees assess proposals against an assessment plan. For consultancy services the plan will set out criteria such as understanding of the requirement, methodology to deliver the requirement, demonstrated ability to carry out the methodology, and project cost. The criteria do not include political donor status.

Over the years I took part in a few hundred procurement processes as a member or chair of a tender evaluation committee, and signed off on the results of a few hundred more, entering into contracts with the successful tenderer.

In virtually every case I was unaware of whether or not a tenderer had made a political donation. Donation awareness would presumably be a lot higher among Ministers and their office staff. Yet even though over the years I dealt with a few extremely pushy Ministerial advisers (who would try to tell me what my advice to the Minister should be), none of them ever attempted to interfere in a tender evaluation process.

Based on my experience, I am sure that the share of contracts going to donor companies reflects their size, capacity and business practices, not their donor status.

In the health economics, evaluation and data analysis area the “big four” consultancy firms would respond to virtually every Request for Tender. They could do so because they had the staff resources available to deploy to preparing proposals, even if they weren’t confident of success. They also knew that if they did win the tender, they could muster the staff resources available to do the work – even if they had to draw on overseas branches for some assistance.

The big four strategy was to kiss lots of frogs, knowing that some of them would turn out to be princes.

The specialist (far smaller) health consultancy firms were far more discriminating. They would only submit proposals for tenders where they thought they had a good chance of success, and if there was space in their forward work program to conduct the project. They would kiss a frog only if they could see the outline of a diadem under the skin over the ears.

I would not be surprised if the success rate of the big four donor firms in health tenders was actually lower than the success rate for smaller specialist firms.

The higher value of contracts awarded to donor firms reflects the unrelated facts that: (a) large firms are more likely to be donors than smaller ones; and (b) as large firms they are more likely to have the capacity to undertake larger, more complex and more expensive tasks.

The Centre for Public Integrity is making an important contribution to the public debate on improved transparency and accountability and reducing the impact of money on politics. It shouldn’t rely on poor analysis to support its arguments.

Editors’ note:

The Centre for Public Integrity was contacted for comment prior to publication.  CPI’s statement is included below:

“As an organisation committed to ensuring the rigour of our research, the Centre is always happy to engage with queries (resources permitting – we’re a 2 FTE org). When the author approached us with his concerns we listened and, as a consequence, modified the expression of our conclusion. We also had an economist verify that the analysis and modified conclusion are sound, as we indicated to the author. We provided the author with a detailed justification of our result and invited him to submit it to the scrutiny of a qualified quants expert and come back to us with any problems (we do not know whether the author took up this invitation; certainly, no problems were subsequently raised with us).

When this article was submitted to P&I, we explained that it had been independently verified and requested that P&I’s editors have an appropriately-qualified expert examine our justification before a decision was made in respect of publication. It is regrettable that they did not agree such a precaution would be prudent.

We have great regard for the work of P&I, but in these particular circumstances it’s unclear how the article enhances the quality of the discourse on such important issues for our democracy, and we’re disappointed by the decision to publish it in the absence of the concerns raised in it being appropriately verified.” (December 11, 2023).

For more on this topic, P&I recommends:

Share and Enjoy !

SHARE TWEET SHARE COPY EMAIL PRINT