CTU economist shakes the confidence we place in confidence surveys

CTU economist Bill Rosenberg has pitched into the question of ‘business confidence’ and whether it is really falling.

Writing in his 100th CTU Economic Bulletin, he asks whether business confidence surveys are anything more than opinion polls among some chief executives about what they think of the Government of the day or to get their impressions (“which may not be any more accurate than yours or mine”) of where the economy is going?

Rosenberg acknowledges that business decision-makers need confidence when thinking about investing or hiring new staff, to decide if they can take a punt on whether they can increase sales to justify the additional cost.

But in practice confidence cannot be directly observed and survey results are affected by many things that may be irrelevant to those decisions, he says.  Its relevance therefore comes down to whether stronger investment and employment follows higher confidence ratings.

The evidence that it does do this “is at best weak; often it is just not true”.  

He proceeds to insist that assertions of falling business confidence are used by business and its political allies to try to frighten public opinion against current Government policies, but they are not based on rational argument or evidence, nor on what is good for New Zealand as a whole “unless we accept that what is good for business is always good for New Zealand”.

Ah, but isn’t this what you would expect a CTU economist to say?

Probably.  But Rosenberg references Kirk Hope, chief executive of Business New Zealand, whose column in the Dominion Post on 21 June listed half a dozen reasons why “business should [be] feeling confident”.

Fair to say, Hope also listed several Government policies which business doesn’t like, including the end of future oil and gas exploration and a raft of proposed changes to employment legislation, such as pay equity and changes to the Employment Relations Act.  He said these “have the potential” to add cost and reduce international competitiveness.

Rosenberg challenges this.  The changes in oil and gas exploration rules, for example, “were always coming as the world gets serious about climate change”. Anyway, real changes are far off, work is going on to replace those industries, and many in the oil industry internationally could see such changes coming and have been preparing for change by moving into other forms of energy.

But besides opining on the worth of reports that business confidence is falling, Rosenberg helpfully explains the mechanics of the surveys which generate the headlines:

  • Business NZ’s Economic Conditions Index is not found on their public website other than in references in their publications. It is a mixture of economic indicators and confidence surveys, and the weighting between them is unclear.
  • The NZIER’s Quarterly Survey of Business Opinion (QSBO) has been around in various forms since 1961 and its current methodology makes it the most credible. Each quarter it surveys “a panel of around 3,500 chief executives or their nominees in the three main sectors – manufacturing and building, merchants, and services”. It uses a sample of enterprises from the business directory of Statistics New Zealand and the UBD New Zealand Business Directory. It is weighted towards larger firms: it includes all firms with more than 200 employees and excludes all firms with fewer than six employees. But it is not publicly available other than through very brief media releases each quarter.
  • The ANZ Business Outlook survey goes back in various forms to 1983. Its data are publicly available and were supplied as soon as Rosenberg for them. It is not entirely clear how they choose their relatively small sample of around 500, he says, but it appears to be by self-selection (they ask for volunteers to join the survey). ANZ then weight the survey to make it more representative.

The NZIER and ANZ surveys – and most similar surveys internationally – are designed to be quick and simple to be filled in, Rosenberg observes.

Typically the questionnaires comprise just one page and business respondents just tick boxes in answer to various questions as to whether things will improve, stay the same or deteriorate. The number answering “deteriorate” is subtracted from the number answering “improve” and that is divided by the number ticking any of those answers. This ‘net’ score is reported in some surveys as a positive or negative percentage (as ANZ does). In others, 50 is neutral, so above 50 means improvement and below 50 means deterioration.

Questions cover general perceptions and views about the firm that is being surveyed (‘own activity’). For example, the ANZ survey asks in the first place, “With regard to the New Zealand economy, do you believe that General Business Conditions in 12 months time will have” [tick ‘improved’, ‘remain the same’, or ‘deteriorated’].

Later it asks, “With regard to Your Business, how do you expect the following variables to have changed in 12 months”, and the variables include “Real Business Activity (i.e. volume, Not $ revenue)”, “Employment”, and “Investment in buildings, plant, equipment”.

The ‘net’ method ignores the ‘stay the same’ responses. For example if from 100 respondents, 60 tick ‘deteriorate’ and 40 ‘improve’, giving a score of 20 per cent (60 minus 40, divided by 100), the score is the same as 30 ticking ‘deteriorate’ and 10 ‘improve’, with the majority (60) thinking it will stay the same, a very different situation.

There can be considerable variation from period to period within such combinations which will not show up in the reported results.

Rosenberg then checks out international research on business confidence surveys and – at some length – reports what he found. The results are mixed but overall show very weak support for the notion they add anything to official economic information sources.

Given that, they can tell us very little about “confidence”, to the extent it does have an impact on the real world.

In conclusion, Rosenberg says we would be foolish to entirely dismiss the concept of business confidence as having a role in future investment and employment.

However it is clear that many surveys, including the ANZ Business Outlook survey are not reliable measures either of confidence (it is not even clear what is being measured) or of future behaviour. The NZIER Quarterly Survey of Business Opinion, judging from the research, is probably not a great deal better despite its more rigorous design. They are at best a rough guide and should be treated with considerable scepticism and caution. The surveys and the vague concept of business confidence are too often a code for “what business approves of”.

But if policy was made simply on this basis, “nothing would ever change, except to the advantage of business”.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.