Archive for June 30th, 2008

Overconfidence should be a part of every forecast/prescription

June 30, 2008

I had earlier pointed overconfidence seems to be in our genes. I had also referred to a paper by Erik Angner in the same post, which said economists are overconfident. On reading the paper, I found it so true. Though, it is in our genes, it is nice to see papers to understand how overconfidence affects each specie-type and this one looks at economists. (Infact, the paper also shows overconfidence being common in doctors, lawyers, physicists etc)

The paper actually focuses on a Swede Economist – Anders Aslundwho was the main advisor to Russian Government in 1991 which had embarked on a path to reforms. Aslund’s suggestions were:

Aslund was one of the strongest proponents of shock therapy, which he called ‘the only cure’ (1992a). He predicted that the Russian Gross Domestic Product (GDP) might drop by ‘at least 20 percent’ as a result of the therapy before taking off again (1992b),

 

but he promised that the therapy would work fast (1993a), and have positive social consequences

However, reality was different:

According to official World Bank statistics, Russia’s GDP in constant 1995 US dollars dropped by almost 40 percent between 1991 and 1998  This is twice Aslund’s estimate. Though GDP increased between 1998 and 2003, the last date of available World Bank statistics at the time of writing, GDP remains more than 16% below the 1991 level.

Similarly, Aslund’s predictions about the positive effects of shock therapy do not appear borne out by subsequent events. For example, between 1991 and 1994 life expectancy in Russia dropped by as much as 4.7 years (6.2 years for men). Though the numbers have since picked up, the last available figure is still a full three years below that of 1991.

In sum, the confidence Aslund expressed in the truth of his judgments may well have exceeded their accuracy.

Moreover, Aslund was not happy with the reality so he devised his own measure of GDP and criticised the policymakers.

The author takes caution and says Aslund’s case is not a one-off case and points to statments by Jagdish Bhagwati, Stiglitz who criticised the development experts on similar grounds. Stiglitz said:

IMF experts believe they are brighter, more educated, and less politically motivated than the economists in the countries they visit. In fact, the economic leaders from those countries are pretty good – in many cases brighter or better-educated than the IMF staff, which frequently consists of third-rank students from first-rate universities.

He says economists are overconfident on such policy matters as these tasks like transition etc are in highly uncertain times and economists have to be confident of their advice.  And why don’t they correct their prognosis with time?

  • First, they don’t receive adequate feedback as there is hardly a proper statistics system in these countries which says what is working .
  • ]Second, we exaggerate the predictability of past events.
  • There are no noticeable penalties on economists for expressing overconfidence. So, it continues.
  • Selection bias- as economists are over-represented in a committee , the overall prescription is on similar lines. Moreover, confidence is mistaken for competence. so we usually have overconfidence all over the place

Consequences of overconfident policies are well known- misguided policies. Further, as forecasts and prescriptions fail economists advice is discounted or ignored. This gives the entire economics field a bad name.

He also says, this does not imply we don’t need economists (thank god) but we have to devise ways to limit overconfidence.  First, economists need to be made more accountable and should also predict how likely is it that this forecast doesn’t happen. Specific questions need to be asked on what is it that their performance has to be assessed (for e.g which measure of GDP, inflation??). Second, we need to keep official scores of either same economist or similar such predictions. I liked this line:

If you are a baseball player, you have to live with your batting average until the day that you die. If you are an economist, you can advice the government of Russia for years without being forced to critically examine your hit rate. In spite of the many practical problems associated with the proposal, there is no in principle reason why economics should be different from baseball in this regard.

 

:-)

This is an excellent paper and an answer to many a problems in economics. I had pointed earlier that Indian economy trackers are flipping  forecasts by every second and there is no accountability. My suggestions were also similar- track their score and if they have deviated from previous forecast ask for reasons.

I have analysed forecast experience in India, covered Fed experiences, Riksbank experience and see a lot of deviations from the actual. The best in the business find it difficult but still, the forecasting business continues with aplomb without any fear (of poor performance). The people simply move from one forecast to another, one prescription to another. The forecasts should include the above suggestions to  help people understand it is uncertain. Another area where you see overconfidence is the various 10 steps to boom reports.

I think all such reports/prescriptions should now have a measure of over-confidence. High-time the field becomes more accoutable and humble.

 

 

 

Assorted Links

June 30, 2008

1. WSJ Blog has a profile of the new Fed board nominee- Liz Duke

2. WSJ Blog points consumer sentiment is a helpful predictor

3. MR on what else- speculation

4. TTR pointsto governance issue at Ranbaxy

5. Mankiw points to Bob Shiller article who says recent fiscal stimulus is not enough.

6. I discovered a blog from Annamaria Lusardi, a professor at Dartmouth who has done some superb work on fin literacy.


Follow

Get every new post delivered to your Inbox.

Join 1,241 other followers