I came across an excellent paper from Prakash Loungani and Jair Rodriguez of IMF. The paper reviews the forecasting experience of various private forecasters in forecasting recessions.
Only two recessions were predicted a year in advance and one of those predictions came toward the turn of the year, as shown in column 3 of Table 2. Requiring recessions to be predicted a year ahead may seem like an unreasonably high bar to set. Lowering the bar to the start of the year in which the recession occurred does indeed improve the performance somewhat: 8 of the 26 recessions were predicted in February of the year in which they occurred and 16 were predicted by August. But even as the year drew to a close, 6 of 26 recessions remained undetected by forecasters.
Moreover, while forecasters increasingly start to recognize recessions in the year in which they occur, the results in column 4 show that the magnitude of the recession is underpredicted in the vast majority of cases. For instance, even as late as December of the year of the recession, the forecast is more optimistic than the outcome in 15 cases.
The remaining columns of Table 2 show the average forecast error at the different forecast horizons over all 26 episodes and also for the G7 and EM7 countries separately. Note that average forecast errors continue to be quite substantial even for forecasts made fairly late in the year of the recession. For instance, in August the average forecast error is 0.6 percentage points for industrialized countries and 2.4 percentage points for developing countries.
In all forecasting has been as Bernanke in his speech (November 14, 2007) said:
The only economic forecast in which I have complete confidence is that the economy will not evolve along the precise path implied by our projections. Nevertheless, as I have already noted, because policy affects spending and inflation with a lag, Committee members have no choice other than to make medium-term forecasts–provisional and subject to uncertainty though they may be.
Now, what were the reasons for the misses?
The tendency not to revise forecasts promptly, while particularly costly in a recession, is a feature of forecasts in all years. Our inspection and analysis of forecasts for the 14 countries in non-recession years shows that forecasts are revised in a very smooth fashion: the forecast revisions—the changes in the forecast between successive horizons—are smooth.
Meaning people keep sitting and don’t revise forecasts. In a shorter article Loungani explains:
Efficient forecasts, Nordhaus showed, “appear jagged because they incorporate all news quickly. Inefficient forecasts appear smoother … for they let the news seep in slowly.”
This tendency to smooth forecasts excessively is particularly costly at present, when some economies around the globe appear poised at turning points and could experience outright recessions or marked slowdowns. Making predictions at turning points requires unusual alertness on the part of forecasters to incoming economic information and a willingness to raise alarms about possible recessions, even at the risk that some of these calls will turn out to be wrong. However, the evidence shows that forecasters are unwilling or unable to signal that the economy is heading for a recession until one is absolutely imminent; and even then they initially underestimate the extent of the decline.
The authors also show people are slow to respond to not just national news but also foreign news and as a result we have the decoupling theory going for a lot longer than the reality.
The questions authors don’t answer is why do forecasters not revise their forecasts often? I mean news is all over the place these days and still teh revisions do not happen. Why? Let me throw two answers to it – Both from field of behavioral economics.
First reason is overconfidence , which I have explained here. The main tenet is economists suffer from overconfidence bias and think their outlook is supreme. The nature of the job is also such that unless they display confidence no one will hear them.
Second, which is more relevant to the above experiment is that people ignore negative news a lot longer than positive news. This is called extreme aversion. This bias leads people to hold loosing stocks and sell winners (expecting them to rebound) and in this case hold on to the negative news a lot longer than is needed.
Third, people suffer from anchoring bias and rely much more on the past trend than new development. their Infact, I have shown earlier that forecasters in India do not revise their forecasts even amidst high growth and as a result their forecasts were much lower than actual. It is only when the high figure becomes quite apparent do they revise their forecasts upwards.
The paper raises concerns but I don’t see this behavior correcting. It is human nature and will take a lot of time to change. I have made a case earlier as well that in order to improve forecasting we need some accountability. The economists/analysts need to also project how likely is it that this forecast doesn’t happen. This will also help people make their own estimates and not rely blindly on the experts. Second, we need to keep official scores of either same economist or similar such predictions.
Anyways, a fantastic paper and tells you much more about forecasts going wrong in a simple manner. It is unlike those papers where only lesson you get is the model was wrong.
But still I don’t know why did the authors give behavioral economics a miss. I mean they could have just thrown these ideas for future researchers. I think one of the main reasons is most economists still don’ give importance to behavioral economics. It is high time this attitude is changed.