Forecast bias of government agencies – CBO and OMB (along with bluechip)

Robert Krol of California State University in the recent Cato Journal has this interesting paper.

There is always criticism over GDP forecasts by govt agencies. They are usually seen as upward biased in terms of growth and downward biased for inflation.

The author looks at GDP projections given by two US agencies – CBO and OMB:

This article evaluates real GDP forecasts of the Congressional Budget Office and the Office of Management and Budget. As a basis for comparison, the Blue Chip Consensus forecast is also evaluated. Tests in previous work assumed the forecast loss function was symmetric. This implies the political costs of a high or low GDP forecast are equal, so forecasts should be unbiased. This article differs from previous work by conducting tests assuming the forecast loss function may not symmetric.

Public choice models of political decisionmaking suggest government agencies such as the CBO and OMB face pressures that are likely to result in systematically biased forecasts. In this article, a flexible loss function allows for estimation of a parameter that captures the degree and direction of any forecast asymmetry. Elliott, Komunjer, and Timmermann (2005, 2008) show that failing to account for loss function asymmetry negatively affects tests that evaluate forecast accuracy and efficiency in the use of information available to forecasters.

His results are different from the other research papers:

Evidence from the existing literature examining CBO and OMB forecast performance using the standard symmetric loss function is mixed. Some studies evaluate budget forecasts while others evaluate forecasts of economic activity, such as real GDP growth. Based on these efforts, three general conclusions can be drawn. First, short-run forecasts of GDP and revenues are generally unbiased while long-run forecasts of these variables have an upward bias.1 Second, both short and long-run forecasts of GDP and revenues usually fail tests of information use efficiency. Researchers find that forecasters do no use available information to improve their forecasts.2 Third, despite what are likely to be different political pressures on different agencies, most of the studies find forecast biases to be similar across agencies.3

Using a flexible loss function to evaluate the CBO, OMB, and Blue Chip Consensus forecasts, I find significant evidence of asymmetry in the forecast loss functions. The CBO and the Blue Chip Consensus have a downward bias in their forecasts of real GDP growth two and five years out. The CBO forecast is consistent with the private sector consensus. The OMB forecast loss function is also asymmetric. However, the OMB bias is in the opposite direction. OMB forecasters overforecast real GDP growth at the two- and fiveyear horizons by 5 percent and 14 percent respectively. I argue that this finding is consistent with incentives facing the two agencies. In addition, once the asymmetry of the forecast loss function is taken into account, the traditional finding that available information is not used in the forecasts is rejected in favor of the finding that government forecasters use available information efficiently. These results illustrate the importance of taking into account loss function asymmetries when evaluating the forecast performance of government agencies that are subjected to political pressures.

The paper has this really interesting discussion on how institutional design of these agencies leads to certain kinds of forecasts. On CBO and OMB it says:

The CBO and OMB are interesting agencies to study as their institutional designs differ. The OMB, as part of the executive branch, is controlled directly by the president and is likely to face significant pressure to bias its forecast. In contrast, the CBO, which reports to Congress rather than an individual or single party, is more independent.  The CBO is accountable to members of both political parties who have different political goals. By design, the CBO budget is independent of congressional budget committees (Krause and Douglas 2005). Given the greater institutional independence of the CBO compared to the OMB, the costs associated with more objective forecasts should be lower, resulting in less optimistic forecasts.

Former OMB and CBO director Rudolph Penner (2002) argues that the CBO does not want to differ from the consensus outlook. According to Penner, large deviations from the consensus would make the CBO look partisan. Also, having a forecast that aligns with the consensus makes it easier to defend it before Congress. Furthermore, Penner points out that outside advisors contribute to the CBO forecast, which is likely to move the forecast in the direction of the consensus. Frankel (2011b) makes the more general argument that outside input can temper overly optimistic outlooks and limit the influence of politics.

The results in this article support these ideas. First, the OMB loss function suggests a low real GDP forecast is more costly to an administration than a rosy outlook. In a sense, the forecast is biased in a direction—upward—that helps the administration avoid politically costly spending cuts or tax increases. Second, the greater independence from political pressure of the CBO and its desire to produce forecasts consistent with the private sector seems to hold. Both the CBO and the Blue Chip Consensus forecasts of real GDP growth have a similar downward bias.

Nice bit especially the discussion on institution design and forecasting. As the blogger was once part of the forecasting industry, he has seen all kinds of behaviors. Nice to see how researchers have connected to these behaviors.