This was originally published on HS2, the Regional Impact: Prominent supporter of HS2 admits economic case is not rigorous. By Prof Mike Geddes.
Bridget Rosewell of the Volterra consultancy has joined in the debate triggered by the KPMG report on the possible economic benefits of HS2. Rosewell rejects the criticisms of the KPMG study by Professors Overman and Graham that the KPMG work is insufficiently rigorous and the conclusions too optimistic (1).
Notably, Rosewell does not engage with the specific criticisms of KPMG’s methodology made by Overman. Instead, in a rambling argument, she makes a series of controversial claims.
She appears to dismiss a statistically rigorous approach to estimating economic benefits, arguing that the available data is messy and imprecise, and ‘statistical rigour is not a substitute for careful consideration of what the issues might be and what the statistics might miss’.
This displays confusion about what constitutes rigorous methodology. A rigorous approach is precisely one which assesses the strengths and limits of the data available, utilises it appropriately while making transparent how it is used, and draws appropriate conclusions which are justified by the methodology and respect the limits of the data. The criticisms of the KPMG work are that it fell far short of such a rigorous approach.
Rosewell then seems to try to rubbish serious methodological problems with estimating the economic impact of transport investments, such as the problem of causation, ie does the economy generate the transport system or the other way around? Here she says she finds it hard to envisage the former. In fact, she need look no further than the Jubilee line extension to London Docklands, which followed on from, and clearly was a case of government responding to, economic development in Docklands.
More dissing of the possibility for a rigorous methodological approach then leads to the punchline. Rosewell ‘judges’ that Crossrail will ‘generate total output returns of around £80bn on an investment of £15bn’ and therefore HS2’s investment of ‘around £40bn should, based on analogy like this, generate £200bn in new jobs, incomes, additional commuting to productive centres, taxes, dividends and so on. This kind of broad, historically informed analysis is in line with the sort of results that KPMG have generated’.
Now this is curious, because it was Bridget Rosewell’s Volterra who helped to produce the assessment of the economic benefits of Crossrail. This has been updated several times, and the latest (2007) version estimates the GDP benefits over 60 years from Crossrail under three economic scenarios – high, mid and low (2). The (unlikely) high figure is only £67bn, while the low one is £36bn, and all of these are considerably higher than the earlier estimate in 2005 of £20bn. This raises two issues. In the first place, Buchanan/Volterra’s work on Crossrail purports to employ the kind of rigorous methodology which Rosewell now seems to dismiss. Secondly, none of the figures are anywhere near the £80bn which Rosewell now ‘judges’ to be valid. So where does this ‘judgement’ come from? Surely not just from a desire to come up with a figure which helps to bolster the much-criticised KPMG work?
The conclusion to be drawn from this is that ‘judgement’ must not be allowed to substitute for a rigorous assessment of the case for and against HS2, in which both quantitative and qualitative data and evidence are carefully and honestly analysed. Where ‘hard’ evidence can usefully be supplemented by Judgement, the basis upon which the judgement is made needs to be set out fully and transparently.
1. http://volterra.co.uk/statistical-rigour-and-hs2
2.The Economic Benefits of Crossrail. Colin Buchanan and Partners with Volterra. 2007