When Good Data Goes Bad: The NHS2011
This piece was Â publishedÂ today in the Globe and Mail’s Economy Lab.Â
Two findings stand out in the National Household Survey (NHS) data released Wednesday, both critical in this post-recession era of uncertainty:
1)Â A quarter of Canadian households spent 30 per cent or more of their pre-tax income on shelter, the official measure of housing affordability.
2) There was virtuallyÂ no change in the rate of home ownershipÂ between 2006 and 2011. It was 68.4 per cent in 2006, and 69 per cent in 2011.
Iâ€™ll explain the significance of both; but first, letâ€™s consider how this information was collected.
The federal Conservative government spent an extra $22-million* to replace the long-form census questionnaire with the survey, going from a 20-per-cent sample of households receiving the long form in which responding was mandatory, to a 30-per-cent sample of households who could answer the questions if they felt like it. Why? The feds insisted the census was a coercive instrument of government that violated of your privacy. (I know. See my blogs from summer and fall 2010Â here.Â Statistics Canada has a peerless record in assuring complete confidentiality in both voluntary and mandatory information-collecting.)
Of course, non-response rates are higher with voluntary surveys, and various studies show there is a predictable bias to those who are less likely to respond: The rich and the poor; those who are not fluent in either official language; Aboriginal populations; and people with disabilities.
In places where more than half the sample didn’t respond, StatCansuppressed the findings. Fewer than 60 per cent of Saskatchewan’s census subdivisions had useable information.
For these reasons, Statistics Canada made clear that youÂ canâ€™t compare NHS findings to previous census or other surveysÂ they have conducted. The NHS is a family portrait with a lot of people missing.
That means we canâ€™t ask the important questions about the differences in how we fared before and after the 2008 crisis on critical issues such as housing affordability. We also donâ€™t really know how big our problems are, and to what degree they are widespread or localized.
So when we read that a quarter of Canadian households are paying more than 30 per cent of their pre-tax incomes on shelter, we know itâ€™s a conservative estimate (if you’ll pardon the pun) because of the higher non-response rates among low-income households. When we read that twice the proportion of renters than owners are paying more than 30 per cent of their pre-tax incomes on shelter (40 per cent of renters versus 18 per cent of owners), we know that this understates whatâ€™s going on, because more renters are low-income than home-owners.
And how many Canadian households are renting? Itâ€™s not clear that we even have that basic question right. Statistics Canada notes someÂ unexpected resultsÂ with the NHS. It should be the inverse of the 69 per cent of owners â€“ that is, 31 per cent of Canadian households rent their dwelling. But the proportions may be off.
Itâ€™s odd that the NHS shows the rate of growth in home ownership fell precisely when interest and mortgage rates dropped to historic lows between 2006 and 2011. (After all,Â interest ratesÂ werenâ€™t this low even in the 1930s, andÂ mortgage ratesÂ are lower than at any time since the 1950s, when authorities started collecting the data.)
Home ownership is the way many people create economic security and maintain purchasing power when incomes fluctuate. Tracking that variable authoritatively can tell us whether troubling levels of household debt are being driven by a rising incidence of home ownership (perhaps not such a bad thing), higher housing prices (bigger mortgages), or other causes. The census did that.
Census results also alerted us to systemic over-crowding in First Nations and Inuit housing. The NHS slapped blurry goggles on our vision. We know less than before about the living conditions of these communities now. Same story for immigrant and newcomer communities.
These are the two nodes of future population growth in Canada â€“ where the evidence weÂ doÂ have suggests that troubles are brewing.
The main finding of the NHS is that, with weaker ability to track change or measure a growing portion of society, weâ€™re losing sight of the Canada weâ€™re becoming.
* Note:Â An earlier version of this piece said the National Household Survey cost an additional $30-million over the cost of the long-form census that it replaced. It was budgeted to cost that much more, but Statistics Canada has indicated that the additional cost was only $22-million.
The point that Armine forgot to mention- it is not the 22 million extra cost of the NHS that matters, it is the entire cost of the NHS that matters- as it actually destroyed the outcomes from a tradional census, so in fact the tory blunder is 250 Million dollars which is the cost of the census, not $22 million in added costs that she estimated. And that is just the cost of the data, not to mention the whole massive array of cost and benefits that we now forgo with having a massive flawed data set guiding everything from school building, hospital expansions, to workforce and training demands. And then throw in the whole political cost of repairing this and you are well into multi billions of dollars.
Great article Armine.