Main menu:

History of RPE Thought

Posts by Tag

RSS New from the CCPA

  • Unpacking the details of Manitoba Hydro September 9, 2019
    What would a love view of Manitoba Hydro all entail.  Read report here.
    Canadian Centre for Policy Alternatives
  • CCPA submission to Treasury Board consultation on regulatory modernization September 6, 2019
    On June 29, 2019, the federal government launched a public consultation on initiatives intended to "modernize" the Canadian regulatory system. Interested Canadians were invited to provide input on four current initiatives: Targeted Regulatory Reviews (Round 2) Review of the Red Tape Reduction Act Exploring options to legislate changes to regulator mandates Suggestions for the next […]
    Canadian Centre for Policy Alternatives
  • Join us in November for the 2019 CCPA-BC Gala, featuring Nancy MacLean September 3, 2019
    Tickets are available for our 2019 Annual Gala Fundraiser, which will take place in Vancouver on November 21. This year’s featured speaker will be Nancy MacLean, an award-winning historian and author whose talk, The rise of the radical right: How libertarian intellectuals, billionaires and white supremacists shaped today’s politics, is very timely both in the US and here in […]
    Canadian Centre for Policy Alternatives
  • Report looks at captured nature of BC’s Oil and Gas Commission August 6, 2019
    From an early stage, BC’s Oil and Gas Commission bore the hallmarks of a captured regulator. The very industry that the Commission was formed to regulate had a significant hand in its creation and, too often, the interests of the industry it regulates take precedence over the public interest. This report looks at the evolution […]
    Canadian Centre for Policy Alternatives
  • Correcting the Record July 26, 2019
    Earlier this week Kris Sims and Franco Terrazzano of the Canadian Taxpayers Federation wrote an opinion piece that was published in the Calgary Sun, Edmonton Sun, Winnipeg Sun, Ottawa Sun and Toronto Sun. The opinion piece makes several false claims and connections regarding the Corporate Mapping Project (CMP), which we would like to correct. The […]
    Canadian Centre for Policy Alternatives
Progressive Bloggers

Meta

Recent Blog Posts

Posts by Author

Recent Blog Comments

The Progressive Economics Forum

Fairy Tales and Labour Force Surveys

This is a true story.

I made porridge for my children this morning, as I usually do.   When my son first tried it, it was too hot.  When I finally got around to eating, it after making the kid’s lunches, it was too cold.  But when my daughter tried it, it was just right (and no, she doesn’t have golden locks, but a recipe tip is that  peanut butter tastes great mixed in with porridge).

At about the same time, the media report on the labour force survey came on the 7AM news. And it was just about the same story. 

In June, the increase in employment had been too hot (rising by 93,000).  In July, it was too cold (falling by 9,000).  But the August figures were just about right, rising by 36,000.   This was almost exactly in the middle of the range that I predicted it would be after last month’s report.

Just like with fairy tales, I find this story comes around again and again with Statscan’s Labour Force Survey.  I haven’t done any back analysis, but from my casual observation, there often seems to a three month cycle with these labour force  figures–too hot, too cold, then about right–that repeats itself.

What I find odd is that so media attention is paid, comments made and probably markets moved over figures that clearly have a lot of noise in them.  There’s way more hyperventilating expended over this than exterted by my children over the temperature of their porridge.

What is almost always completely ignored in the media reports is how statistically unreliable the monthly labour force figures are.   For instance, the standard error (S.E. in the third column of major tables in the detailed report) for employment at a national level is 28,000.   This means that there is only 68% confidence that the “real value” of an increase of 36,000 is in the range of +8,000 to  +64,000.   At a more reliable confidence interval of 95% (the familiar “19 times out of 20” reported for public opinion polls), the range is -20,000 to +92,000, as is explained in the data quality section of the report and in a bit more detail in the Guide.   There’s still a 5% chance that the real value is outside this wide range.

And those are the confidence intervals for the unadjusted figures.   I’m sure that the seasonal adjustment process adds more variability.   And I expect that the unique sampling method of the LFS, with its panel data and possible reporting errors, may add even more noise and variability to the data.  

After obliging with some quotes, I explained these reliability problems to a well-known and intelligent CBC reporter when he called for some commentary, but after after expressing some surprise about this he didn’t show any interest.   Of course, no news is bad news for the media.

I’m not exactly sure when it happened, but I stopped paying any serious attention to the monthly labour force figures about ten years ago when there seemed to be fair too much back and forth variability in the monthly numbers they reported.   I’ve never been part of the labour force survey sample, but an economist friend of mine was when I lived up in the Yukon (where it is hard to escape being sampled) and he came out of the experience quite concerned about the reliability of the data.

Households are kept in the sample for six months, only the first interview is detailed and the information for all members of the household are subsequently reported by one member of the household by telephone.   This six month rotation doesn’t necessarily explain what appears to be be a three month hot, cold, just right cycle of accuracy, but perhaps there is some other adjustment they make that accounts for that.  

Don’t get me wrong.  I still use the annual labour force figures and I think the quarterly or three-month moving average figures are still fairly reliable.   But I wish Statscan would fix the problems that seem to exist with the monthly labour force figures.   They report that they use some of these modified sample methods in order to reduce response burden, but I suspect that a lot of thsi was really done to reduce costs.   We now know that similar cost cutting in the 1990s for the survey size of employment, earnings and hours resulted in major problems for the reliability of that survey at the provincial level and they’ve made some recent changes to improve those figures.

Is it expecting far too much to hope (especially in light of looming federal budget cuts and this government’s attititude towards accurate information) that Statistics Canada will try to fix some of the problems with the Labour Force Survey, considering how much attention is paid to it?  

I’m sure there are many others much more informed about the internal workings of this survey who could comment more knowledgably about it.  Is there anybody at the Stats barn (or elsewhere) with something to add about this?

Enjoy and share:

Comments

Comment from Paul T.
Time: September 10, 2010, 11:59 am

This is a true story, as was reading Toby’s post I was listening to Nick Cave’s song – We Call Upon the Author

I am not joking.

We call Upon the Author to explain,

mainly because we are tired of getting lost in the variance games. (my line)

Toby you get most of it all right- lots of problems with the LFS and with the Census about to take a hit- we are only bound for even more variance-and undoubtedly a lot more bias.

Soon the LFS will only measure employment within the walls of the castle- the sea of poverty surrounding it, is not worthy of being measured.

At a minimum they are going to have to think very long about how to bring the reliability of the estimates up to a level that is reasonable. We are moving in the wrong direction- of course that is undoubtedly what the tories want- no legitimacy in the information makes it a little harder to pinpoint the problem.

Like I said yesterday- have a look at consumer bankruptcies in some of the CMAs had hit by the recession, and you will get a lot clearer picture of main street. On top of all the quantitative problems the extension of precarious work into the inner reached of the middle class is not showing up in the LFS so again the qualitative is discounted.

I will end this comment with a Nick Cave line from the song – as I think there is a reason it was playing when I read this post.

“Who is this great burdensome slavering dog-thing that mediocres my every thought?
I feel like a vacuum cleaner, a complete sucker, it’s fucked up and he is a fucker
But what an enormous and encyclopedic brain
I call upon the author to explain

Oh rampant discrimination, mass poverty, third world debt, infectious disease
Global inequality and deepening socio-economic divisions
Well, it does in your brain
And we call upon the author to explain”

Write a comment





Related articles