News & Events
After the Long-Form: Pursuing Sound Public Policy in a Land that has Lost its Census
On October 10, 2012, Environics Institute President Michael Adams gave an invited address at the 20th Anniversary celebration of Caledon Institute of Social Policy, a leading non-partisan think tank conducting research and analysis on important social policy issues in Canada. The full text of his remarks are provided below >>>>>>>>>>>>>>>>>>>>>>
It is an honour for me to be asked to speak at Caledon's 20th anniversary celebration.
I have admired the work of Caledon for a very long time and count Alan, Judy, Ken, Michael and Sherri as friends as well as respected allies in the pursuit of evidence-based, progressive public policy.
I know I am not alone in my admiration for the Caledon Institute. I think it is fair to say that as Canadians we can thank Caledon for the National Child Tax Benefit; for the Family Caregiver Tax Credit introduced in the 2011 budget; and for substantial intellectual contributions to the federal government’s move toward a First Nations Education Act.
My own respect for the Caledon brain trust is such that I have asked one of the brains, Michael Mendelson, to chair the board of the Environics Institute. I also regularly involve Alan and his colleagues at Maytree in my research projects, most recently our Citizenship Study. Speaking of Maytree, Ratna Omidvar, Maytree’s president, also serves on the board of the Environics Institute.
All this to say, we are practically family. But this is one of those families where you have to sing for your dinner, which is coming up shortly I believe.
Despite these fond introductory remarks, I am not here to praise Caledon but to lament the cancellation of the mandatory long-form census, something I wrote about in the November 2011 edition of IRPP’s publication Policy Options.
Policy Options editor Ian Macdonald told me the census article was the most downloaded in the history of the publication. Perhaps he tells all his contributors they are eminently downloadable—but I will assume this was not a matter of personal flattery, but actually a measure of how many Canadians who care about evidence-based public policy were deeply upset about the government’s decision on the census.
I for one was surprised at the strength and breadth of the reaction. I knew that people like those of us in this room would be upset: academics, policy wonks, pollsters, and other data nerds. But the decision was met with much more resistance from the public and from civil society groups than I would have expected.
The website Data Libre has tracked organizations supporting and opposed to the census decision; the tally is currently 488 organizations and opinion-leaders against, and 11 in favour.
Among the 488 are some pretty strange bedfellows: the Canadian Association of Retired Persons as well as a range of student groups; the Province of Quebec and the Canada West Foundation; the Toronto Star and the C.D. Howe Institute; the Rural Ontario Institute and the Federation of Canadian Municipalities; bank economists and anti-poverty groups; a group called Queer Ontario and the Roman Catholic Bishops. Strange bedfellows indeed.
I guess Canadians of all ages, ideological stripes, regions, and sexual orientations love their data and value solid evidence about their society and how it is changing over time.
The census might not be a ballot question for more than about 10,000 people across the country—but I believe that in a democracy, decisions like this will ultimately come home to roost. Although annoying one group to please another is certainly a normal part of politics, the census decision seems so narrowly tactical, crudely populist, and contrary to the public interest that it may have put a burr under the saddle of even some Conservative Senators, MPs and cabinet members. Who knows—with a few more burrs, they might even be inspired to speak up.
But back to the substance of the census issue. Let me take a minute to remind you of the questions that are on the short-form census, the questions that are on the long-form, and the uses to which Canadians’ answers are put.
The short form, which I presume everyone in this room filled in, unless you wanted to test the government’s tough-on-crime agenda, asks about the number of residents in your home, their relationships to one another, their age and gender, whether they farm, and what languages they speak. Basic population data for the most part.
The long form (formerly mandatory, now the optional National Household Survey) asks a wider range of questions, touching on citizenship and immigration status, income, occupation, child care, housing, and so on.
Long form data has in the past been used by businesses, NGOs, religious groups—and, oh right! governments!—to make plans and evaluate the success of their activities.
You have certainly heard of the United Way’s “poverty by postal code” work. Enabled by long form data, of course.
Experts have used long-form data over the past several years to warn us about worsening employment outcomes for immigrants.
U of T professor Jeffrey Reitz has used the Ethnic Diversity Survey, itself founded on long-form data, to analyze the complex relationships among ethnic belonging, discrimination, and economic inequality—important issues for a country that sees itself as a world leader in multiculturalism.
If you want to know whether the educational attainment of the Aboriginal population has improved in the last decade, you may have trouble. The long-form Census was the only information we had on the number of Aboriginal students who completed high school,community college or university. There will be no similarly reliable population-based data on educational attainment not just by Aboriginal people, but by any other grouping of the population.
In addition to its research value, census data has a huge range of very practical applications. If you work in public health and can only set up one TB clinic in an area, wouldn’t it be helpful to know where there is a high concentration of recent immigrants from countries with a lot of TB?
Without long-form data, there is more guesswork for people delivering public services, which makes our government not leaner but simply stupider and less efficient.
My Institute did the first ever major study of Aboriginal people living in Canadian cities. The study would have been impossible—or certainly vastly less credible—if the census hadn’t enabled us to set our quotas for how many people we should be reaching in each identity group (First Nations, Mètis, and Inuit) by gender, age, and education, specifically for example, how many young female Mètis with a post-secondary degree we should interview in Saskatoon.
No sample frame means data whose statistical reliability is greatly reduced.The less reliable our data becomes, the more our policy decisions will be vulnerable to guess work and, of course, anecdote. And as I heard a clever person say once, the plural of anecdote is not data.
Some might protest that the long-form still exists and millions of Canadians will fill it out; it’s just not mandatory anymore. This is true. The problem with a voluntary census is that some people don’t answer it, and those who don't answer are often the most vulnerable in our society.
The response rate for the 2011 National Household Survey (aka the long-form) was 68.6 per cent. (This is 68.6 percent of the one-third of the population to which the forms were delivered; so it means we have answers from about 23% of Canadians.) Who didn’t answer? People with below-average literacy? People whose first language is not English or French? People who have come from countries where it is not safe to share personal information with the government? Maybe.
A 68.6 percent response rate is excellent for an ordinary survey. But the census is not an ordinary survey. It’s the one that lays the foundation for all the others, including the Labour Force Survey and the Survey of Household Spending.
Even if the quality of the 2011 snapshot of Canada were excellent, the methodological leap we have made in this wave makes it more difficult to compare the data with data from past waves. It is also more difficult to compare Canadian data with data from other countries. Longitudinal and comparative data are the foundation of good social science, (not to mention, selfishly, my six books on social change).
Readers of the Economist magazine will be familiar with those wonderful charts they do that so cleanly compare various countries on one metric or another. I have a nightmare in which I open my copy one day and find blanks where Canada should be—or an asterisk and some disclaimer about the unreliability of the data.
So much for Northrop Frye’s observation of Canadians venerating the accountant more than any other society. You know: Americans make money; we count it.
Of course, a lot of researchers are working hard to draw sound, usable data from the National Household Survey. Environics Analytics, for instance, has made big investments in correcting for the bias expected to reside in the data. And because there are past census waves and other surveys to draw on, they should be able to get a pretty good picture of Canadian demographics and trends—at least for the 2011 census wave, if not beyond. Still, fixing the problems with the census data is a task we didn’t have five years ago. Turning the National Household Survey into a source of solid insights will take a lot of extra time and analytical steps that would not be necessary if our federal government had been willing to sustain our country’s traditional commitment to sound quantitative research methods.
Sometimes public policy based on conviction alone can be fine. It worked quite nicely with the abolition of slavery. But for a great many policy questions we need the tools of science to guide our decisions. The tools of natural science help us understand and respond to issues like climate change, and the tools of social science play the same role when it comes to making decisions about city planning, where to locate services, where to target social programs, and so on.
It is science above all, not ignorance of the facts, anecdote or ideology, that has vastly extended the span and quality of human life over the past few centuries. I agree with my golfing buddies that it would be nice to see that trend continue.
We have a government that has become quite selective in its use of data. In economic policy, data seems to be acceptable to them. But data does not seem very welcome at tables where social policy is being discussed. With crime in decades-long decline, for instance, this government feels it’s time to build more prisons and impose stricter sentences.
If our leaders are so cavalier about the evidence when good data exists and is widely available, what will happen when the facts are actually unknown? And this is by no means a concern that pertains only to the current government. I suspect fiscal conservatives would be very troubled to see a more left-leaning government pour money into, say, teacher compensation without gathering any data about whether those investments in our educators are actually helping student outcomes. Data helps to keep us all honest, regardless of our politics. And that’s one reason why all those disparate pro-census groups I mentioned earlier find common ground on this issue.
The question of data and ideology is one I would like to reflect on a little more. Some have called the elimination of the long-form census a libertarian decision. From one perspective, that makes sense. First, it makes sense that someone who is suspicious of government would not want government to be poking around asking about their religion or ethnicity. Secondly, a certain kind of small-government advocate might say that data is food for the hungry beast of government. Data exposes problems, and then government and its supporters, meddlesome bleeding hearts by nature, want to start fixing them.
People who see things this way believe that if a thing is worth doing—if a trend is worth measuring, or if a problem is worth fixing—then the private sector or voluntary organizations like religious groups will take care of it. If you want small government, the thinking goes, then you have to starve the government beast of the data that feeds its brain. This is a kind of Ron Paul take on the census, shared by the likes of the Fraser Institute north of the border in their efforts to roll back the welfare state western societies built in the 20th century.
There is, of course, another small-government perspective, articulated by William Robson of C.D. Howe, who argues that the census helps to tell you whether government programs like public education are working. “For those who want government to do less but do it better,” Robson writes in a defence of the mandatory long-form census, “good information is indispensable.” Unfortunately, Robson’s rather sensible part of the small-government camp is not winning the battle at the moment.
The side that’s winning is the side that favours a battle between your opinion and mine, your anecdote and mine, your unverifiable assumption versus mine. And as we know from American talk radio, this kind of debate is extremely rich and addictive food—but not very nutritious.
But let’s not cede the field to Rush Limbaugh just yet. While we wait to see whether the long-form will be restored to us or the still compulsory short form grows to include more essential items, lots of people are hard at work making the best of a disappointing situation. Caledon itself has taken over the major task of producing annual cross-Canada data on welfare. This work was previously undertaken by the National Council on Welfare, an in-house agency abolished by the federal government in the 2012 budget.
The Canadian Council on Social Development (CCS), under the leadership of Peggy Taillon, has pulled together over 25 consortia across the country. These consortia comprise federal, provincial and local government bodies, social agencies such as United Ways, health agencies, police services and the like. They have created the most comprehensive data hub in Canada to increase access to good information, and also to offer more sophisticated analytical capability –all with a view to facilitating the development of data-driven, empirically based social policy, programs, and service delivery. (To mention just one example, the United Way Poverty by Postal Code project I mentioned earlier relied on the CCSD data program and received analysis support from CCSD.) Environics Analytics is providing access to its Adjusted Census Data for CCSD and its members at no cost. It is also offering a reporting and analytical tool called Envision, at greatly reduced cost.
Personally, I don’t think Canadians have heard the last of the census debate. But you have heard the last of it from me tonight.
But before I end, however, in the spirit of Caledon and to mark the Institute’s 20th anniversary, I would like to propose an idea. I would like to see our community of supporters of evidence-based policy celebrate some of those who have championed good data—even when that stance has been unpopular with the powers that be.
I would like us to honour those who have shown courage to speak up and stand out, in a country better known for its deference to elite power than for its appreciation of contrarians. We have many awards for those who play by club rules in the service of their country, province, or city. And generally this is a good thing. But what I propose is a prize for those who dare to defy: those we have spoken truth to power in the wider public interest in the full knowledge that power can be mean and vindictive.
Munir Sheikh strikes me as an obvious first candidate for the award, the former chief statistician who resigned from his dream job rather than act as the fig leaf that covered the otherwise naked emperor of the National Household Survey. A deputy minister who resigns on principle. When was the last time that happened?
My fellow pollster Allan Gregg recently stuck his neck out at Carleton University in a rousing defence of intellectual honesty, warning of disturbing trends toward Orwellian governance. Frank Graves, another pollster, dared do scientific surveys for the Council of Canadians on the Robocall scandal and was subsequently attacked not for the quality of his research but ad hominem for his character and supposed political bias.
You will think of other shot messengers, I’m sure.
I think this could be a wonderful annual event for Caledon—a yearly celebration of brave Canadian nerds: people with both brains and…guts. (I might use another word on the golf course.) “Worthwhile Canadian initiatives” (as the famous boring headline put it) have a long and distinguished history. We should be proud of those whose intellectual integrity has helped lay the foundation for them. And we should be very grateful to those who make clear lenses through which we can view our society.
Anyone like the idea of a Canadian Galileo award to recognize those who dare defend the ideals of the enlightenment in this country?