Cafe Hayek

Sickness is bad for your health

Evidently, smoking, obesity and diabetes are bad for your health. So is sickness. Death is bad for your life expectancy. So discovers the New York Times:

THROUGHOUT the 20th century, it was an American birthright that each generation would live longer than the last. Year after year, almost without exception, the anticipated life span of the average American rose inexorably, to 78 years in 2005 from 61 years in 1933, when comprehensive data first became available.

But new research shows that those reassuring nationwide gains mask a darker and more complex reality. A pair of reports out this month affirm that the rising tide of American health is not lifting all boats,...

I'm going to stop the quote there for a moment.

Is the reality really "darker and more complex?" Does anyone really think that "the rising tide of American health" lifts all boats?

Does anyone think that rising life expectancy is really a birthright?

If you go sky-diving every week without a parachute or even with one, you don't live as long as the average. If you smoke a lot and eat too much, you might not live as long as the average:

The most startling evidence came last week in a government-sponsored study by Harvard researchers who found that life expectancy actually declined in a substantial number of counties from 1983 to 1999, particularly for women. Most of the counties with declines are in the Deep South, along the Mississippi River, and in Appalachia, as well as in the southern Plains and Texas.

The study, published in the journal PLoS Medicine, concluded that the progress made in reducing deaths from cardiovascular disease, thanks to new drugs, procedures and prevention, began to level off in those years. Those gains, as they shrank, were outpaced by rising mortality from lung cancer, chronic obstructive pulmonary disease and diabetes. Smoking, which peaked for women later than for men, is thought to be a major contributor, along with obesity and hypertension.

Read the rest of the story if you really want to know why this unsurprising absolutely unstartling finding confirms the world view of John Edwards.

Mark Perry on Krugman and the Jobless Rate

Carpe Diem's Mark Perry carefully investigates and clearly explains -- much better than I do here -- the likely source of Paul Krugman's mistaken claim, in yesterday's New York Times, that "the percentage of prime-working-age Americans without jobs . . . is historically high."

Mark reached the same conclusion that I reached: Krugman mistook "men" for "all workers."

Using what is likely the same data source used (but carelessly so) by Krugman -- and looking at the jobless rates of both men and women -- Mark concludes

Krugman says "the percentage of prime-working-age Americans without jobs is historically high," which is clearly not accurate. It would be more accurate to say that it is close to being historically low.

Krugman's Peculiar Sense of History

Paul Krugman makes a claim in his column (appearing in today's New York Times) that is both wrong and misleading.

The column features his familiar theme that ordinary Americans are suffering terribly as a result of the decrepit shape of America's current, GOP-ransacked economy.  Krugman's offending statement is the one in bold:

Our [that is, ordinary Americans'] bleakness partly reflects the fact that most Americans are doing considerably worse than the usual economic measures let on. The official unemployment rate may be relatively low — but the percentage of prime-working-age Americans without jobs, which isn’t the same thing, is historically high. [emphasis added]

Sounds ominous.  But what does this figure mean?  The closest statistic that I can identify to "percentage of prime-working-age Americans without jobs" is to start with (one minus) the labor-force-participation rate of Americans aged 25-54.  Then adjust that figure for the unemployment rate of Americans in that age group.

For example, if 90 percent of Americans between the ages of 25-54 are participating in the labor force (that is, either have or are seeking employment), then ten percent of Americans in this age-group are without jobs by choice (some, for example, are stay-at-home parents).  These ten percent of Americans without jobs are not "unemployed," for they are not in the labor force.

Then, to determine the actual number of prime-working-age Americans who are "without jobs," we must add to this 10 percent of Americans, in this age group, who are without jobs because they aren't in the labor force whatever percentage of the remaining 90 percent of Americans aged 25-54 are unemployed.  BUT because there's no reason to suppose that unemployment is hitting workers in this age group today significantly any harder or less hard (relative to workers in other age groups) than in the past -- and because, by Krugman's own admission, today's unemployment rate of 5.1 percent isn't especially high -- we can ignore this unemployment rate for my purposes.

So what am I getting at?  This: Krugman's statement that "the percentage of prime-working-age Americans without jobs, which isn’t the same thing, is historically high" is, as I said above, both wrong and misleading.

Look at Figure 1B on this page from the San Francisco Fed.  (HT Russ.)  It does not show the labor-force-participation rate for all Americans aged 25-54; rather, it breaks down the participation rate for Americans in this age group by sex.

Look at the labor-force-participation rate of men (aged 25-54).  That rate is indeed lower than in the past.  For the full time period reported in this Figure, that rate is indeed at an all-time low.  It peaked in 1953, and has been declining ever since.

Now look at the labor-force-participation rate of women (aged 24-54).  Not surprisingly, it has surged in recent decades, although leveling off a bit since around 1990 and even declining a tiny bit since around 2000.

It's highly unlikely that, given that today prime-working-age women participate in the labor force at rates vastly higher than was true even in, say, 1975, that the percentage of all Americans aged 25-54 who are "without jobs" is today lower than it was thirty or even twenty years ago.

Whatever is the percentage of prime-working-age Americans who are today "without jobs," it surely isn't -- contrary to Krugman's hysterical claim -- "historically high."

One possibility is that Krugman saw these, or similar, data and confused "men aged 25-54" for "all workers aged 25-54."  If so, then he's correct that the percentage of men in this age group "without jobs" is "historically high" (where "history" here is confined by the data that go back no farther than the late 1940s).  But today's "historically high" figure is part of a trend that began during the first year of the presidency of Dwight Eisenhower.  At that time, George W. Bush was seven years old.

(Note also that, although the labor-force participation rate of prime-working-age men has been declining for a half century, the trend is slight.  Even today's "historically" low figure is pretty darn high.)

So, to summarize.  Krugman is simply wrong to assert that the percentage of Americans of prime-working-age without jobs is "historically high" -- and misleading to suggest that, whatever this percentage might be today, that it is evidence of some major economy malady.  A rise in the percentage of prime-working-age persons "without jobs" might just as well reflect greater middle-class prosperity -- resulting in more years of schooling or early retirement -- as it reflects economic hardship (such as workers so discouraged by their futile job searches that they just call such searches to a halt).

BTW, this post from Megan McArdle on labor-force-participation is worth reading.

Striking out

Sometimes I get depressed about the quality of statistical work in economics. Then I read something from another social science. Here is a recent study where psychologists find that having the initial "K" increases your chance of striking out when playing professional baseball. Why? Well, it's obvious isn't it? The letter "K" is used when keeping score in baseball to represent striking out. So it's obvious now isn't it? Still don't get it? Neither do I. But hey, it's in the data. Between 1913 and 2006, players with first or last initial "K" struck out 18.8% of the time compared to 17.2% for the fortunate players unhandicapped by their initials. Here is the "explanation" of the authors:

Despite a universal desire to avoid striking out, K-initialed players strike out more often.  For those players, we argue that the explicitly negative performance outcome may feel implicitly  positive. Even Karl “Koley” Kolseth would find a strikeout aversive, but on the whole, he might  find it a little less aversive than players who do not share his initials, and avoid it less  enthusiastically.

But why? Why would having the initial "K" make striking out more pleasant? I just don't get it. The authors go on to "test" their theory by looking at grades of a sample of MBA students:

The MBA students in our sample are well aware of a direct connection between academic  performance and successful job placement. Nevertheless, despite the pervasive desire to achieve  high grades, students with an unconsciously-driven fondness for C’s and D’s were slightly less  successful at achieving their conscious goal.

That is, Charles Darwin received poorer grades than Alan Alda. But it turns out that Alan Alda didn't do better than the non-ABCD initialed:

Interestingly, A- or B-initialed students did not perform better than students whose  initials were grade-irrelevant. There are two possible explanations for this. First, students with  grade-irrelevant initials may already be maximally motivated to succeed. Second, because  performance is determined by motivation and ability, any increased motivation to succeed that  arises from having initials that match positive performance outcomes may not necessarily  translate into increased performance.

There is, of course, a third explanation: there is no real relationship and the authors have been fooled by randomness. Yes, their results are statistically significant. But how many relationships did they explore before finding the ones that were statistically significant. And ho many relationships are there to explore? To really test the theory, you'd have to look at baseball players with the initial "E" and see if they commit more errors than others. You'd have to look at guards in the NBA to see if those with initials "A" have more assists. Centers whose initials include an "R" should be better rebounders. You'd have to look and see whether students with the initials IC were more likely to take an "incomplete" in a class.

I guess Rabbi Jonathan Sacks, the Chief Rabbi of England should have been a football player. Or maybe he just gets fired more often than the average Briton because it doesn't bother him as much as someone with a different last name.

Did Kafka know baseball scoring? Does this explain why he found success in life so difficult? Is this why he named a character "K"?

Do players whose initials are a backwards "K" strike out looking more than the average?

Are workers falling behind because of rising health care costs?

As is often the case, the glass is half-empty over at the Washington Post (HT: Steve Chapman). The headline on today's front page:

Rising Health Costs Cut Into Wages

Higher Fees Squeeze Employers, Workers

The article begins:

Recent history has not been kind to working-class Americans, who were down on the economy long before the word recession was uttered.

The main reason: spiraling health-care costs have been whacking away at their wages. Even though workers are producing more, inflation-adjusted median family income has dipped 2.6 percent -- or nearly $1,000 annually since 2000.

Employees and employers are getting squeezed by the price of health care. The struggle to control health costs is viewed as crucial to improving wages and living standards for working Americans. Employers are paying more for health care and other benefits, leaving less money for pay increases. Benefits now devour 30.2 percent of employers' compensation costs, with the remaining money going to wages, the Labor Department reported this month. That is up from 27.4 percent in 2000.

So the point is that because of rising health-care costs, workers are worse off. Pretty depressing. And there's a graphic to help make the point:

Unfortunately, the text accompanying the graphic is a non-sequitur--"The annual increase in the cost of benefits for workers has far outpaced wage increases in recent years." That sounds bad. That sounds like the costs of health care is growing faster than wages and higher health care costs would seem to imply a lower standard of living for workers. But it doesn't imply that at all. It simply says that the benefits component of compensation has been growing faster than the wage component.

And what does it mean that the "cost of benefits" is rising? What does the word "costs" mean in that phrase? It's a cost to employers. But to workers, benefits are, well, benefits. Higher compensation. What the chart shows is that benefits have been rising rapidly. That's actually a good thing. Yes, part of that increase is that employers are covering health care costs that have been rising. But it also includes other types of benefits, forms of compensation that are usually tax-free to employees.

To find out whether workers are doing better or worse than they once did, you want to look at whether total compensation is growing faster or slower than inflation.  Look at the graphic--it shows that other than a couple of months at the end of 2005, total compensation has been growing between 3% and 4% a year since 2000. But between 2000 and 2007, the CPI increased about 20% or less than 3% per year. So real compensation has been rising.

It's not clear from the graphic whether this is compensation per worker or per hour or whether it's total compensation across all employees. So the conclusion could be distorted by increases in the labor force since 2000. But what is clear is that the graphic has nothing to do with the pessimism of the article and seems to refute it.

As for the 2.6% fall in median income since 2000, the one seemingly hard piece of evidence in the  article, if the number is from the Census (which I assume it is), the Census income figures don't include health care benefits or employer contributions to retirement. So those income figures tell us nothing about the role of rising health care costs' effect on the standard of living of the median worker.

The rest of the article is a series of anecdotes about how expensive health care is. There is no hard evidence for the central claim that rising health care costs have reduced the standard of living of the average American. In fact, the graphic implies the exact opposite.

 

How do we know what we know about economics?

I was asked the other day by a reporter if NAFTA had been a good thing or a bad thing for America. I said that both the proponents and opponents of NAFTA had no legitimate, unassailable or even suggestive, statistical evidence on their side. I said it was absurd to think that in a $14 trillion economy, you could tease out the impact of increased trade with Mexico and Canada and disentangle it from the thousands of other changes going on.

I suggested that anyone who provided an empirical case for or against the agreement was essentially being dishonest--using statistics selectively to make the case for a pre-existing world-view.

The reporter found this viewpoint unacceptable. Surely, he said, economics can help us answer the question of whether NAFTA has been good for America or bad. Or at least good or bad, for say Ohio.

I said no, there was no empirical evidence that would be decisive. It isn't just that it's hard to measure the net impact precisely. I argued that it can't be measured.

He refused to accept this answer. Surely I could answer the question of NAFTA's worth. I was just ducking the question.

I said, no, I wasn't ducking the question. I was answering the question. He just didn't like my answer.

But he couldn't accept the answer as being "I don't know," or more accurately, we can't know if by "know," you mean some kind of defensible statistical evidence.

The most well-known think tank that views NAFTA negatively, the Economic Policy Institute, argues that between 1993 and 2004, Ohio lost 49,886 jobs because of NAFTA.

Not 50,000, but 49,886.

For the U.S. as a whole, EPI estimates that 1,015,290 jobs have been lost. Not a million, but 1,015,290.

But it's not just the precision of the estimate that makes the calculation ridiculous. It's the methodology that measures jobs displaced (the verb the EPI study usually uses) by assuming that imports destroy jobs and exports create jobs. With this methodology, a trade deficit reduces the number of jobs.

By this logic, America would have fewer jobs if foreigners suddenly decided to give us cars rather than selling them to us. Free foreign cars would destroy jobs in the American car industry. Inexpensive foreign cars do the same thing. So do cars made by Americans that are produced with robots instead of human welders. If you believe that imports destroy American jobs, so does better technology that makes workers more productive.

If the EPI was as anti-technology as they are anti-trade deficit, they could go out and measure the number of buggy manufacturing jobs destroyed by the auto industry and the number of eyeglass manufacturing jobs destroyed by lasik surgery or the number of jobs in the medical profession destroyed by pharmaceutical innovation. Surely those numbers could be estimated with some rough accuracy.

And surely they would tell us nothing about whether the world is a better place because of those innovations. And they would tell us nothing about the impact of those innovations on total employment.

But most of us understand that higher productivity doesn't mean fewer jobs in the United States overall. There are two ways to see it. One is to see the increase in the total number of jobs over time as we have replaced people with machines in every corner of the economy where it's possible. So this suggests that technology doesn't destroy jobs.

The same is true of the apparent effect of trade deficits. Manufacturing employment was shrinking as a share of the total employment between 1950 and 1975 when the trade deficit was essentially zero. We run a trade surplus in agriculture. Yet agricultural employment shrinks rather than rises.

There appears to be no relationship between the number of jobs in America (or in Ohio for that matter) and the trade deficit or increased productivity.

But the opponents of technology or trade could argue that the failure to observe a negative correlation between jobs and technology or jobs and trade deficits is spurious. True, the number of jobs in America has grown while technology has gotten better and better. But if we hadn't had technology we'd have even more jobs. So we have to control for the other factors that might have made jobs increase rather than decrease.

That's very hard to do. I'd settle for hearing a list of the factors that might plausibly be offsetting the alleged effects of trade and technology. In the absence of that list, I'm pretty comfortable arguing that trade deficits and technology have little effect on the total number of jobs (rather than the number of jobs in particular industries). And the reason I feel that way is partly because of the total number of jobs rising but mainly, I suspect, because I have a belief about how the world works.

I use the word "belief" which might lead some to dismiss my views as simply a matter of ideology. But my faith in my view of how the world works has empirical support for its tenets. It's not just a fantasy or a dogma or convenient. I just don't have holistic empirical evidence on trade having no impact on the number of jobs.

My view of the world is that when we widen the ability of our fellow citizens and ourselves to trade with people other than just our own kind or nationality or religion or color, that that in turn allows trade to take place more productively. That in turn creates wealth, a higher standard of living. And that in turn creates new opportunities for employment that come along and replace the old.

This view of the world is supported by logic and lots of empirical evidence. But again, it's not the empirical evidence of an overarching kind that let's me evaluate one corner of increased specialization, say the overall impact of NAFTA.

When I told the reporter that the net job loss numbers of the EPI are meaningless because they miss many of the jobs that are created, he said, so tell me where to look for those new jobs. And again, I had to disappoint him. I said that I couldn't tell him where to look.

Why?

Because when we trade more with each other and use our limited resources more effectively, two kinds of new opportunities get created. We take the wealth and resources that have been freed up by more efficient trade (or the same effect when productivity increases) and we use those extra resources to produce more of what we like and to get some new stuff we couldn't have had before. And yes, we also get an expansion of export-related jobs.

But the export-related jobs can't capture all the job gains from having a more effective use of our skills and time and energy that comes from trading more widely. Especially if you run a trade deficit. That means that a lot of the new jobs that are going to be created aren't going to have anything to do with trade. Just like finding ways to make steel with fewer people and more machines leads to new jobs in the steel-machinery making business but in lots of other industries that you can't identify, that come from people being able to spend less on steel and having more resources to spend on other things.

So I can't tell the reporter where to look. He has to look everywhere. It doesn't make for a very good feature story.

So I'm sorry I can't be more helpful. But I do think economics has a lot to say in helping us understand the effects of increased trade. It just doesn't come from the bottom line of an empirical study.

P.S. I am ignoring any bureaucratic complications that occur in a real-world trade agreement. I wish we would just eliminate our tariffs and quotas rather than do all the bureaucratic and legal compliance wrangling involved in NAFTA.

P.P.S. For a pictorial version of these ideas with the data alluded to in the discussion, go here.

More on debt

Martin Brock, in response to this post, writes:

Still, the median debt is up 151% since 1989, adjusted for inflation, and if the "wealth effect" theory of the personal saving rate is correct, much of this increase is consumer debt (the cars and such). I don't see how the clarification makes this increase any less gloomy. Maybe it's not very gloomy at all, but if it's gloomy before the clarification, it's still gloomy after.

It's funny how numbers can be used to scare and mislead. One hundred and fifty one percent seems like a big increase. But the size of that number tells you nothing. It does sound scary. But it tells you nothing. Nothing. All it tells you is that in 1989, the median family with debt had debt of 22,000. In 2004, the median family with debt had debt of 55,000. Good or bad news?  You can't tell. It tells you nothing by itself. It could mean people are living more and more beyond their means. It could mean that people have more wealth and income and are buying bigger houses and nicer cars. There is nothing inherently gloomy or cheerful until you know more information.

The fact that net worth is up from 68,000 to 93,000 makes it more likely that it's cheery. But let's look a little deeper. Here's another chart from the Federal Reserve Board's Survey of Consumer Finances, the source of all of these data:

Look at the line called Goods and Services. Flat. That's trying to measure people living beyond their means, using debt to buy furniture, food and so on. That's the line the Washington Post should have used to look at whether debt was being used for what the article called "day-to-day expenses." What this chart shows is that the increase in debt is due to people buying bigger houses and investing in a little more education.

Worse than I thought

As I mentioned in this post, there were some strange things in the Washington Post article on the middle class, particularly in the graphic:

So let's take a closer look. As Randy pointed out in the comments, $55,000 seems like a lot of debt for the median family in 2004.  Look again. According to the chart, debt is $55,000 but mortgage debt by itself is $95,000.  A reader might think that the $55,000 is on top of mortgage debt. The description around the graphic says: "household debt more than doubled as more families used debt to finance day-to-day expenses." Is that what that $55,000 is about? Is that on top of mortgage debt? Whoa.

Well, no, it turns out.

The chart is incredibly misleading. Only the top line (income) and the bottom line (net worth) are true medians measured across all families.

The debt figure of $55,000 in 2004 (which supposedly is 151% higher than in 1989 to pay for day-to-day expenses) is actually ALL forms of debt INCLUDING mortgage debt. So how can that be? How can the median family have only $55,000 of all kinds of debt when there's $95,000 of mortgage debt all by itself?

That's because each line of the chart (other than the top line and the bottom line) is a subset of all families and a different subset.

So among families that have mortgage debt (maybe 40-50% of all families) the median mortgage debt among those families is $95,000.

But among families that have any kind of debt, (about 3/4 of all families) the median indebtednes including all kinds of debt is $55,000. That includes mortgages debt. So that $55,000 number has nothing to do with "day-to-day" expenses. And the scary 151% increase in "Debt" from 1989 to 2004 includes people choosing larger mortgages and nicer cars and going to college and everything people finance via borrowing.

So you can't add up any of the lines of the chart or even compare them to each other. They're each for a different subset of the population, the population who have that kind of debt or asset.

(This explanation of how those numbers must have been calculated comes from a Federal Reserve Board economist who compared the numbers in the Post graphic to Fed data from the Survey of Consumer Finances.)

 

The bottom line is the bottom line—net worth is up. That's a good place to start. The rest of the chart is basically meaningless for drawing the kind of conlcusions implied in the text.

Different Data

Here's a letter that I sent today to the New York Times:

Paul Krugman continues his drumbeat message that ordinary Americans are stagnating economically ("Winter of Our Discontent," November 26).  But data that he frequently cites to support his claim (especially from economists Thomas Piketty and Emmanuel Saez) are not of real flesh-and-blood persons through time; they are of statistical categories such as deciles or quintiles of income earners. Changing demographics and movements of persons from quintile to quintile mask potentially huge changes in the underlying reality.

Sure enough, recent data from the IRS that are of real-life persons reveal that ordinary Americans are prospering.  Economist Thomas Sowell summarizes some germane revelations of these data: "People in the bottom fifth of income-tax filers in 1996 saw their incomes rise 91 percent by 2005. The top 1 percent ... saw their incomes decline a whopping 26 percent.  Meanwhile, the average taxpayers' real income rose 24 percent between 1996 and 2005."

Sincerely,
Donald J. Boudreaux

A caveat: I briefly looked for these IRS data on line and couldn't find them (a fact, I'm certain, due to my being pressed today for time).  Sowell doesn't say if these data are adjusted for inflation or not.  My guess is that they are so adjusted.  But even if the reported percentage changes are in nominal dollars, then (1) the growth by 2005 in the real incomes of 1996's bottom fifth of income-tax filers would still be an impressively large 73 percent, and (2) the fall by 2005 in the incomes of 1996's top one percent of income-tax filers would be even larger than what the nominal figures (if nominal they be) suggest.

UPDATE: Tom Armstrong sent to me the pdf containing the IRS data.

Caveat emptor

Buyer beware, even when the item is only truth and the price is zero. The Washington Post reports (ht: Drudge):

The United Nations' top AIDS scientists plan to acknowledge this week that they have long overestimated both the size and the course of the epidemic, which they now believe has been slowing for nearly a decade, according to U.N. documents prepared for the announcement.

AIDS remains a devastating public health crisis in the most heavily affected areas of sub-Saharan Africa. But the far-reaching revisions amount to at least a partial acknowledgment of criticisms long leveled by outside researchers who disputed the U.N. portrayal of an ever-expanding global epidemic.

The latest estimates, due to be released publicly Tuesday, put the number of annual new HIV infections at 2.5 million, a cut of more than 40 percent from last year's estimate, documents show. The worldwide total of people infected with HIV -- estimated a year ago at nearly 40 million and rising -- now will be reported as 33 million.

Self-interest appears to have played a role in the estimates, as we cynical economists often point out:

Having millions fewer people with a lethal contagious disease is good news. Some researchers, however, contend that persistent overestimates in the widely quoted U.N. reports have long skewed funding decisions and obscured potential lessons about how to slow the spread of HIV. Critics have also said that U.N. officials overstated the extent of the epidemic to help gather political and financial support for combating AIDS.

"There was a tendency toward alarmism, and that fit perhaps a certain fundraising agenda," said Helen Epstein, author of "The Invisible Cure: Africa, the West, and the Fight Against AIDS." "I hope these new numbers will help refocus the response in a more pragmatic way."

We're Richer Today. Period.

Tyler, Arnold, and other bloggers have mentioned this nice study by Terry Fitzgerald that appears in the Minneapolis Fed's September 2007 issue of The Region.  (It's the first of three articles; the remaining two articles by Fitzgerald on this topic aren't yet published.)  In this opening number, Fitzgerald offers

a glimpse of the key findings from this article on wages. Microeconomic statistics showing stagnation, and macroeconomic statistics exhibiting growth, measure “wages” quite differently. When the data are adjusted so that they more closely measure the same conceptual object, the disparity between the microeconomic and macroeconomic statistics largely evaporates, and I find that labor income per hour for middle America has not stagnated. Rather, the economic compensation for work for middle Americans has risen significantly over the past 30 years.

This conclusion (and Fitzgerald's analysis from which it follows) makes sense to me.  While being aware of the dangers lurking in the practice of drawing conclusions about complex reality from one's own personal experiences, I nevertheless am certain that ordinary Americans today are immensely wealthier than they were 30 years ago.  I well remember the mid-1970s.  All telephones in middle America were attached to walls or desks; the television repairman was still necessary; going on-line was something New Yorkers did whenever they queued -- which, back then, they did often in order to buy gasoline for their automobiles that broke down much more frequently than do today's cars.  This was a time before cable t.v. was widespread, and before anyone but the super-rich could watch movies on demand in their own homes.  Prepared foods in supermarkets were tasteless and probably toxic.  And no one outside of New York City and a handful of other major metropolitan areas had any hope of browsing in a bookstore with a respectable selection of titles.

For further evidence that the 1970s was no golden age of abundance, come shop with me from a 1975 Sears catalog - and then let's see how much these goodies will cost us.

A Note on Income Data

This letter appearing in today's edition of the Wall Street Journal makes a good point:

Mr. [Alan] Reynolds [in an October 25 WSJ article] uses the term "individual" income several times when explaining the top 1% of income earners, and therein lies his problem. If only the IRS taxed me and my working wife based on our "individual" income we would save several thousands of dollars per year. The problem is that both taxes and the statistics are based on "household" income, which is really a euphemism for "marriage" income. If the statistics were based purely on "individual" income, I'm sure the wage disparity would not be that much worse today as when Ward and June Cleaver were raising the Beaver.

What's happened is that women entered the workforce and in the past few decades educated women have had their incomes match or even exceed that of men. Educated and upper income people have a tendency to marry one another. And because lower-income people are less likely to be married than upper-income individuals, the statistics are skewed even more. To have an honest discussion on this subject you first have to start out with honest statistics.

Steve Walde
Easton, Conn.

My response to Robin's latest

Robin Hanson continues to give me a hard time. First he quotes me:

When scholars can run hundreds of multivariate regressions at very low cost, it easy to convince yourself that the results that confirm your prior beliefs are the "right" results.  The Pragmatists ... believed that the rationalism of Descartes had a dangerous element of hubris. ... Your grandmother is right. She believes in certain things. When you ask her to justify her beliefs she shrugs and says she can't. ... Norms of behavior that survive, survive because they're effective even when no one understands why.

 

The Cartesian belief that you should examine every one of your beliefs ... the pragmatists argued that that was akin to examining the planks of your boat while you were at sea. ... All of which is to say that we shouldn't pretend to be scientific when we're only doing something that has the veneer of science. That's much more dangerous than saying, I don't know or we can't answer that question. ... I also did not mean to imply that data and evidence are irrelevant in how we form our beliefs about what is true. Or that our biases never get overturned.

Then Robin responds, combining my observations on pragmatism with my argument that we are unlikely to be able to use econometric analysis to settle the question of whether concealed handguns reduce crime:

I'm still confused.  Does Russ think his life will "sink" if he further considers his reasons for his believing in concealed handguns, even though it was he who raised this issue?  Does he think only scientists should want reasons for their beliefs?  Does he believe concealed handguns deter crime because his grandma had hidden guns and lived a long life?  Or is it because grandma's society had them and was prosperous?

Yes, if one prosperous society had concealed handguns that is at least weak evidence for the reasonableness of that behavior.  But since other prosperous societies forbid concealed handguns, I don't see how you can have more than a weak belief on this basis, unless you rely on one of those complex data analyzes Russ distrusts.  Russ doesn't seem to think there is simple clear data on concealed handguns, and he hasn't taken up my suggestion to claim that he is persuaded by theory, either complex and subtle or simple and clear.   And yet Russ still seems to retain a strong belief.  This seems literally unreasonable to me.

Robin is taking my observation about pragmatism and applying it to handguns. I didn't mean to. I brought up pragmatism in order to highlight the general dangers of excessive faith in reason. Assuming that econometric analysis always trumps an anecdote is an example of the potential dangers of econometric analysis. Yes, relying on anecdotes is lousy science. But lousy econometrics is lousy science, too. What my podcast with Ayres made me realize is that lousy econometrics may be the norm rather than the exception.

Such cynicism can come cheap. It also seems to leave us with anecdotes. Well, there's also common sense, intuition and general lessons gleaned from experience and empirical work that is less prone to manipulation.

So again, my question to my better-read colleagues in the profession--give me an example of a statistical analysis that relies on instrumental variables, for example, that is done well enough, that is so iron-clad, that it can reverse the prior beliefs of a skeptic.

Consider this excerpt from the summary of a paper by Dube, Eidlin and Lester that finds that Wal-Mart lowers wages when it comes to town:

A potential problem with studying store openings to estimate the impact on wages is that  Wal-Mart does not choose randomly where to expand.  If Wal-Mart’s expansion into  local markets were random throughout the United States across the ten-year period of study, then we could simply look at what happened to wages in counties after Wal-Mart's entry as compared to before.  But Wal-Mart may have taken into account several factors  for expanding into certain markets and not others, including the cost of labor in those  areas at that time.  Economists call this problem “selection bias.” In other words, Wal-Mart’s own criteria for expansion into certain markets may interfere with our ability to  test for a causal relationship between Wal-Mart entry and a change in local wages.   

In this paper, we devise a novel way to resolve this problem.  We take the fact that WalMart store openings spread out over time starting from Arkansas and moved outward to  the coasts, much like a ripple from a drop of water.  In other words, the farther a county was from Arkansas—ground zero for Wal-Mart—the later it experienced the Wal-Mart  growth spurt.  This was an actual pattern of expansion, one that made sense for the  company as it focused on utilizing its distribution networks most effectively and lowering  overall costs of expansion.  By following this ripple of store openings across the country  and over time, we are able to test whether retail wages follow a similar ripple pattern.   Looking at store openings based on both how far the county is from Wal-Mart’s “ground  zero” and the year in question, our estimates are not subject to the selection bias that is often a problem for similar studies. 

Results and Implications 

We find strong evidence that in urban and suburban counties (counties that are part of a  Metropolitan Statistical Area), a Wal-Mart store opening led to a 0.5% to 0.8% reduction  in average earnings per workers in the general merchandising sector.  This finding is consistent with Wal-Mart jobs paying about 10% lower wages than the jobs they  displaced.  A Wal-Mart store also reduced average earnings per grocery worker in that  county by 0.8% to 0.9%.  Taking both wage and possible employment effects into  account, we found that a single Wal-Mart store reduced the total earnings of general  merchandise and grocery workers in that county by about 1.3%. In rural counties, the pattern was different.  A Wal-Mart store opening there was associated with an increase in the average earnings per general merchandise worker and a  decrease in the average earnings per grocery worker.  However, combining wage and  employment effects, the impact on the total take-home pay of the affected retail  workforce was a wash.

Are you convinced by this evidence? I'm not. Does it lead you to conclude that large cities and suburbs should keep out Wal-Mart in order to protect workers?

At a conference, I asked one of the authors (Dube) why wages in urban areas fell more than in rural areas. Wouldn't you expect an urban area with lots of jobs to be less affected by the entry of Wal-Mart than in a rural area? Even if you believe that Wal-Mart has some monopoly power, wouldn't it be smaller in larger cities than in rural areas? His response: only if you have a neoclassical view of the world. We got interrupted before I could ask him what his view of the world was. But I suspect his view of the world is agnostic--he lets the data speak. The truth is there. I say no. The illusion of truth is there. Using distance from Bentonville is clever but does it really work? How would you know? Does it convince you that Los Angeles should keep out Wal-Mart? (I don't know what the authors actually believe on this latter question. I do know that some people have interpreted their results as evidence for keeping Wal-Mart out of an area.)

Here's how my biased neoclassical view works. One, I'm not convinced that Wal-Mart reduces wages because I'm not convinced that distance from Bentonville is a good instrument. What other instruments did you try first? (Here is Emek Basker's paper that critiques a paper that uses a similar methodology.) But even if you could convince me that it is a good instrument, my next argument is that Wal-Mart might lower wages, but that's because it draws low-wage workers into the labor force and the lower overall wages you observe are a composition effect. Those lower wage are a blessing not a curse. You won't convince me that increasing the demand for labor is going to lower the wages of workers who already are employed. Their wages can still go up when Wal-Mart arrives. And even if wages are lower in places where Wal-Mart arrives, I'm still going to argue that causation runs the other way, that Wal-Mart's entry is a response to lower wages, that you didn't hold everything else constant, your instrument is flawed. You're going to find it very hard to convince me that an increase in demand holding everything else constant lowers wages. I'm going to presume you really didn't/couldn't hold everything else constant.

So whose view is more scientific? I'm not so sure. I'm open to attack. Give me a hard time. I'm not interested in debating the merits of Dube, Eidlin, and Lester—if you think their paper is not a good example of instrumental variables, give me a better one. Give me one that's so well done it will cause me to doubt my neoclassical world view or other views I hold on policy or human behavior. Is it possible? Is there an empirical study on the other side that would convince Dube, Eidlin and Lester that Wal-Mart is good for workers? Show me an example where sophisticated statistical analysis has won over the skeptics or trumped common sense because it was so well done.

For what it's worth, I am less of a hard-core neoclassical economist than when I left Chicago. Why? I suspect that various kinds of evidence have persuaded me to take a richer viewpoint. So again, evidence and facts matter.   

Finally, a commenter at Overcoming Bias, Constant, does a much better job than I did explaining the differences between the Pragmatist and the Cartesian:

I will try to approximate the pragmatist position as it contrasts with the Cartesian position. The Cartesian approach is to discard every idea unless it can be verified. The point is to believe only things which are certainly true, and to avoid believing falsehoods. It is a maximally skeptical approach. In contrast, the pragmatist approach is to keep every idea until it is falsified (or found wanting in some other particular way). So the Cartesian gets rid of all the planks of his boat that he can't demonstrate to his satisfaction are necessary for the boat's operation, while the pragmatist keeps all the planks of his boat unless he can demonstrate to his satisfaction that a particular plank is unnecessary.  

Applying this to the current case, the pragmatist approach says that if you have a belief about guns and the evidence does not contradict the belief, then keep the belief. The Cartesian approach says that if you have a belief about guns and the evidence does not demonstrate this belief, then discard the belief.

A few more empirical thoughts

Colleague Robin Hanson gives me a hard time over at Overcoming Bias (reacting to this earlier post):

If Russ relies little on data to draw his conclusions, then on what does he rely?  Perhaps he relies on theoretical arguments.  But can't we say the same thing about theory, that we mainly just search for theory arguments to support preconceived conclusions?  If so, what is left, if we rely on neither data nor theory? 

Try saying this out loud: "Neither the data nor theory I've come across much explain why I believe this conclusion, relative to my random whim, inherited personality, and early culture and indoctrination, and I have no good reasons to think these are much correlated with truth."  That does not seem a conclusion worth retaining.  If this is really your situation, you should move to a nearly intermediate position of uncertainty.   Either you should believe that truth-correlated data or theory has substantially influenced your belief, or you should retain only a very weak belief.

In this week's EconTalk podcast I added this postscript to try and clarify what I was saying:

In last week’s podcast I spoke with Ian Ayres about the power and limitations of data analysis. Ayres emphasized the power and I kept mentioning the limitations, especially in the postcript I added after the interview. I want to clarify a few issues. My basic point was that when it comes to high-powered sophisticated statistical techniques, our biases as researchers and as consumers of that research often triumph over truth. The truth is elusive in complex systems with many things changing at once. It’s hard to isolate the independent effect of one particular variable. When scholars can run hundreds of multivariate regressions at very low cost, it easy to convince yourself that the results that confirm your prior beliefs are the “right “ results. The ones that failed must be the “bad ones.”

When I was in college at U of NC, I took a wonderful course from Richard Smyth where I learned about the American philosopher Charles Peirce and the philosophy of pragmatism. Peirce and the Pragmatists, which include William James and others, believed that the rationalism of Descartes had a dangerous element of hubris. The worship of rationality could lead to deluding oneself to the reliability of one’s thoughts. Prof. Smyth but it this way—your grandmother is right. She believes in certain things. When you ask her to justify her beliefs she shrugs and says she can’t. Some things you do because that’s just the way they’ve always been done. You feel superior to your grandmother because you only do things that are rational. If you can’t justify something via reason, you simply reject it. But your grandmother (and Hayek), were on to something. Norms of behavior that survive, survive because they’re effective even when no one understands why.

Prof. Smyth discussed the Cartesian belief that you should examine every one of your beliefs. If it passes the test of reason, keep it. Otherwise, throw it out. Seems reasonable. But the pragmatists argued that that was akin to examining the planks of your boat while you were at sea. Tearing them out because they look imperfect is the road to ruin. It’s particularly true when you’re less than objective in deciding whether to reject or accept a belief.

Smyth quoted Benjamin Franklin: When fortresses and virgins get to talking, the end is in sight. That is—when you’re besieged, once you start negotiating, it’s easy to talk yourself into giving in and finding a way to justify it as the right thing to do.

All of which is to say that we shouldn’t pretend to be scientific when we’re only doing something that has the veneer of science. That’s much more dangerous than saying, I don’t know or we can’t answer that question.

I certainly didn’t mean to imply that Ian Ayres or John Lott (whose work came up in the conversation with Ayres) are biased researchers. I also did not mean to imply that data and evidence are irrelevant in how we form our beliefs about what is true. Or that our biases never get overturned. The example of Friedman and Schwartz’s Monetary History of the United States is the gold standard. Facts can be decisive. Statistical analysis can persuade. But I am struck by how few controversial viewpoints in the field of economics have been accepted based on sophisticated statistical analysis. By sophisticated analysis, I mean for example, the use of instrumental variables with a data set that has limited information about the myriad of factors that affect the variable we care about.

Where does that leave us? Economists should do empirical work, empirical work that is insulated as much as possible from confirmation bias, empirical work that isn’t subject to the malfeasance of running thousands of regressions until the data screams Uncle. And empirical work where it’s reasonable to assume that all the relevant variables have been controlled for. And let’s not pretend we’re doing science when we’re not.

Fooling ourselves

At the University of Chicago, where I went to graduate school, a quote from Lord Kelvin, was carved in stone at the Social Sciences Research Building: When you cannot measure...your knowledge is meager and unsatisfactory. We graduate students took in this credo like mother’s milk. I came out of Chicago with a tremendous confidence in the power of economics and the ability to quantify that power.

But over the years, I have become increasingly skeptical of the power of statistical techniques to measure causation in complex systems. Edward Leamer’s indictment of modern econometrics, “Let’s take the ‘con’ out of econometrics” is the best known critique of our habits as empirical economists but it has not been taken to heart by the profession.

My thoughts on this issue came to a head with my recent podcast with Ian Ayres on his new book SuperCrunchers. The book is about the power of statistics to improve decision-making. And of course, facts and numbers are crucial for making wise decisions. And there are many examples where statistical analysis helps us in our private and political lives to overcome irrational prejudice or bad ideas. 

But in the course of preparing for the interview, I realized in a way I hadn’t before, that how we feel about the reliability of statistical results lines up incredibly neatly with our political and ideological biases. One example I use in the podcast is the debate over whether allowing citizens to carry concealed handguns deters crime. John Lott and others say yes and trot out the analysis that proves they're right. Their opponents trot out a different analysis and prove the advocates for concealed carry are wrong.

Now I happen to believe that concealed handguns do deter crime and allowing concealed handguns is a good thing. And you can claim that the evidence that shows I'm right is "good" statistical analysis. The other side disagrees. They claim it's "bad" statistical analysis. Who's right? I have no idea. But what's clear to me is that my belief in the virtues of allowing concealed hand guns has little to do with the empirical evidence. And I would argue that the opponents are really in the same boat. They just don't like guns and they've dressed up their prejudices in fancy statistical analysis.

I came to this realization because Ayres thinks LoJack deters car theft. LoJack is a hidden device you put in your car that lets the police trace your car.  Ayres and Steve Levitt found that LoJack has an incredible deterrent effect on car theft. But they think John Lott's work on concealed hand guns is irrevocably flawed. But LoJack is the same thing as a concealed handgun. Ayres and Levitt should like Lott's work and Lott should like Ayres and Levitt. But Lott doesn't like Ayres and Levitt and the feeling is mutual.

It's obvious why neither respects the other. It's not the quality of the empirical evidence. It's just bias. Ayres and Levitt don't like guns and Lott doesn't like the idea that insurance companies (and criminals) could ignore the impact of LoJack if it's really so big. Ayres and Levitt see LoJack as an example of market failure and Lott thinks market failure in this case requires too much ignorance.

The nature of the analysis is such that neither side can convince the other that "their" analysis is reliable. That's not always true. As I suggest in the podcast, Milton Friedman was able to convince the skeptics that inflation is everywhere and always a monetary phenomenon. Friedman won the debate. But how many other studies can you think of where someone staked out a controversial position and convinced the skeptics based on empirical analysis? I think it can be done, but it's rare. And in today's world, most of the interesting empirical claims are being made in cases where  the data are too incomplete and the issue is so complex that we can't move to a consensus. The empirical work doesn't improve our understanding of what's going on. It masks what's going on. It gives a patina of science when in effect the numbers aren't really informing the debate.

In the case of crime, to  isolate the effect of LoJack or concealed hand guns, you have to control for any other causes of crime and control for the simultaneity problem that causation could be running the other way. I'm not sure that can be done. My other favorite example of this is WalMart. There are economists out there who claim that WalMart lowers wages when it comes to a town, even a big town, such as Los Angeles. I don't find this argument believable. But the proponents of such arguments claim to just be using the numbers to tell them what's going on rather than relying on their prejudices as I am. But the numbers can't be crunched sufficiently well to come to a conclusion on WalMart. To do that, you have to control for a bunch of factors that can't be controlled for in real-world data situations.

And that's why there are studies on the other side showing how great WalMart is for a town. But are we moving toward a consensus about the impact of WalMart? Do the analyses improve our understanding? I don't think so. And that's because of the way modern econometrics is done. Regression is cheap so we buy a lot of it. Leamer's point is that this is "faith-based" empirical work. You just keep running the regressions including or excluding this or that, trying this or that specification until you find the result that confirms your worldview before you started the work.

The pragmatists (Peirce and James) and Hayek understood the dangers of rationality and what is essentially fake science. I'll write more on this another time and maybe do a podcast just on this issue.

I've closed comments here. If you want to comment, please listen to the Ayres podcast at EconTalk and let's talk over there in the comments section to the Ayres podcast.

Poverty and immigration

Robert Samuelson points out wisely that the measured poverty rate is a misleading measure of economic progress when there is immigration (a common theme here at the Cafe):

The standard story is that poverty is stuck; superficially, the statistics support that. The poverty rate measures the share of Americans below the official poverty line, which in 2006 was $20,614 for a four-person household. Last year, the poverty rate was 12.3 percent, down slightly from 12.6 percent in 2005 but higher than the recent low, 11.3 percent in 2000. It was also higher than the 11.8 percent average for the 1970s. So the conventional wisdom seems amply corroborated.

It isn't. Look again at the numbers. In 2006, there were 36.5 million people in poverty. That's the figure that translates into the 12.3 percent poverty rate. In 1990, the population was smaller, and there were 33.6 million people in poverty, a rate of 13.5 percent. The increase from 1990 to 2006 was 2.9 million people (36.5 million minus 33.6 million). Hispanics accounted for all of the gain.

Consider: From 1990 to 2006, the number of poor Hispanics increased 3.2 million, from 6 million to 9.2 million. Meanwhile, the number of non-Hispanic whites in poverty fell from 16.6 million (poverty rate: 8.8 percent) in 1990 to 16 million (8.2 percent) in 2006. Among blacks, there was a decline from 9.8 million in 1990 (poverty rate: 31.9 percent) to 9 million (24.3 percent) in 2006. White and black poverty has risen somewhat since 2000 but is down over longer periods.

Monkeys and Typewriters

In Fooled by Randomness, Nassim Nicholas Taleb gives the example of a person who wants to hire someone to type a copy of Homer's Odyssey. Suppose an infinite number of monkeys are typing on an infinite number of typewriters. One of them comes up with a perfect copy of the Illiad. This enterprising monkey shows up at your door, applying for the job of Odyssey typist. But you're a monkey, you reply skeptically. He takes out the perfect copy of the Illiad. You're stunned. And you apologize and hire him on the spot, foolishly interpreting the perfect manuscript of the Illiad as an indication of the monkey's skills.

Taleb's point is that before you decide that monkeys are good at typing, you want to know more about the denominator—the number of monkeys trying to type the Illiad. When people claim that this stock-picker or that fund manager is a genius because he's beaten the S&P 500 for 15 straight years, you should ask how many people are out there picking stocks and what proportion of those would be expected to beat the S&P for purely random reasons. It's possible there are fund managers who are genuinely skilled but the fact that they do well over some limited time period is not proof.

A more mundane application is asking for references from a contractor. I always ask for two or three references. But a contractor who has been in business for any length of time will be able to find a few good ones—even a few raves—that may not be representative of the typical experience of the firm's clients. The better strategy is to ask for references from the last three clients or clients from the last six months.

Here is a wonderful page on the infinite monkey metaphor. It will give you a healthy sense of what low-probability events really mean:

One computer program run by Dan Oliver of Scottsdale, Arizona, according to an article in The New Yorker, came up with a result on August 4, 2004: After the group had worked for 42,162,500,000 billion billion years, one of the "monkeys" typed, “VALENTINE. Cease toIdor:eFLP0FRjWK78aXzVOwm)-‘;8.t . . ." The first 19 letters of this sequence can be found in "The Two Gentlemen of Verona". Other teams have reproduced 18 characters from "Timon of Athens", 17 from "Troilus and Cressida", and 16 from "Richard II".^[20]

A website entitled The Monkey Shakespeare Simulator, launched on July 1, 2003, contained a Java applet that simulates a large population of monkeys typing randomly, with the stated intention of seeing how long it takes the virtual monkeys to produce a complete Shakespearean play from beginning to end. For example, it produced this partial line from Henry IV, Part 2, reporting that it took "2,737,850 million billion billion billion monkey-years" to reach 24 matching characters:

RUMOUR. Open your ears; 9r"5j5&?OWTY Z0d…

Even more entertaining is this study of actual monkeys:

In 2003, lecturers and students from the University of Plymouth MediaLab Arts course used a £2,000 grant from the Arts Council to study the literary output of real monkeys. They left a computer keyboard in the enclosure of six Sulawesi Crested Macaques in Paignton Zoo in Devon in England for a month, with a radio link to broadcast the results on a website. One researcher, Mike Phillips, defended the expenditure as being cheaper than reality TV and still "very stimulating and fascinating viewing".^[24]

Not only did the monkeys produce nothing but five pages^[25] consisting largely of the letter S, the lead male began by "bashing the hell out of" the keyboard with a stone, and the monkeys continued by urinating and defecating on it. The zoo's scientific officer remarked that the experiment had "little scientific value, except to show that the 'infinite monkey' theory is flawed". Phillips said that the artist-funded project was primarily performance art, and they had learned "an awful lot" from it. He concluded that monkeys "are not random generators. They're more complex than that. … They were quite interested in the screen, and they saw that when they typed a letter, something happened. There was a level of intention there."^[24][26]

Clutch hitting

It is very hard for sports fans, sports writers and even athletes do accurately assess which of their  players perform well under pressure and which do not.

This brilliant dissection (Rated R for language) of a recent sports column on the topic of "hitting in the clutch" shows how powerful numbers can be for clarifying fuzzy thinking.

More on Interpreting Statistics

Here's more, now from The Economist.com, on the need to be careful when interpreting statistics -- such as that infant-mortality rates in the United States are higher than in many countries much poorer than America.  A snippet:

Comparing infant mortality rates between countries is fraught with uncertainty—after all, it's hard to argue that every country's figures are reliable. But it's still worth asking what more we can do to stop babies from dying. Defined as death before one year of age, infant mortality frequently gets framed in the United States as a problem of insufficient health-care funding. In December, for example, a New York Times column blamed it on the lack of a single-payer health insurer. However, a closer look reveals the counterintuitive possibility that high infant mortality in the United States might be the unintended side effect of increased spending on medical care.

Read the whole Economist.com post.

(HT: Liberty Alone)

Using and Misusing Statistics

Statistically speaking....

Who Shoulders the Tax Burden?

The IRS just released some data.  One of the most interesting reported facts is this:

For tax year 2004.... Taxpayers with an AGI [adjusted gross income] of at least $328,049, the top 1 percent of taxpayers, accounted for 19 percent of total AGI, representing an increase in income share of 2.2 percentage points from the previous year. These taxpayers accounted for 36.9 percent of the total income tax reported, an increase from 34.3 percent in 2003 [emphasis added].

(HT: Andy Roth at the Club for Growth)

Income and Health

Alert, alert. Do not miss this. Best thing I've seen in a long time. Three people told me about in the last two days. And I'm pretty sure someone else told me about it before that and I just missed it. Don't miss it.

So go here. It is a spectacular presentation on how international measures of life expectancy and other measures of health are getting better over time and the relationship to income. It's also has some spectacular examples of how averages can be misleading. But equally compelling is the way the data are presented. This is so cool.

When you're done, more info here.

HT: Avi Hoffman, Ville (in the comments at EconTalk on the Easterbrook podcast) and Ben Parizek.

Autism and TV

Here's a very interesting article in the WSJ (HT: Steve Saletta) on the use of what is called  instrumental variables in econometrics to argue that television watching causes autism. It seems like a goofy idea and it may be. The more interesting issue for me is the use of sophisticated statistical techniques to try and observe cause and effect. Too much of the time, for my taste, instrumental variables are used to awe and amaze rather than to ferret out the truth. And too many economists have become enamored of technique for technique's sake.

The paper discussed in the Journal is a perfect example and the reporter, Mark Whitehouse does a superb job explaining the econometrics for a non-expert:

Prof. Waldman, a recognized expert in the field of applied microeconomics, doesn't pretend to be an authority on autism. He became engrossed in the subject in the fall of 2003, when his 2-year-old son, David, was identified as having an autism-spectrum disorder. Hoping to eliminate any potential triggers, Prof. Waldman supplemented the recommended therapy with a sharp reduction in television watching. His son had started watching more TV in the summer before the diagnosis, after a baby sister was born.

Prof. Waldman says his son improved within six months and today has fully recovered -- a surprising result, given that autism is typically a lifetime affliction. "When I saw the rapid progress, which was certainly not what anyone had been predicting, I became very curious as to whether television watching might have played a role in the onset of the disorder," he says. He tried to get medical researchers interested in the idea, to no avail.

In late 2004, he decided to look into the subject himself, ultimately putting together a research team with Cornell health economist Sean Nicholson and Nodir Adilov, a professor of economics at Indiana University-Purdue University in Fort Wayne.

In principle, the best way to figure out whether television triggers autism would be to do what medical researchers do: randomly select a group of susceptible babies at birth to refrain from television, then compare their autism rate to a similar control group that watched normal amounts of TV. If the abstaining group proved less likely to develop autism, that would point to TV as a culprit.

Economists usually have neither the money nor the access to children needed to perform that kind of experiment. More broadly, randomized trials seldom lend themselves to studying economic questions, particularly the more traditional ones. It would be unfair to randomly subject some people to a higher tax rate just to see how it affects their spending.

Instead, economists look for instruments -- natural forces or government policies that do the random selection for them. First developed in the 1920s, the technique helps them separate cause and effect. Establishing whether A causes B can be difficult, because often it could go either way. If television watching were shown to be unusually prevalent among autistic children, it could mean either that television makes them autistic or that something about being autistic makes them more interested in TV.

The ideal instrument is a variable that is correlated with A but has no direct effect of its own on B. It should also have no connection to other factors that might cause B. If data in a study nonetheless show that the instrumental variable is linked to B, it suggests that A must be contributing to B.

After giving a few examples of previous studies that used instrumental variables, the article gets back to Waldman and autism:

By putting together weather data and government time-use studies, they found that children tended to spend more time in front of the television when it rained or snowed. Precipitation became the group's instrumental variable, because it randomly selected some children to watch more TV than others.

The researchers looked at detailed precipitation and autism data from Washington, Oregon and California -- states where rain and snowfall tend to vary a lot. They found that children who grew up during periods of unusually high precipitation proved more likely to be diagnosed with autism. A second instrument for TV-watching, the percentage of households that subscribe to cable, produced a similar result. Prof. Waldman's group concluded that TV-watching could be a cause of autism.

Criticism quickly arose, illustrating some of the perils of the economists' approach. For one, instruments are often too blunt. As Prof. Waldman concedes, precipitation could be linked to a lot of factors other than TV-watching -- such as household mold -- that could be imagined to trigger autism. At best, his data reflect the effect of television on those children who changed their habits because of rain or snow, not on those who did it for other reasons such as a desire to watch educational shows.

On the surface, the whole thing sounds vaguely plausible. Watching TV is a brain activity. It's imaginable that watching TV affects your brain and could, at a crucial developmental stage, cause the brain to do various things, both good and bad.

But then you start to think about it a little more. They didn't show a relationship between TV and autism. They showed a relationship between rainfall and autism. Then they made the leap that more rainfall means more TV. But more rainfall means a thousand other things, too. The "theory" that encouraged the study was based on ONE observation, that Waldman's kid watched less TV and got better. But surely there were thousands of things that were different in the Waldman household over those six months.

Then there is the problem of defining autism. There are different degrees of autism. Were they all lumped together? How precise was the relationship between rainfall and autism? What was the confidence interval? Did the definition of autism change over the period? The diagnosis of autism is much more common today than in the past, either because of environmental factors or more plausibly, a higher awareness of the phenomenon by both doctors and patients and a widening of the definition of what is considered autistic. Maybe an increase in rainfall just happened to coincide with an expansion of the definition. Surely the increase in cable TV subscriptions is correlated with that expansion as well. And is there really so much variation in snowfall and rainfall in Oregon, California and Washington states? Why were those states chosen? If anything, people in Portland and Seattle are used to lots of rain and snowfall and probably don't respond by watching TV a lot when it's raining. Did the authors control for these possible effects?

When I hear these kind of speculative findings (and they are very common in the profession these days) I always want to ask the researchers how confident they really are in their findings and have they really discovered anything we didn't already know.

In the case of the autism study, I'd ask the following: does the econometric firepower brought to bear on the problem add anything more to our knowledge than the anecdotal story that Michael Waldman's son got better after watching less TV? I'm not convinced. But here's what I really want to say to the authors of the study.

You found a correlation between rainfall and autism that might indicate a correlation between TV and autism. Let's take a thousand kids with autism who currently watch a lot of TV. Let's do a real experiment where we remove their access to television. How much are you willing to bet that there will be a statistically significant reduction in autism? A thousand dollars? Ten thousand dollars? Ten dollars? Put your money where your econometric mouth is. You found something. Do you really believe it?