by Alan
Sportswriter Allen Barra has carved out a niche for himself as a contrarian, as evidenced by the title of his 1995 book, That’s not the way it was: (Almost) everything they told you about sports is wrong. Seemingly whenever he gets the chance, Barra likes to take on conventional wisdom in sports.
Viewers of the Super Bowl football game this upcoming Sunday will likely hear the announcers cite statistics purporting to show that some early development in the game may presage a victory by one team over the other. These are the kinds of pronouncements Barra loves to challenge.
One of the lines of thinking he attacked in his book was that, “You need a strong running game to win in pro football.” As noted by Barra, fans often hear a claim of the form: When running back A runs for 100 yards in a game, his team wins some high percent of the time (see this example involving the running back Ahman Green). Yet, statistical analyses by Barra and colleagues did not appear to show that the better teams rushed the ball better than did the poorer teams. Eventually, Barra reached the following conclusion:
What we finally came to discover was that football people were confusing cause with effect in regard to running the ball…Stated as simply as possible, the good teams weren’t winning so much because Tony Dorsett (or Walter Payton or Roger Craig or whoever) was rushing for 100-plus yards – the runners were getting their 100-plus yards because the teams were winning. Teams with a sizable lead in the second half have the luxury of running far more plays on the ground than their opponents; this not only allows them to avoid sacks and interceptions that could help their opponents get back into the game, it allows them to eat up the clock by keeping the ball on the ground (pp. 173-174).
Among the statistical findings presented by Barra were that, “Most playoff teams led in most of their regular season games by halftime,” and that, “Most playoff teams get as much as two-thirds of their rushing yards in the second half when they already have a lead…” (p. 174).
Several years after the publication of Barra’s book, in 2003, I attended an informal gathering of academics and sportswriters in Scottsdale, Arizona to discuss the application of statistics and research methodology to sports decision-making. Another possible case of football spuriosity that came up was the likely correlation between throwing interceptions and losing games. Were such a correlation to be confirmed, many observers would probably interpret it as the throwing of interceptions causing a team to lose (i.e., by giving the opponent good field position and/or killing one’s own drives). Following the same logic as in the above example, it could be that a team falls behind for reasons having nothing to do with interceptions, but once behind, throws a lot of risky passes in an attempt to catch up, which get… intercepted!
Thursday, January 31, 2008
Sunday, January 20, 2008
Cause and Effect in Epidemiology Research
by Alan
One of the more interesting books I’ve read recently has the unusual title The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, by David Salsburg (2001). I thank Lindsay Reed, the statistically trained computer lab director in my academic unit at Texas Tech University, for showing me the book and letting me borrow it.
The book traces roughly 150 years of history of how prominent statisticians developed concepts to help solve practical problems. Of greatest relevance to this blog is Chapter 18, “Does Smoking Cause Cancer?” Given the inability to conduct true, random-assignment experiments on human subjects, epidemiological researchers were left in the 1950s and ’60s with a variety of case-control (retrospective) studies, prospective studies, and studies of animals and tissue cultures.
Arguably the central figure in the book is Sir Ronald A. Fisher (1890-1962). A prolific contributor to methodology and statistics (e.g., experimental design, analysis of variance and covariance, degrees of freedom, time-series analyses, sample-to-population inference, maximum likelihood), Fisher is portrayed as a fervent skeptic of a causal connection between tobacco and lung cancer.
As part of his historical review, Salsburg writes:
Fisher was dealing with a deep philosophical problem – a problem that the English philosopher Bertrand Russell had addressed in the early 1930s, a problem that gnaws at the heart of scientific thought, a problem that most people do not even recognize as a problem: What is meant by “cause and effect”? Answers to that question are far from simple (p. 183).
Salsburg then reviews proposals (and their failures) for conceptualizing cause and effect, including the use of symbolic logic, and “material implication” (suggested by Russell and elaborated by Robert Koch).
Finally, Salsburg appears to conclude, the most satisfactory approach is that of Jerome Cornfield and others, as exemplified by their 1959 review article entitled “Smoking and lung cancer: recent evidence and a discussion of some questions.” I don’t believe the word “triangulation” is ever used by Salsburg, but that is what he is crediting Cornfield and colleagues with:
Each study is flawed in some way. For each study, a critic can dream up possibilities that might lead to bias in the conclusions. Cornfield and his coauthors assembled thirty epidemiological studies run before 1958… As they point out, it is the overwhelming consistency across these many studies, studies of all kinds, that lends credence to the final conclusion. One by one, they discuss each of the objections. They consider [Mayo Clinic statistician Joseph] Berkson’s objections and show how one study or another can be used to address them… (p. 190).
Cornfield and colleagues did this for other critics’ objections, as well.
Another research milestone Salsburg cited, which looks interesting to me, is the development of a set of criteria for matching in case-control studies, attributed to Alvan Feinstein and Ralph Horwitz. These authors’ contributions are cited in this 1999 review of epidemiologic methods by Victor J. Schoenbach.
One of the more interesting books I’ve read recently has the unusual title The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, by David Salsburg (2001). I thank Lindsay Reed, the statistically trained computer lab director in my academic unit at Texas Tech University, for showing me the book and letting me borrow it.
The book traces roughly 150 years of history of how prominent statisticians developed concepts to help solve practical problems. Of greatest relevance to this blog is Chapter 18, “Does Smoking Cause Cancer?” Given the inability to conduct true, random-assignment experiments on human subjects, epidemiological researchers were left in the 1950s and ’60s with a variety of case-control (retrospective) studies, prospective studies, and studies of animals and tissue cultures.
Arguably the central figure in the book is Sir Ronald A. Fisher (1890-1962). A prolific contributor to methodology and statistics (e.g., experimental design, analysis of variance and covariance, degrees of freedom, time-series analyses, sample-to-population inference, maximum likelihood), Fisher is portrayed as a fervent skeptic of a causal connection between tobacco and lung cancer.
As part of his historical review, Salsburg writes:
Fisher was dealing with a deep philosophical problem – a problem that the English philosopher Bertrand Russell had addressed in the early 1930s, a problem that gnaws at the heart of scientific thought, a problem that most people do not even recognize as a problem: What is meant by “cause and effect”? Answers to that question are far from simple (p. 183).
Salsburg then reviews proposals (and their failures) for conceptualizing cause and effect, including the use of symbolic logic, and “material implication” (suggested by Russell and elaborated by Robert Koch).
Finally, Salsburg appears to conclude, the most satisfactory approach is that of Jerome Cornfield and others, as exemplified by their 1959 review article entitled “Smoking and lung cancer: recent evidence and a discussion of some questions.” I don’t believe the word “triangulation” is ever used by Salsburg, but that is what he is crediting Cornfield and colleagues with:
Each study is flawed in some way. For each study, a critic can dream up possibilities that might lead to bias in the conclusions. Cornfield and his coauthors assembled thirty epidemiological studies run before 1958… As they point out, it is the overwhelming consistency across these many studies, studies of all kinds, that lends credence to the final conclusion. One by one, they discuss each of the objections. They consider [Mayo Clinic statistician Joseph] Berkson’s objections and show how one study or another can be used to address them… (p. 190).
Cornfield and colleagues did this for other critics’ objections, as well.
Another research milestone Salsburg cited, which looks interesting to me, is the development of a set of criteria for matching in case-control studies, attributed to Alvan Feinstein and Ralph Horwitz. These authors’ contributions are cited in this 1999 review of epidemiologic methods by Victor J. Schoenbach.
Tuesday, January 15, 2008
Welcoming Statement
by Alan and Bo
Welcome to our new blog on correlation and causality. For four academic years (2003-04 to 2006-07 inclusive) we both taught research methods in the Department of Human Development and Family Studies (HDFS) at Texas Tech University. Bo has now moved to Penn State. Alan, who arrived at Texas Tech for the 1997-98 year, remains there.
Each of us has used a “validities” framework in teaching our methods courses (cf. Campbell & Stanley, 1963; Cook & Campbell, 1979), where internal validity deals with causal inference, external validity with generalizability, and construct validity with whether a test measures what it purports to measure (Linda Woolf has a page that concisely summarizes the different types of validity).
Bo one time said that, with reference to academics at least, internal validity is what gets him up in the morning and motivates him to come in to work. Alan’s passion is perhaps more evenly split between external validity (on which he operates another website) and internal validity.
On this blog, we seek to raise and discuss various issues pertaining to correlation and causality, much like we did during our frequent conversations at Texas Tech. In fields that study human behavior in “real world” settings, many potentially interesting phenomena are off-limits to the traditional experimental design that would permit causal inferences, for practical and ethical reasons.
Does the birth of a child increase or decrease couples’ marital/relationship satisfaction? Does growing up with an alcohol-abusing parent damage children’s development of social skills? How does experiencing a natural disaster affect residents’ mental and physical health?
For none of these questions could researchers legitimately assign individuals (or couples) at random to either receive or not receive the presumed causal stimulus. Much of our discussion, therefore, will be aimed at formulating ideas for how to make as strong a causal inference as possible, for a given research question.
By raising issues of how researchers might approach a given research question from the standpoint of internal validity, we hope to fulfill a “seeding” process, where our initial commentaries will be generative of further discussion and suggestions. We are thus permitting (and encouraging!) comments on this blog, for this purpose. We hope to learn as much (or more) from you, as you might learn from us.
In addition, we’ll write about stories in the news media that raise causal questions and review scholarly articles and books that do the same.
We recognize that issues of causality are implicated in a wide variety of academic disciplines. At the outset at least, we will probably stick closely to fields such as psychology, sociology, and HDFS. Later on, we hope to expand into other domains such as philosophy and legal studies (within the law, many states have homicide or wrongful death statutes with wordings that allude to situations in which someone “causes the death” of another).
We invite you to visit this blog often and chime in with comments when the feeling strikes. Requests from readers to write lead essays as guest contributors will be considered (or we may even invite some of you to do so).
Thanks for stopping by!
Welcome to our new blog on correlation and causality. For four academic years (2003-04 to 2006-07 inclusive) we both taught research methods in the Department of Human Development and Family Studies (HDFS) at Texas Tech University. Bo has now moved to Penn State. Alan, who arrived at Texas Tech for the 1997-98 year, remains there.
Each of us has used a “validities” framework in teaching our methods courses (cf. Campbell & Stanley, 1963; Cook & Campbell, 1979), where internal validity deals with causal inference, external validity with generalizability, and construct validity with whether a test measures what it purports to measure (Linda Woolf has a page that concisely summarizes the different types of validity).
Bo one time said that, with reference to academics at least, internal validity is what gets him up in the morning and motivates him to come in to work. Alan’s passion is perhaps more evenly split between external validity (on which he operates another website) and internal validity.
On this blog, we seek to raise and discuss various issues pertaining to correlation and causality, much like we did during our frequent conversations at Texas Tech. In fields that study human behavior in “real world” settings, many potentially interesting phenomena are off-limits to the traditional experimental design that would permit causal inferences, for practical and ethical reasons.
Does the birth of a child increase or decrease couples’ marital/relationship satisfaction? Does growing up with an alcohol-abusing parent damage children’s development of social skills? How does experiencing a natural disaster affect residents’ mental and physical health?
For none of these questions could researchers legitimately assign individuals (or couples) at random to either receive or not receive the presumed causal stimulus. Much of our discussion, therefore, will be aimed at formulating ideas for how to make as strong a causal inference as possible, for a given research question.
By raising issues of how researchers might approach a given research question from the standpoint of internal validity, we hope to fulfill a “seeding” process, where our initial commentaries will be generative of further discussion and suggestions. We are thus permitting (and encouraging!) comments on this blog, for this purpose. We hope to learn as much (or more) from you, as you might learn from us.
In addition, we’ll write about stories in the news media that raise causal questions and review scholarly articles and books that do the same.
We recognize that issues of causality are implicated in a wide variety of academic disciplines. At the outset at least, we will probably stick closely to fields such as psychology, sociology, and HDFS. Later on, we hope to expand into other domains such as philosophy and legal studies (within the law, many states have homicide or wrongful death statutes with wordings that allude to situations in which someone “causes the death” of another).
We invite you to visit this blog often and chime in with comments when the feeling strikes. Requests from readers to write lead essays as guest contributors will be considered (or we may even invite some of you to do so).
Thanks for stopping by!
Subscribe to:
Posts (Atom)