The following is a message from Judea Pearl to the SEMNET (Structural Equation Modeling) discussion list (conveyed by Dr. Pearl's colleague Stephen Sivo). Given Dr. Pearl's apparent intent to have these announcements publicized to a wide audience, I (Alan) have reprinted it below (lightly edited for apparent typos and ease of accessing links)...
Dear Colleagues in Causality,
Below, a few items that I thought would be of interest to researchers active in causal reasoning.
1. A new article, authored by Ilya Shpitser and myself is now posted on the UCLA Causality-Blog (see also here). It offers a solution to the problem of evaluating "Effects of Treatment on the Treated (ETT)." The problem is of theoretical interest because ETT, despite its blatant counterfactual character (e.g., "I just took an aspirin, perhaps I shouldn't have?"), can be evaluated from experimental studies in many, though not all, cases. Characterizing those cases illuminates therefore the empirical content of counterfactuals.
2. Many of you have commented on my article "Myth, Confusion and Science in Causal Analysis" (inspired by Don Rubin), a revised version of which is now posted on our website here . I would like to encourage a blog-discussion on the main points raised there. For example:
2.1. Whether graphical methods are in some way "less principled" than other methods of analysis.
2.2. Whether confounding bias can only decrease by conditioning on a new covariate.
2.3 Whether the M-bias, when it occurs, is merely a mathematical curiosity, unworthy of researchers' attention.
2.4. Whether Bayesianism instructs us to condition on all available measurements.
If you feel strongly about defending any of these claims (which seem to be
still simmering in certain circles, see video), the Causality-Blog can be an effective arena for airing them in an open discussion. Requests for anonymity will be honored.
3. Forbes magazine ran an issue on artificial intelligence last week, to which I contributed a popular article on progress in causal analysis (from my humble perspective, of course). Comments are welcome.
4. I have volunteered to give a tutorial at the JSM meeting (Washington, DC, August 5, 2009, 2-4 pm) on "Causal Analysis in Statistics: A Gentle Introduction." If any of your colleagues or students could benefit from such tutorial, I promise to be truly gentle.
5. Just before the tutorial, at 12 noon, there will be a book-signing gathering at the Cambridge University Press booth, where I will be signing copies of the 2nd Edition of Causality (and will engage in gossip and debates about where causality is heading).
Best wishes, and May clarity shine over causality land,
Judea Pearl
Note that all of Dr. Pearl's requests for blog discussion pertain to his blog at UCLA. I have thus turned off the comments option here.
Thursday, July 9, 2009
Monday, June 15, 2009
Moderate Alcohol Consumption and Heart Health
by Alan
Today's New York Times has an article on the link between moderate alcohol consumption and lower heart disease, and whether health officials should actively recommend a daily drink or two for the public. At a correlational level, the alcohol-heart health relationship appears well established. As the piece in the Times delves into, the lack of true experimentation (in this case via Randomized Clinical Trials) gives skeptics something to hang their hats on. The following are some key excerpts of the article:
For some scientists, the question will not go away. No study, these critics say, has ever proved a causal relationship between moderate drinking and lower risk of death — only that the two often go together. It may be that moderate drinking is just something healthy people tend to do, not something that makes people healthy.
“The moderate drinkers tend to do everything right — they exercise, they don’t smoke, they eat right and they drink moderately,” said Kaye Middleton Fillmore, a retired sociologist from the University of California, San Francisco, who has criticized the research. “It’s very hard to disentangle all of that, and that’s a real problem.” ...
“The bottom line is there has not been a single study done on moderate alcohol consumption and mortality outcomes that is a ‘gold standard’ kind of study — the kind of randomized controlled clinical trial that we would be required to have in order to approve a new pharmaceutical agent in this country,” said Dr. Tim Naimi, an epidemiologist with the Centers for Disease Control and Prevention.
The article goes on to discuss how clinical trials might be designed, but also their potential ethical and logistical difficulties.
Today's New York Times has an article on the link between moderate alcohol consumption and lower heart disease, and whether health officials should actively recommend a daily drink or two for the public. At a correlational level, the alcohol-heart health relationship appears well established. As the piece in the Times delves into, the lack of true experimentation (in this case via Randomized Clinical Trials) gives skeptics something to hang their hats on. The following are some key excerpts of the article:
For some scientists, the question will not go away. No study, these critics say, has ever proved a causal relationship between moderate drinking and lower risk of death — only that the two often go together. It may be that moderate drinking is just something healthy people tend to do, not something that makes people healthy.
“The moderate drinkers tend to do everything right — they exercise, they don’t smoke, they eat right and they drink moderately,” said Kaye Middleton Fillmore, a retired sociologist from the University of California, San Francisco, who has criticized the research. “It’s very hard to disentangle all of that, and that’s a real problem.” ...
“The bottom line is there has not been a single study done on moderate alcohol consumption and mortality outcomes that is a ‘gold standard’ kind of study — the kind of randomized controlled clinical trial that we would be required to have in order to approve a new pharmaceutical agent in this country,” said Dr. Tim Naimi, an epidemiologist with the Centers for Disease Control and Prevention.
The article goes on to discuss how clinical trials might be designed, but also their potential ethical and logistical difficulties.
Tuesday, March 10, 2009
New Book: The Numbers Game
by Alan
Journalist Michael Blastland and economist Andrew Dilnot host a radio show on Great Britain's BBC4 called More or Less, which attempts to engage the public on issues of data quality, statistics, and conclusion-drawing with regard to numbers reported by the media, politicians, etc. Blastland and Dilnot have also put their ideas into book form, with the recent U.S. release of The Numbers Game: The Commonsense Guide to Understanding Numbers in the News, in Politics, and in Life (previously published in the UK as The Tiger That Isn't).
I've just read The Numbers Game and, for purposes of the present blog, Chapter 12 on "Causation" is most relevant. The book's chapters are generally around 10-20 pages each, with the Causation chapter toward the shorter end. Much of this chapter presents standard material, such as a set of correlational scenarios that might tempt a reader to draw causal conclusions but ultimately turn out to be more plausibly explained by third variables. Beyond this, however, I feel the authors provide some useful insights:
What seems often to determine how easily we spot causation/correlation errors is how fast a better explanation comes to mind: thinking of decent alternatives slows conclusions and sows skepticism (p. 186).
AND
Restlessness for the true cause is a constructive habit, an insurance against gullibility. And though correlation does not prove causation, it is often a good hint, but a hint to start asking questions, not to settle for easy answers (191-192)
[As somewhat of an aside, the first of the two above statements bears some similarity in my mind to the late causal-attribution theorist Hal Kelley's concept of discounting.]
I would urge teachers of undergraduate research methods to consider using The Numbers Game (or specific chapters therein) to supplement their main textbooks. The writing is lively and the examples should help students grasp key concepts.
Journalist Michael Blastland and economist Andrew Dilnot host a radio show on Great Britain's BBC4 called More or Less, which attempts to engage the public on issues of data quality, statistics, and conclusion-drawing with regard to numbers reported by the media, politicians, etc. Blastland and Dilnot have also put their ideas into book form, with the recent U.S. release of The Numbers Game: The Commonsense Guide to Understanding Numbers in the News, in Politics, and in Life (previously published in the UK as The Tiger That Isn't).
I've just read The Numbers Game and, for purposes of the present blog, Chapter 12 on "Causation" is most relevant. The book's chapters are generally around 10-20 pages each, with the Causation chapter toward the shorter end. Much of this chapter presents standard material, such as a set of correlational scenarios that might tempt a reader to draw causal conclusions but ultimately turn out to be more plausibly explained by third variables. Beyond this, however, I feel the authors provide some useful insights:
What seems often to determine how easily we spot causation/correlation errors is how fast a better explanation comes to mind: thinking of decent alternatives slows conclusions and sows skepticism (p. 186).
AND
Restlessness for the true cause is a constructive habit, an insurance against gullibility. And though correlation does not prove causation, it is often a good hint, but a hint to start asking questions, not to settle for easy answers (191-192)
[As somewhat of an aside, the first of the two above statements bears some similarity in my mind to the late causal-attribution theorist Hal Kelley's concept of discounting.]
I would urge teachers of undergraduate research methods to consider using The Numbers Game (or specific chapters therein) to supplement their main textbooks. The writing is lively and the examples should help students grasp key concepts.
Sunday, December 28, 2008
Psychological Methods Article: Schafer & Kang
by Alan
The December 2008 issue of Psychological Methods includes an article by Joseph Schafer and Joseph Kang, entitled, "Average Causal Effects From Nonrandomized Studies: A Practical Guide and Simulated Example" (abstract).
The article addresses the seemingly age-old issue of how best to approximate a causal inference when participants' levels of the purported "causal" variable have not been assigned at random. Although the article contains a fair amount of jargon and technical formulas, the key foundation appears to boil down to the following quote:
In a typical observational study... it is unlikely that [treatment condition] will be independent of [individuals' outcome scores]. The treatments may have been selected by the individuals themselves, for reasons that are possibly related to the outcomes. With observational data, a good estimate of the [average causal effect] will make use of the covariates... to help account for this dependence (p. 281).
Via a large simulation study, Schafer and Kang compare nine different approaches for covariate-based adjustment, including analysis of covariance (ANCOVA), regression, matching, propensity scores, and weighting schemes. Near the end of the article, the authors present a section entitled "Lessons Learned," containing practical recommendations.
As social-science research articles continue to use increasingly sophisticated statistical and analytical methods, Schafer and Kang's article should be a useful resource for researchers looking to remain current with state-of-the-art approaches.
The December 2008 issue of Psychological Methods includes an article by Joseph Schafer and Joseph Kang, entitled, "Average Causal Effects From Nonrandomized Studies: A Practical Guide and Simulated Example" (abstract).
The article addresses the seemingly age-old issue of how best to approximate a causal inference when participants' levels of the purported "causal" variable have not been assigned at random. Although the article contains a fair amount of jargon and technical formulas, the key foundation appears to boil down to the following quote:
In a typical observational study... it is unlikely that [treatment condition] will be independent of [individuals' outcome scores]. The treatments may have been selected by the individuals themselves, for reasons that are possibly related to the outcomes. With observational data, a good estimate of the [average causal effect] will make use of the covariates... to help account for this dependence (p. 281).
Via a large simulation study, Schafer and Kang compare nine different approaches for covariate-based adjustment, including analysis of covariance (ANCOVA), regression, matching, propensity scores, and weighting schemes. Near the end of the article, the authors present a section entitled "Lessons Learned," containing practical recommendations.
As social-science research articles continue to use increasingly sophisticated statistical and analytical methods, Schafer and Kang's article should be a useful resource for researchers looking to remain current with state-of-the-art approaches.
Sunday, December 21, 2008
Do Vitamin Supplements Prevent Disease?
by Alan
Today's Los Angeles Times has an article on the failure of several recent randomized clinical trials to show benefits of taking vitamin supplements for disease prevention. The article provides some good explanation for the layperson on the differences between experimental (i.e., RCT) and observational (i.e., correlational) studies for being able to make causal inferences, in my view. It also discusses some problems, specific to vitamin research, that complicate the interpretation of experimental/RCT studies, even though they're designed to hold extraneous factors constant.
First, here are some examples of the article's discussion of experimental vs. observational research:
Randomized clinical trials are designed to test one factor at a time, but vitamins and minerals consumed as part of a healthy diet work in concert with each other.
"You don't eat a food that just has beta carotene in it," said Dr. Mary L. Hardy, medical director of the Simms/Mann UCLA Center for Integrative Oncology...
Also:
"The observational studies that originally linked vitamins to better health may have been biased because people who take supplements are often healthier overall than people who don't.
"They tend to be more physically active, better educated, eat better diets, and tend not to be smokers," [the NIH's Paul] Coates said. "So you can't say for certain it's your item of interest that causes" the health benefit.
The issue that's specific to vitamin studies is as follows, focusing on the need for a "placebo" or "control" group that does not receive the treatment being tested:
For several reasons, researchers say, vitamins don't lend themselves to randomized controlled trials. Chief among them is that there is no true placebo group when it comes to vitamins and minerals, because everyone gets some in their diet."
For drugs, someone either has [the anti-impotence drug] Cialis in their system or he doesn't have Cialis," said Paul Coates, director of the NIH Office of Dietary Supplements in Bethesda, Md. But with vitamins, "there's a baseline exposure that needs to be taken into account. It makes the challenge of seeing an improvement more difficult."
My (Alan's) initial reaction to this statement was that extraneous exposure to vitamins from food consumption should not necessarily compromise researchers' ability to make a causal inference about the vitamin supplements. The experimental group would be ingesting vitamin pills plus vitamins in food, whereas the control group would be ingesting placebo (sugar) pills plus vitamins in food. Assuming the food intake to be similar in the experimental and control groups -- which is what random assignment to groups is supposed to buy us -- then the only difference between the groups should be the vitamin vs. placebo pills.
However, there's yet another complication:
A vitamin's benefit may become apparent only if people aren't getting enough of it. That could explain why vitamin D has been linked to reduced rates of heart disease, cancer and diabetes.
"Most people are vitamin-D-deficient, and that's not true for vitamin E," [the USDA's Jeffrey] Blumberg said.
Today's Los Angeles Times has an article on the failure of several recent randomized clinical trials to show benefits of taking vitamin supplements for disease prevention. The article provides some good explanation for the layperson on the differences between experimental (i.e., RCT) and observational (i.e., correlational) studies for being able to make causal inferences, in my view. It also discusses some problems, specific to vitamin research, that complicate the interpretation of experimental/RCT studies, even though they're designed to hold extraneous factors constant.
First, here are some examples of the article's discussion of experimental vs. observational research:
Randomized clinical trials are designed to test one factor at a time, but vitamins and minerals consumed as part of a healthy diet work in concert with each other.
"You don't eat a food that just has beta carotene in it," said Dr. Mary L. Hardy, medical director of the Simms/Mann UCLA Center for Integrative Oncology...
Also:
"The observational studies that originally linked vitamins to better health may have been biased because people who take supplements are often healthier overall than people who don't.
"They tend to be more physically active, better educated, eat better diets, and tend not to be smokers," [the NIH's Paul] Coates said. "So you can't say for certain it's your item of interest that causes" the health benefit.
The issue that's specific to vitamin studies is as follows, focusing on the need for a "placebo" or "control" group that does not receive the treatment being tested:
For several reasons, researchers say, vitamins don't lend themselves to randomized controlled trials. Chief among them is that there is no true placebo group when it comes to vitamins and minerals, because everyone gets some in their diet."
For drugs, someone either has [the anti-impotence drug] Cialis in their system or he doesn't have Cialis," said Paul Coates, director of the NIH Office of Dietary Supplements in Bethesda, Md. But with vitamins, "there's a baseline exposure that needs to be taken into account. It makes the challenge of seeing an improvement more difficult."
My (Alan's) initial reaction to this statement was that extraneous exposure to vitamins from food consumption should not necessarily compromise researchers' ability to make a causal inference about the vitamin supplements. The experimental group would be ingesting vitamin pills plus vitamins in food, whereas the control group would be ingesting placebo (sugar) pills plus vitamins in food. Assuming the food intake to be similar in the experimental and control groups -- which is what random assignment to groups is supposed to buy us -- then the only difference between the groups should be the vitamin vs. placebo pills.
However, there's yet another complication:
A vitamin's benefit may become apparent only if people aren't getting enough of it. That could explain why vitamin D has been linked to reduced rates of heart disease, cancer and diabetes.
"Most people are vitamin-D-deficient, and that's not true for vitamin E," [the USDA's Jeffrey] Blumberg said.
Wednesday, December 3, 2008
Call for Papers -- Edited Volume
The following Call for Papers appeared on the SEMNET (Structural Equation Modeling) discussion list:
CAUSALITY IN THE SCIENCES
A volume of papers on causality across the sciences edited by Phyllis McKayIllari, Federica Russo and Jon Williamson under contract with Oxford University Press.
http://www.kent.ac.uk/secl/philosophy/jw/2009/cits/
This book will contain original research papers that deal with causality and causal inference in the various sciences and with general questions concerning the relationship between causality, probability and mechanisms. Some chapters will be invited contributions; others will be submitted to a call for papers. All papers will be subject to a reviewing process.
TIMETABLE
1st July 2009: deadline for submission of full papers for publication to be emailed to Phyllis McKay Illari (P.McKay@kent.ac.uk) or Federica Russo (F.Russo@kent.ac.uk)
1st November 2009: notification of acceptance of papers for publication.
1st December 2009: deadline for final version of papers accepted for publication.
THE VOLUME
The volume will run to about 600 pages and will be subdivided into the following parts:
Introduction
Health Sciences
Social Sciences
Natural Sciences
Psychology and Neurosciences
Computer science and statistics
Causality, probability and mechanisms
ORGANISATION
This volume is organised by the Centre for Reasoning at the University of Kent. It is associated with the Causality in the Sciences series of conferences, and with the research project Mechanisms and Causality funded by the Leverhulme Trust.
CAUSALITY IN THE SCIENCES
A volume of papers on causality across the sciences edited by Phyllis McKayIllari, Federica Russo and Jon Williamson under contract with Oxford University Press.
http://www.kent.ac.uk/secl/philosophy/jw/2009/cits/
This book will contain original research papers that deal with causality and causal inference in the various sciences and with general questions concerning the relationship between causality, probability and mechanisms. Some chapters will be invited contributions; others will be submitted to a call for papers. All papers will be subject to a reviewing process.
TIMETABLE
1st July 2009: deadline for submission of full papers for publication to be emailed to Phyllis McKay Illari (P.McKay@kent.ac.uk) or Federica Russo (F.Russo@kent.ac.uk)
1st November 2009: notification of acceptance of papers for publication.
1st December 2009: deadline for final version of papers accepted for publication.
THE VOLUME
The volume will run to about 600 pages and will be subdivided into the following parts:
Introduction
Health Sciences
Social Sciences
Natural Sciences
Psychology and Neurosciences
Computer science and statistics
Causality, probability and mechanisms
ORGANISATION
This volume is organised by the Centre for Reasoning at the University of Kent. It is associated with the Causality in the Sciences series of conferences, and with the research project Mechanisms and Causality funded by the Leverhulme Trust.
Tuesday, November 25, 2008
Propensity Scores
A collaborative effort of the individuals named below, summarized by Alan...
Jackie Wiersma, who successfully defended her Ph.D. research at Texas Tech University a few months ago, just had her officially approved dissertation posted in the university's online repository (click here to read the dissertation). Alan and Bo each served on Jackie's committee (along with Chairperson Judith Fischer and Kitty Harris), Bo via speakerphone given his move from Texas Tech to Penn State a couple years ago.
Jackie used longitudinal data, which can be of some help in strengthening one's argument for particular directions of causality (see here and here). To strengthen some of Jackie's arguments further, however, her committee recommended the use of a technique known as propensity scores (Bo, in particular, played a key role by finding online resources such as this one on the technique).
Jackie's study of adolescent and young-adult drinking involved a number of hypotheses. For simplicitly of presentation, the remainder of this entry focuses on one that predicted an individual's level of drinking as an adolescent would be associated with his or her romantic partner's drinking when the focal individual was a young adult. In other words, would being a drinker as an adolescent propel someone to select a relatively heavy drinker for a romantic partner in the future?
The key predictor variable -- in this case, adolescents' own drinking status -- is discussed in the dissertation analogously to being "assigned" to a "treatment" condition, even though such drinking status is measured as it occurs naturalistically (known in epidemiology as an "observational" variable). The main ideas involving the propensity scores are discussed in the following excerpts from Jackie's dissertation:
For the selection hypothesis, the first prediction was that assignment to group (adolescent drinker and nondrinker) would be related to partner drinking in young adulthood. Thus, it is important to take into account the possible covariates of the assignment to adolescent drinking (p. 56)...
Within the proposed hypotheses, the groups (drinkers and nondrinkers) should show differences in the differentiating variables, thus, this study examined relevant background information (e.g., parental drinking, sensation seeking, peer drinking, college enrollment) that might plausibly affect the group with which individuals are identified. A propensity score is a measure from 0 to 1 of the likelihood of being in one of two designated groups. In this study, a score of 0 means a high probability of a person being a drinker and a 1 score means a person is a nondrinker. These scores were created in SAS using logistic regressions to predict "drinker" versus "nondrinker" (p. 57).
There are many approaches that are used for propensity score matching to adjust for group differences. For this study, the stratifying propensity scores approach was used... After this step, implementing regression models, one can compare the drinker group to the nondrinker group without worrying about the impact of any baseline differences on selection into the groups (Lowe, 2003) (p. 57).
One limitation of the way propensity scores were implemented here was that, in trying to create drinker and non-drinker groups that did not differ on the other covariates, a substantial loss of cases occurred. The following is a hypothetical example (which may have approximated what actually happened). Finding a large number of adolescent drinkers who had high values on traditional predictors of adolescent drinking (e.g., peer drinking) is easy; finding non-drinkers with similarly high levels on the traditional predictors is not. Thus, the number of participants in the former group would have to be shaved down to match the cell size for the latter group. For this reason, the propensity-score analyses were de-emphasized in Jackie's dissertation, in favor of controlling for covariates by entering them as individual predictors in regression analyses.
The fact that propensity scores did not work out in this particular instance should not be taken to disparage the technique. Rubin (1997), in an introduction to propensity scores for the non-expert, discusses the general superiority of propensity scores to ordinary regression as a way of controlling for potentially confounding variables. Indeed, Jackie's attempts to use propensity scores should be considered a learning experience for all involved with her research.
References
Lowe, E. D. (2003). Identity, activity, and the well-being of adolescents and youths: Lessons from young people in a Micronesian society. Culture, Medicine and Psychiatry, 27, 187-219.
Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757–763.
Jackie Wiersma, who successfully defended her Ph.D. research at Texas Tech University a few months ago, just had her officially approved dissertation posted in the university's online repository (click here to read the dissertation). Alan and Bo each served on Jackie's committee (along with Chairperson Judith Fischer and Kitty Harris), Bo via speakerphone given his move from Texas Tech to Penn State a couple years ago.
Jackie used longitudinal data, which can be of some help in strengthening one's argument for particular directions of causality (see here and here). To strengthen some of Jackie's arguments further, however, her committee recommended the use of a technique known as propensity scores (Bo, in particular, played a key role by finding online resources such as this one on the technique).
Jackie's study of adolescent and young-adult drinking involved a number of hypotheses. For simplicitly of presentation, the remainder of this entry focuses on one that predicted an individual's level of drinking as an adolescent would be associated with his or her romantic partner's drinking when the focal individual was a young adult. In other words, would being a drinker as an adolescent propel someone to select a relatively heavy drinker for a romantic partner in the future?
The key predictor variable -- in this case, adolescents' own drinking status -- is discussed in the dissertation analogously to being "assigned" to a "treatment" condition, even though such drinking status is measured as it occurs naturalistically (known in epidemiology as an "observational" variable). The main ideas involving the propensity scores are discussed in the following excerpts from Jackie's dissertation:
For the selection hypothesis, the first prediction was that assignment to group (adolescent drinker and nondrinker) would be related to partner drinking in young adulthood. Thus, it is important to take into account the possible covariates of the assignment to adolescent drinking (p. 56)...
Within the proposed hypotheses, the groups (drinkers and nondrinkers) should show differences in the differentiating variables, thus, this study examined relevant background information (e.g., parental drinking, sensation seeking, peer drinking, college enrollment) that might plausibly affect the group with which individuals are identified. A propensity score is a measure from 0 to 1 of the likelihood of being in one of two designated groups. In this study, a score of 0 means a high probability of a person being a drinker and a 1 score means a person is a nondrinker. These scores were created in SAS using logistic regressions to predict "drinker" versus "nondrinker" (p. 57).
There are many approaches that are used for propensity score matching to adjust for group differences. For this study, the stratifying propensity scores approach was used... After this step, implementing regression models, one can compare the drinker group to the nondrinker group without worrying about the impact of any baseline differences on selection into the groups (Lowe, 2003) (p. 57).
One limitation of the way propensity scores were implemented here was that, in trying to create drinker and non-drinker groups that did not differ on the other covariates, a substantial loss of cases occurred. The following is a hypothetical example (which may have approximated what actually happened). Finding a large number of adolescent drinkers who had high values on traditional predictors of adolescent drinking (e.g., peer drinking) is easy; finding non-drinkers with similarly high levels on the traditional predictors is not. Thus, the number of participants in the former group would have to be shaved down to match the cell size for the latter group. For this reason, the propensity-score analyses were de-emphasized in Jackie's dissertation, in favor of controlling for covariates by entering them as individual predictors in regression analyses.
The fact that propensity scores did not work out in this particular instance should not be taken to disparage the technique. Rubin (1997), in an introduction to propensity scores for the non-expert, discusses the general superiority of propensity scores to ordinary regression as a way of controlling for potentially confounding variables. Indeed, Jackie's attempts to use propensity scores should be considered a learning experience for all involved with her research.
References
Lowe, E. D. (2003). Identity, activity, and the well-being of adolescents and youths: Lessons from young people in a Micronesian society. Culture, Medicine and Psychiatry, 27, 187-219.
Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757–763.
Subscribe to:
Posts (Atom)