A collaborative effort of the individuals named below, summarized by Alan...
Jackie Wiersma, who successfully defended her Ph.D. research at Texas Tech University a few months ago, just had her officially approved dissertation posted in the university's online repository (click here to read the dissertation). Alan and Bo each served on Jackie's committee (along with Chairperson Judith Fischer and Kitty Harris), Bo via speakerphone given his move from Texas Tech to Penn State a couple years ago.
Jackie used longitudinal data, which can be of some help in strengthening one's argument for particular directions of causality (see here and here). To strengthen some of Jackie's arguments further, however, her committee recommended the use of a technique known as propensity scores (Bo, in particular, played a key role by finding online resources such as this one on the technique).
Jackie's study of adolescent and young-adult drinking involved a number of hypotheses. For simplicitly of presentation, the remainder of this entry focuses on one that predicted an individual's level of drinking as an adolescent would be associated with his or her romantic partner's drinking when the focal individual was a young adult. In other words, would being a drinker as an adolescent propel someone to select a relatively heavy drinker for a romantic partner in the future?
The key predictor variable -- in this case, adolescents' own drinking status -- is discussed in the dissertation analogously to being "assigned" to a "treatment" condition, even though such drinking status is measured as it occurs naturalistically (known in epidemiology as an "observational" variable). The main ideas involving the propensity scores are discussed in the following excerpts from Jackie's dissertation:
For the selection hypothesis, the first prediction was that assignment to group (adolescent drinker and nondrinker) would be related to partner drinking in young adulthood. Thus, it is important to take into account the possible covariates of the assignment to adolescent drinking (p. 56)...
Within the proposed hypotheses, the groups (drinkers and nondrinkers) should show differences in the differentiating variables, thus, this study examined relevant background information (e.g., parental drinking, sensation seeking, peer drinking, college enrollment) that might plausibly affect the group with which individuals are identified. A propensity score is a measure from 0 to 1 of the likelihood of being in one of two designated groups. In this study, a score of 0 means a high probability of a person being a drinker and a 1 score means a person is a nondrinker. These scores were created in SAS using logistic regressions to predict "drinker" versus "nondrinker" (p. 57).
There are many approaches that are used for propensity score matching to adjust for group differences. For this study, the stratifying propensity scores approach was used... After this step, implementing regression models, one can compare the drinker group to the nondrinker group without worrying about the impact of any baseline differences on selection into the groups (Lowe, 2003) (p. 57).
One limitation of the way propensity scores were implemented here was that, in trying to create drinker and non-drinker groups that did not differ on the other covariates, a substantial loss of cases occurred. The following is a hypothetical example (which may have approximated what actually happened). Finding a large number of adolescent drinkers who had high values on traditional predictors of adolescent drinking (e.g., peer drinking) is easy; finding non-drinkers with similarly high levels on the traditional predictors is not. Thus, the number of participants in the former group would have to be shaved down to match the cell size for the latter group. For this reason, the propensity-score analyses were de-emphasized in Jackie's dissertation, in favor of controlling for covariates by entering them as individual predictors in regression analyses.
The fact that propensity scores did not work out in this particular instance should not be taken to disparage the technique. Rubin (1997), in an introduction to propensity scores for the non-expert, discusses the general superiority of propensity scores to ordinary regression as a way of controlling for potentially confounding variables. Indeed, Jackie's attempts to use propensity scores should be considered a learning experience for all involved with her research.
References
Lowe, E. D. (2003). Identity, activity, and the well-being of adolescents and youths: Lessons from young people in a Micronesian society. Culture, Medicine and Psychiatry, 27, 187-219.
Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127, 757–763.
