Testing the Assumptions of Independence and Stationarity
This site will contain comments and discussion concerning the proposed method of Regenwetter, Dana, and Davis-Stober (2011) for analyzing structural properties in choice data. They proposed a method for analyzing transitivity in choice data, based on the assumption that responses to the same choice over repetitions are iid; that is, they assume that they are independent and identically distributed. I have previously written a comment on their method (Birnbaum, in press), to which Regenwetter, et al. (in press) have responded.
However, subsequent to these papers, some additional progress has been made on this topic. In the paper linked below, you can find statistical tests based on the Monte Carlo procedure of Smith and Batchelder (2008) for testing iid assumptions and applications thereof to the data of Regenwetter, et al (2011). These tests indicate that the assumptions of iid can be rejected for those data.
Here is a paper for download on the statistical tests and reanalysis of Regenwetter, et al. (2011).
Furthermore, we have now analyzed data from three similar experiments by Birnbaum and Bahra, which also refute the iid assumptions. A paper on these results will shortly be posted at this site.
One might think that the only purpose of the iid assumptions in Regenwetter, et al. (2011) is their use to develop statistical tests. However, the marginal choice proportions can be misleading if they are averaged over data that violate iid. Therefore, choice proportions do not represent a proper basis for testing structural properties if they are based on data that violate iid.
- Birnbaum, M. H. (2011). Testing mixture models of transitive preference: Comments on
Regenwetter, Dana, and Davis-Stober (2011). Psychological Review, in press.
- Regenwetter, M., Dana, J. & Davis-Stober, C. (2010). Testing Transitivity of Preferences on
Two- Alternative Forced Choice Data. Frontiers in Psychology, 1, 148. doi:
- Regenwetter, M., Dana, J., & Davis-Stober, C. P. (2011). Transitivity of Preferences.
Psychological Review, 118, 42-56.
- Regenwetter, M., Dana, J., Davis-Stober, C. P., and Guo, Y. (in press). Parsimonious testing of transitive or intransitive preferences: Reply to Birnbaum (2011). Psychological
Review, in press.
Here is a link to Birnbaum's publications