This chapter is an effort to summarize a few best practices for using esoteric samples. As we shall see, standard principles of quality research design apply to samples of any size, but smaller samples are more vulnerable to bias. In order to address the common problems of bias that will weaken the quality of the findings, we consider the crucial importance of providing a well-developed rationale for using a small sample. In this way your decisions are transparent and well-grounded.
Much of the current conventions for sample size may be credited to Jacob Cohen’s (1988; 1992) tireless efforts to improve the quality of social science research. His voice is joined by a large group of methodologists deploring the failure of researchers to fully report the details needed to put the findings in context with the rest of the field. Therefore, an empirical study based on a limited sample is further weakened by superficial reporting that will not allow fellow scientists to replicate the complete study if not merely duplicate the calculations. This is a pointed cautionary note for novice researchers who have not yet grasped the nuances of the decisions to be made.
There are myriad ways in which your dataset may be compromised, and each of these influences the quality of the study. The first point of best practice is therefore to provide a compelling rationale for using a small sample instead of a large sample. This means that throughout the report, you must justify the method and analysis, connecting every decision to professional literature. A rigorous literature review provides theoretical justification for the particular population. It should clearly extend the existing body of knowledge by replicating or improving the sample or the method of study. Note that a rigorous reading of the literature does not mean a lengthy report of same.
Thus, the conceptual rationale, the statistical rationale, and the voice of cautious parsimony all help mediate the reception of a design that at first blush appears insufficient. Addressed here in the broad discussion of first ‘why’ and then ‘how’ the research design uses relatively few participants. These broad topics are followed by some discussion of the many ways in which a study may be considered small, each of which will need to be addressed in the rationale.