How should a researcher decide what statistical analysis to use? Across medicine and many other disciplines, best practice is now defined as being evidence-based. As a professional, you must be able to justify your choice of diagnosis or therapies in terms of evidence that they are appropriate to the situation, and effective. No less a standard should apply to researchers choosing how to analyse their data. Alas, statistical practices in psychology, education, and across the social sciences too often fail to meet this standard: The journals are full of studies using outdated techniques, inappropriate analyses, errors, wrong conclusions, and opportunities missed.
Several questions are addressed in this introductory chapter. First, what do we mean by evidence-based statistical practice? Second, what are the problems with the dominant approach to quantitative analysis in the social sciences, Null Hypothesis Significance Testing (NHST)? How can evidence guide adoption of better practices? What lessons are there from attempts in other disciplines to do better? Finally, what attitudes should researchers adopt to statistical practices in the twenty-first century?