Best Practices in Quantitative Methods
A series of books and articlesJason W. Osborne, Ph.D.
Why should you care about Best Practices?
Decision makers are increasingly pressured to make evidence-based decisions, yet these decisions are only as good as the evidence they are based on. It seems every year we are hearing about new, expensive “miracle drugs” and educational interventions and policy initiatives that initially show promise and later turn out to be no more effective than simple, cheap, previously available medicines, interventions, or pedagogies.
Research has ethical and moral consequences. Decisions to expend resources and affect lives are based on our results (or lack thereof), If you are a researcher and you use best practices, you maximize the probability that your research will actually be of use to someone else. The world depends on research to give accurate, unbiased evidence for decisionmaking, and sometimes to speak truth to power. If you believe this as I do, then you cannot engage in quantitative research without attempting to do it in the best way possible. There is almost never a (good) reason not to use best practices.
In best practices, we attempt to leave behind the baggage of the 20th century, to wipe the canvas clean and paint a new era of research methods. We want you to succeed, so that you may make our world, and our children’s world, a better place.
The world doesn’t need another textbook reviewing how to calculate correlation coefficients, that treats ANOVA and regression like two different worlds, that genuflects at the altar of p < .05. These books are a challenge to you, fellow researcher. Shrug off the shackles of 20th century methodology, and the next time you sit down to examine your hard-won data, challenge yourself to implement one new methodology that represents a best practice. And each time afterward, add one more.
There it is. The gauntlet has been cast down. Do you pick it up, accepting my challenge?
Looking for resources related to one of my books (link to Amazon.com list of my books)?
08/15/2016: Check out the Research in Action Podcast highlighting my work on data cleaning
Regression and Linear Modeling: Best Practices and Modern Methods
Download the RESOURCES PDF (still in development) that contains links to data, syntax from chapters, and more!
Contact me if you would like to beta test the book with your class...
Exploratory Factor Analysis using SAS: Best Practices and Modern Methods
ISBN # 978-1-62960-064-2
Explore the mysteries of Exploratory Factor Analysis (EFA) with SAS with an applied and user-friendly approach.
Exploratory Factor Analysis with SAS focuses solely on EFA, presenting a thorough and modern treatise on the different options, in accessible language targeted to the practicing statistician or researcher. This book provides real-world examples using real data, guidance for implementing best practices in the context of SAS, interpretation of results for end users, and it provides resources on the book’s author page. Faculty teaching with this book can utilize these resources for their classes, and individual users can learn at their own pace, reinforcing their comprehension as they go.
Exploratory Factor Analysis with SAS reviews each of the major steps in EFA: data cleaning, extraction, rotation, interpretation, and replication. The last step, replication, is discussed less frequently in the context of EFA but, as we show, the results are of considerable use. Finally, two other practices that are commonly applied in EFA, estimation of factor scores and higher-order factors, are reviewed. Best practices are highlighted throughout the chapters.
A rudimentary working knowledge of SAS is required but no familiarity with EFA or with the SAS routines that are related to EFA is assumed.
Using SAS University Edition? You can use the code and data sets provided with this book.
Best Practices in Exploratory Factor AnalysisAvailable in printed and Kindle formats from: Amazon.com.
Best Practices in Exploratory Factor Analysis (EFA) is a practitioner-oriented look at this popular and often-misunderstood statistical technique. We avoid formulas and matrix algebra, instead focusing on evidence-based best practices so you can focus on getting the most from your data.
Each chapter reviews important concepts, uses real-world data to provide authentic examples of analyses, and provides guidance for interpreting the results of these analysis.
Not only does this book clarify often-confusing issues like various extraction techniques, what rotation is really rotating, and how to use parallel analysis and MAP criteria to decide how many factors you have, but it also introduces replication statistics and bootstrap analysis so that you can better understand how precisely your data are helping you estimate population parameters. Bootstrap analysis also informs readers of your work as to the likelihood of replication, which can give you more credibility.
One chapter briefly reviews important issues around data cleaning and missing data, including evidence-based recommendations as to how to deal with important issues that are often not clearly discussed: random responding, outliers, missing data, etc.
At the end of each chapter, the author has recommendations as to how to enhance your mastery of the material, including access to the data sets used in the chapter through his web site. Other resources include syntax and macros for easily incorporating these progressive aspects of exploratory factor analysis into your practice. The web site will also include enrichment activities, answer keys to select exercises, and other resources.
The fourth "best practices" book by the author, Best Practices in Exploratory Factor Analysis continues the tradition of clearly-written, accessible guides for those just learning quantitative methods or for those who have been researching for decades.
For a preview, you can examine:
RESOURCES PDF for links to syntax, data sets, and more. NEW-- just added R syntax and materials -- thanks to Alexander Beaujean!
Best Practices in Logistic RegressionAvailable from: Sage Publishing, and Amazon.com.
Jason W. Osborne’s Best Practices in Logistic Regression provides students with an accessible, applied approach that communicates logistic regression in clear and concise terms. The book effectively leverages readers’ basic intuitive understanding of simple and multiple regression to guide them into a sophisticated mastery of logistic regression. Osborne’s applied approach offers students and instructors a clear perspective, elucidated through practical and engaging tools that encourage student comprehension.
Best Practices in Logistic Regression explains logistic regression in a concise and simple manner that gives students the clarity they need without the extra weight of longer, high-level texts.
"This is a superb primer on a lot of aspects of statistics and research design, but it's masked as a book on logistic regression. The author is hilarious without turning to the bawdy, Andy-Field style cracks where every example is sex, drugs, or rock and roll. (Not that Andy isn't funny in his own way.) I share the author's biases so I think he's completely even-handed on the controversial aspects of decisions about data. The section on OLS and MLE is worth the price of admission, and the appreciation of curvilinear approaches is literally the best I've ever seen. The clarity on all topics is exemplary. He takes you by the hand without being a pedant. What I like best is that he explains why each issue is important in a way that motivates readers to go through the details. This book inspired me to buy his book on data cleaning, too. I envy the folks who can walk down the hall and ask him a question."- review on Amazon
For a preview, you can explore:
Logits and tigers and bears, oh my! A brief look at the simple math of logistic regression and how it can improve dissemination of results.
Download the RESOURCES PDF that contains links to data, syntax from chapters, and more!
Email me if you are using the book with your class and would like to see my Powerpoint slides that I have developed for my classes.
Sweating the Small Stuff: Does data cleaning and testing of assumptions really matter in the 21st century?Available from: Frontiers Research Foundation (free ebook)
Modern statistical software makes it easier than ever to do thorough data screening/cleaning and to test assumptions associated with the analyses researchers perform. However, few authors (even in top-tier journals) seem to be reporting data cleaning/screening and testing assumptions associated with the statistical analyses being reported. Few popular textbooks seem to focus on these basics. In the 21st Century, with our complex modern analyses, is data screening and cleaning still relevant? Do outliers or extreme scores matter any more? Does having normally distributed variables improve analyses? Are there new techniques for screening or cleaning data that researchers should be aware of? Are most analyses robust to violations of most assumptions, to the point that researchers really don’t need to pay attention to assumptions any more? My goal for this special issue is examine this issue with fresh eyes and 21st century methods. I believe that we can demonstrate that these things do still matter, even when using “robust” methods or non-parametric techniques, and perhaps identify when they matter MOST or in what way they can most substantially affect the results of an analysis. I believe we can encourage researchers to change their habits through evidence-based discussions revolving around these issues. It is possible we can even convince editors of important journals to include these aspects in their evaluation /review criteria, as many journals in the social sciences have done with effect size reporting in recent years.
Best Practices in Data Cleaning: A Complete Guide to Everything You Need to Do Before and After Collecting Your Data.Available from: Sage Publishing, and Amazon.com
Many researchers jump straight from data collection to data analysis without realizing how analyses and hypothesis tests can go profoundly wrong without clean data. This book provides a clear, step-by-step process to examining and cleaning data in order to decrease error rates and increase both the power and replicability of results. Jason W. Osborne, author of Best Practices in Quantitative Methods (SAGE, 2008) provides easily-implemented suggestions that are research-based and will motivate change in practice by empirically demonstrating for each topic the benefits of following best practices and the potential consequences of not following these guidelines. If your goal is to do the best research you can do, draw conclusions that are most likely to be accurate representations of the population(s) you wish to speak about, and report results that are most likely to be replicated by other researchers, then this basic guidebook is indispensible.
Go to chapter resources and data sets, and more!"Best Practices in Data Cleaning by Jason Osborne provides a comprehensive guide to data cleaning. Although I have had a great deal of training associated with the process of setting up and reviewing data collection and analysis I had been away from the field for several years, and recent work required that I consult prior to beginning a new project. This book made the process of getting back up to speed enjoyable. I was very pleased by the step-by-step explanations provided, and found the book to be an excellent resource."- review from Amazon
Best Practices in Quantitative Methods
The contributors to Best Practices in Quantitative Methods envision quantitative methods in the 21st century, identify the best practices, and, where possible, demonstrate the superiority of their recommendations empirically. Editor Jason W. Osborne designed this book with the goal of providing readers with the most effective, evidence-based, modern quantitative methods and quantitative data analysis across the social and behavioral sciences.
The text is divided into five main sections covering select best practices in Measurement, Research Design, Basics of Data Analysis, Quantitative Methods, and Advanced Quantitative Methods. Each chapter contains a current and expansive review of the literature, a case for best practices in terms of method, outcomes, inferences, etc., and broad-ranging examples along with any empirical evidence to show why certain techniques are better.
"I bought this book for some research I was doing for my thesis, and when I started reading the book I was amazed at how many different practical issues are addressed. This has become by far my favorite statistical book and is easy to understand for statistical graduate students. I plan on using this book as a resource throughout my career as a data analyst." -review from Amazon
Don't overlook my horrible "A love poem for statisticians." Perfect for telling that "special someone" that they really aren't all that special after all...
Also available on Amazon's Kindle store for those of you with more cents than sense.
Select Recent publications (links to my most-cited articles on Google Scholar)
Osborne, J. W. (2015). What is rotating in exploratory factor analysis? Practical Assessment, Research, and Evaluation, 20(2), 1-7.
Osborne J.W. (2013) Is data cleaning and the testing of assumptions relevant in the 21st century? Frontiers in Psychology. 4:370. doi: 10.3389/fpsyg.2013.00370
Osborne, J. W., & *Fitzpatrick, D. (2012). Replication Analysis in Exploratory Factor Analysis: what it is and why it makes your analysis better. Practical Assessment, Research, and Evaluation, 17(15), 1-8. http://pareonline.net/getvn.asp?v=17&n=15
Osborne, J. W. (2012). Logits and tigers and bears, oh my! A brief look at the simple math of logistic regression and how it can improve dissemination of results. Practical Assessment, Research, and Evaluation, 17(11), p 1-10. http://pareonline.net/pdf/v17n11.pdf
Carleton, R. N., Thibodeau, M. A., Osborne,
J. W., & Asmundson, G. J.G. (2012). Exploring item order in
anxiety-related constructs: Practical impacts of serial position. Practical
Assessment, Research & Evaluation, 17(7). Available online:
Osborne, J. W. (2011). Best Practices in using large, complex samples: The importance of using appropriate weights and design effect compensation. Practical Assessment, Research, and Evaluation, 16(12) 1-7. http://pareonline.net/pdf/v16n12.pdf
Osborne, J.W., & Blanchard, M. R. (2011). Random responding from participants is a threat to the validity of social science research results. Frontiers in Psychology, Vol 1, Article 220, pp. 1-7 doi: 10.3389/ fpsyg.2010.00220.
Osborne, J. W. (2010). Improving your data transformations: Applying Box-Cox transformations as a best practice. Practical Assessment, Research, and Evaluation, 15(12), 1-9. Retrieved from http://pareonline.net/pdf/v15n12.pdf.
My Most Cited publications
Costello, A. B. & Osborne, J. W. (2005). Exploratory Factor Analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 1-9. Online at http://pareonline.net/pdf/v10n7.pdf
Osborne, J. W. (2002). Notes on the use of data transformations. Practical Assessment, Research & Evaluation, 8(6). Available online: http://pareonline.net/getvn.asp?v=8&n=6 .
Osborne, J. W., & Waters, E. (2002). Four assumptions of multiple regression that researchers should always test. Practical Assessment, Research, and Evaluation, 8(2). [Available online at http://pareonline.net/getvn.asp?v=8&n=2 ].
Osborne, J. W., & *Costello, A. B. (2004). Sample size and subject to item ratio in principal components analysis. Practical Assessment, Research, and Evaluation, 9. Online at http://pareonline.net/getvn.asp?v=9&n=11 .
Osborne, J. W. (2000). Advantages of Hierarchical Linear Modeling. Practical Assessment, Research & Evaluation, 7(1). [Available online: http://pareonline.net/getvn.asp?v=7&n=1].
Osborne, J. W., & *Overbay, A. (2004). The power of outliers (and why researchers should ALWAYS check for them). Practical Assessment, Research, and Evaluation, 9(6) Online at http://pareonline.net/getvn.asp?v=9&n=6