Preface

This book represents a fundamental rethinking of a calculus based first course in probability and statistics. We offer a breadth first approach, where the fundamentals of probability and statistics can be taught in one semester.1 The statistical programming language R plays an essential role throughout the text through simulations, data wrangling, visualizations and statistical procedures. Data sets from a variety of sources, including many from recent, open source scientific articles, are used in examples and exercises. Demonstrations of important facts are given through simulations, with some formal mathematical proofs as well.

This book is an excellent choice for students studying data science, statistics, engineering, computer science, mathematics, science, business, or any field which requires the two semesters of calculus needed to read this book.

The book assumes a mathematical background of Calculus II; that is, differentiation, integration and some infinite series, though much of the book can be read with a much lower level of mathematics. Since an emphasis is placed on understanding results (and robustness to departures from assumptions) via simulation, most parts of the book can be understood without using calculus. Proofs of many results are provided, and justifications via simulations for many more, but this text is not intended to support a proof based course. Readers are encouraged to follow the proofs, but often one wants to understand a proof only after first understanding the result and why it is important.

Tidyverse and base R tools are interspersed, depending on which is better for a particular job. For most visualizations beginning in Chapter 7, ggplot2 is used, and for most data wrangling beginning in Chapter 6, dplyr is used.

Much of the data in this book is provided in the package fosdata. Most of these data sets are from recent, open access publications that are linked to in the help pages of the data. We encourage readers to spend time reading the publications that were written using the data in the book. We have taken two approaches to the data from original papers. In some instances, we provide essentially all of the data from the paper. This allows you to explore the data further and think about other visualizations and analyses that would be useful. It also typically requires some wrangling to get the data in a format for the analysis. In other instances, we have simplified the data from the paper quite a bit. In particular, in a few instances we have modified the data by filtering out observations or averaging in order to make it reasonable to assume independence. Please see the links provided in the help pages of fosdata for details.

Other data is found in other freely available R packages. You will need to install these packages and fosdata to access the data.
Our philosophy in this book is to not shy away from messy data sets. The book contains extensive sections and many exercises that require data cleaning and manipulation. This is an essential part of the course.

Each chapter in the book contains at least one vignette. These short sections contain interesting additions to the book, which are not part of the development of the base material. Examples include choropleth maps, data and gender, and Stein’s paradox. We imagine these vignettes as starting points for further study for some students, or as an interesting addition to the main material.

This book is written in RMarkdown using the bookdown package, by Yihui Xie. The original idea for a course of this type is due to Michael Lamar. The authors wish to thank Matt Schuelke, Kerith Conron, and Christophe Dervieux for helpful discussions. Thanks to Haijun Gong, Kimberly Druschel, Luis Miguel Anguas, Mustafa Attallah, Xue Li, Caden Beddingfield for working through early editions. Thanks to John Kimmel, our editor at CRC Press.

This book is copyright 2021, Darrin Speegle and Bryan Clair. Do not transmit or reuse without express permission.

0.1 Further reading

No book like this would be complete without a list of books that would be useful for the student who wishes to learn more. Here is a list of other resources that that the authors have enjoyed learning from.

  1. ggplot2, by Hadley Wickham, gives a nice overview of the capabilities of the ggplot2 package . Students interested in data visualization would find this book interesting.
  2. Advanced R, by Hadley Wickham, provides much more information on R than what we cover in this book. Computer Science students might enjoy reading this book.
  3. The Statistical Sleuth, by Ramsey and Schafer, will help the student think more like a statistician when dealing with data sets. This book is on a lower level mathematically.
  4. Modern Applied Statistics with S, by Venables and Ripley, is a book that covers more advanced statistical topics without much mathematics.
  5. Introductory Statistics with R, by Peter Dalgaard, is a concise introduction to using R for many types of statistical procedures.
  6. Mathematical Statistics with Applications, by Wackerly, Mendenhall and Schaeffer, is a more mathematical (but still only requiring multivariate calculus and perhaps basic linear algebra) look at the topics of this book. Students interested in the theory and proofs behind the material in this book would enjoy reading it.
  7. The package data.table. If we were going to include a package that isn’t part of the tidyverse, this would be it. It is widely used for data manipulation, and can be considerably faster than other R tools for many classes of problems. If you find that your code is running too slowly, it may be time to learn data.table.

0.2 Installing R and RStudio

R is a programming language, distributed as its own software program.

To install R:

Mac users

  1. Visit the CRAN archive, at https://cran.r-project.org
  2. Find the link that looks like “R-x.x.x.pkg” under the Latest release heading.
  3. Download the “R-x.x.x.pkg” file, double-click it to open, and follow the installation instructions.

Windows users

  1. Visit the CRAN archive, at https://cran.r-project.org
  2. Click on the “Download R for Windows” link at the top of the page.
  3. Click on the “base” link.
  4. Click the large “Download R x.x.x for Windows” link and save the executable file somewhere on your computer.
  5. Run the .exe file and follow the installation instructions.

RStudio is a graphical interface to R. R can work without RStudio, but RStudio requires R to work. Though you may choose to use R in its native form, the improvements that come with RStudio are absolutely worth the effort to install it. In fact, once you have RStudio installed, there is little need to ever run the R program itself.

To install RStudio:

  1. Go to www.rstudio.com and click on the “Download RStudio” button.
  2. Click on “Download RStudio Desktop.”
  3. Click on the version recommended for your system and install it.

Finally, you might choose to install the tidyverse and the book’s data set at this time, by finding the Console in RStudio and typing:

install.packages("tidyverse")
install.packages("remotes")
remotes::install_github(repo = "speegled/fosdata")

More details on package installation are in Section 1.8.

  1. This book contains more material than can reasonably be covered in one semester for most students. We can also imagine this book being used in a more leisurely and throrough two semester sequence. For a more theoretical course, the on-line supplement contains material which would make this book appropriate for a two-semester course in probability and mathematical statistics.↩︎