Start from the table you want

DC §7.1 gives the single most useful habit in this whole unit:

Before any coding, sketch the glyph-ready table you need, then plan the steps to get there in plain English.

For “how does average body mass differ by species?”, the table you wish you had is:

Here we have one row per species and one column for the number you want. The case is a species, not a penguin. That shift in what one row represents is from summarize() and group_by().

Plan in English first

DC’s smoking example plans the wrangling as four English sentences before any code:

1. Make an age-group variable from age.
2. Drop people under 20.
3. Split into groups by age-decade and sex.
4. For each group, count people and count smokers; divide.

Each sentence maps to one operation. Today we learn the vocabulary for sentences like #3 (split into groups) and #4 (for each group, count / average).

Three kinds of building blocks

DC §7.2 boils all of wrangling down to three shapes of data…

• A data frame: rows (cases) and columns (variables)
• A variable: one column
• A scalar: a single value

and three families of functions, sorted by what goes in and what comes out:

Reduction functions: many → one

A reduction takes a whole column and crushes it down to a single number (DC §7.3).

mean(penguins$body_mass_g, na.rm = TRUE)  # one number out

[1] 4201.754

We’ve used many of these before: mean(), sum(), median(), sd(), min(), max(). Here are two new ones:

• n() — how many cases (takes no arguments)
• n_distinct() — how many different values

Transformation functions: column to column

A transformation runs once per row and hands back a column the same length as the input (DC §7.4).

head(penguins$body_mass_g / 1000)   # grams to kg, one value per row

[1] 3.75 3.80 3.25   NA 3.45 3.65

Comparisons are transformations too, they return a column of TRUE/FALSE:

head(penguins$body_mass_g > 4000)

[1] FALSE FALSE FALSE    NA FALSE FALSE

Data verbs: data frame to data frame

A data verb takes a whole data frame and returns a whole (new) data frame (DC §7.5). Here are three rules every dplyr verb obeys:

1. The first argument is always a data frame.
2. The other arguments name columns, unquoted.
3. The output is always a new data frame; the input is never modified.

penguins |> summarize(avg = mean(body_mass_g, na.rm = TRUE))
# penguins itself is unchanged; you get a NEW data frame back

Each step = one verb + helpers

Each step in data wrangling involves a data verb and one or more reduction or transformation functions.

The verb is the action on the table; the reduction/transformation is the detail of what to compute.

penguins |>
  summarize(avg_mass = mean(body_mass_g, na.rm = TRUE))
#         └─ data verb ┘ └─ reduction function ┘

summarize(): collapse a table to a summary

summarize() turns many rows into one row, using reduction functions.

penguins |>
  summarize(
    n        = n(),
    avg_mass = mean(body_mass_g, na.rm = TRUE),
    max_mass = max(body_mass_g, na.rm = TRUE)
  )

# A tibble: 1 × 3
      n avg_mass max_mass
  <int>    <dbl>    <int>
1   344    4202.     6300

Things to notice:

• Each named argument becomes a column in the output.
• The output is a data frame — one row here, but still a data frame.

The na.rm trap

Watch what happens without na.rm = TRUE:

penguins |>
  summarize(avg_mass = mean(body_mass_g))

# A tibble: 1 × 1
  avg_mass
     <dbl>
1       NA

One missing value anywhere in the column makes the whole answer NA. The reduction can’t average around a hole it doesn’t know how to fill.

group_by(): summarize, but per group

group_by() doesn’t compute anything itself. It tags the data frame so the next verb runs once per group.

penguins |>
  group_by(species) |>
  summarize(
    n        = n(),
    avg_mass = mean(body_mass_g, na.rm = TRUE)
  )

# A tibble: 3 × 3
  species       n avg_mass
  <fct>     <int>    <dbl>
1 Adelie      152    3701.
2 Chinstrap    68    3733.
3 Gentoo      124    5076.

Same summarize() as before, now one row per species instead of one row total.

Always include an n()

Average mass by the finest grouping:

penguins |>
  group_by(species, island, sex) |>
  summarize(
    n        = n(),
    avg_mass = mean(body_mass_g, na.rm = TRUE),
    .groups  = "drop"
  )

# A tibble: 13 × 5
   species   island    sex        n avg_mass
   <fct>     <fct>     <fct>  <int>    <dbl>
 1 Adelie    Biscoe    female    22    3369.
 2 Adelie    Biscoe    male      22    4050 
 3 Adelie    Dream     female    27    3344.
 4 Adelie    Dream     male      28    4046.
 5 Adelie    Dream     <NA>       1    2975 
 6 Adelie    Torgersen female    24    3396.
 7 Adelie    Torgersen male      23    4035.
 8 Adelie    Torgersen <NA>       5    3681.
 9 Chinstrap Dream     female    34    3527.
10 Chinstrap Dream     male      34    3939.
11 Gentoo    Biscoe    female    58    4680.
12 Gentoo    Biscoe    male      61    5485.
13 Gentoo    Biscoe    <NA>       5    4588.

Some groups have a handful of birds, others have dozens. A mean from a group of 3 is not the same kind of fact as a mean from a group of 60.

Good habit

This is also a verification move, exactly like Friday: the number that looks like an answer might be resting on almost no data.

What’s next

Today: the framework + summarize() + group_by(). That’s already enough to answer a lot of “how many / what’s the average, by group” questions.

Tomorrow (DC Ch. 10) we add the four verbs:

• filter() — keep some rows (e.g. drop the NA sexes)
• select() — keep some columns
• mutate() — add a new column (this is where transformations live)
• arrange() — reorder rows

With the two we covered today, these handle “take a messy CSV and produce a useful summary.”

What we covered today

• The mindset: sketch the glyph-ready table, plan in English, one sentence per operation
• The framework: reduction (variable→scalar), transformation (variable→variable), data verb (frame→frame)
• summarize(): many rows to one; named args become columns; output is a data frame
• na.rm = TRUE: one NA poisons a whole reduction; spotting it is the skill
• group_by(): tags the frame so the next verb runs per group; changes what a “case” means
• n(): include it in every summary so you never trust a mean from three rows

Activity: summarising the penguins

Roughly 25 minutes. Individually or with a neighbor.

• Activity Canvas Link

You’ll classify functions into the three families, build up summarize() and group_by() pipelines on penguins, deliberately trip the na.rm trap, and practice predicting the shape of a grouped result before you run it.

To turn in: save your .qmd and rendered HTML.

Wrap-up & looking ahead

Tomorrow (Tuesday): the row and column verbs, filter, select, mutate, arrange, the rest of the dplyr toolkit.