Building a Workflow with Pipes
Pipes are especially powerful when used to chain together multiple data processing steps. Let’s look at a more substantial example.
Suppose we want to:
- Center the variable
age on 18.
- Create scale scores for
extraversion and neuroticism.
- Keep only participants aged 18 or older.
- Select only the scale scores and demographic variables.
- Sort the data by
extraversion in ascending order.
Using dplyr pipes, we can write:
tmp1 <- bfi %>%
mutate(age = age - 18,
extra = rowMeans(across(matches("^e\\d$")), na.rm = TRUE),
neuro = rowMeans(across(matches("^n\\d$")), na.rm = TRUE)) %>%
filter(age >= 0) %>%
select(extra, neuro, age, gen, ed, gm) %>%
arrange(extra)
head(tmp1, 20)
extra neuro age gen ed gm
1 1.0 1.0 5 male 3 man
2 1.0 1.0 1 male 3 man
3 1.6 3.2 0 female 3 woman
4 1.8 4.0 18 female 2 woman
5 2.0 6.0 32 female 3 woman
6 2.2 4.4 5 female 3 woman
7 2.2 4.6 22 male NA man
8 2.2 4.0 17 female 3 woman
9 2.2 2.6 22 female 3 woman
10 2.2 4.2 9 female 5 woman
11 2.2 3.8 36 female 4 woman
12 2.2 2.8 11 female 3 woman
13 2.2 6.0 9 female 3 woman
14 2.2 1.2 3 female 3 woman
15 2.2 4.2 20 female 3 woman
16 2.2 1.4 32 female 4 woman
17 2.4 2.0 50 male 5 man
18 2.4 3.6 3 male 4 man
19 2.4 2.8 36 female 3 woman
20 2.4 1.0 0 female 1 woman
Each step is explicit, readable, and self-contained — it’s immediately clear what happens and in what order.
The same logic written in base R would be much harder to follow:
tmp2 <- bfi
tmp2$age <- tmp2$age - 18
tmp2$extra <- rowMeans(tmp2[grep("^e\\d$", colnames(tmp2))], na.rm = TRUE)
tmp2$neuro <- rowMeans(tmp2[grep("^n\\d$", colnames(tmp2))], na.rm = TRUE)
tmp2 <- tmp2[tmp2$age >= 0, c("extra", "neuro", "age", "gen", "ed", "gm")]
tmp2 <- tmp2[order(tmp2$extra), ]
Both approaches produce the same result — but the piped version is far clearer.
How Pipes Work
The pipe operator simply takes the result from the left-hand side and inserts it as the first argument in the function on the right-hand side.
This means the two lines below are equivalent:
The same applies here:
var(bfi$a1, na.rm = TRUE)
bfi$a1 %>% var(na.rm = TRUE)
What happens when the data isn’t the first argument?
Some functions — like lm() — don’t take the data frame as the first argument. If you try to use a pipe directly, it won’t work:
bfi %>% lm(extra ~ age) # Error
Error in as.data.frame.default(data): cannot coerce class '"formula"' to a data.frame
In these cases, you can use the special placeholder . to tell R explicitly where to insert the piped object:
bfi %>% lm(extra ~ age, data = .)
Call:
lm(formula = extra ~ age, data = .)
Coefficients:
(Intercept) age
3.97269 0.00599
This small trick makes it possible to use pipes with nearly any function, even when the data aren’t the first argument.
Use pipes to perform the following data manipulations on the bfi dataset:
- Create a new variable
age_centered that centers age around its mean.
- Filter the dataset to include only participants with
age greater than 25.
- Select only the new centered age variable, the scores for agreeableness and conscientiousness, and the demographic variables
gen and ed.
- Display the first 20 rows of the resulting dataset.
bfi %>%
mutate(age_centered = age - mean(age, na.rm = TRUE)) %>%
filter(age > 25) %>%
select(age_centered, agree, consc, gen, ed) %>%
head(20)
age_centered agree consc gen ed
1 39.2178571 5.6 2.80 male 5
2 -1.7821429 5.4 3.80 female 2
3 22.2178571 4.0 5.80 female 5
4 4.2178571 3.8 4.20 female 3
5 12.2178571 5.6 4.60 female 3
6 1.2178571 6.0 6.00 male NA
7 19.2178571 4.6 5.20 female 5
8 11.2178571 5.4 3.80 female 3
9 -1.7821429 3.8 4.00 female 4
10 14.2178571 3.8 4.40 female 1
11 -2.7821429 5.2 4.80 female 4
12 -2.7821429 4.4 4.00 male 4
13 -2.7821429 4.6 3.80 male 5
14 21.2178571 5.0 5.40 female 2
15 0.2178571 4.2 4.80 male 5
16 3.2178571 4.2 2.80 male 1
17 3.2178571 5.6 5.25 female 4
18 -2.7821429 4.0 5.20 female 3
19 -1.7821429 5.8 5.00 female 5
20 7.2178571 4.6 4.80 female 5
Back to top