Because data frames are just lists, we can access a data frame’s columns using the same methods we would use for lists. To access a single column by name, the most efficient method is typically the $ operator.
We can also use the single, [], or double, [[]], square bracket operators. As with lists, these operators differ in two respects: how many columns they can select and how they format the resulting selection.
[] Can select multiple columns and always returns a data frame.
[[]] Can select only one column and returns the column contents as a vector (or whatever type of object the column contained).
# Return a one-column data frame comprising the 'b' column from d1d1["b"]
b
1 bar
2 bar
3 foo
4 foo
5 foo
6 bar
7 foo
8 foo
9 foo
10 bar
# The same as above, but using the column index instead of the column named1[2]
b
1 bar
2 bar
3 foo
4 foo
5 foo
6 bar
7 foo
8 foo
9 foo
10 bar
# Return a two-column data frame comprising the 'a' and 'b' columns from d1d1[c("a", "b")]
a b
1 FALSE bar
2 TRUE bar
3 TRUE foo
4 FALSE foo
5 FALSE foo
6 FALSE bar
7 FALSE foo
8 FALSE foo
9 FALSE foo
10 TRUE bar
# The same as above, but using the column indices instead of the column namesd1[1:2]
a b
1 FALSE bar
2 TRUE bar
3 TRUE foo
4 FALSE foo
5 FALSE foo
6 FALSE bar
7 FALSE foo
8 FALSE foo
9 FALSE foo
10 TRUE bar
# Return the 'b' column from d1 as a character vectord1[["b"]]
Data frames also support matrix-style subsetting, where we define the selection by specifying both the row and column indices.
# Return the first two rows of columns `b` and `c`d1[1:2, 2:3]
b c
1 bar 0.05953239
2 bar 0.89668374
# Return all rows from columns `a` and `b`d1[, 1:2]
a b
1 FALSE bar
2 TRUE bar
3 TRUE foo
4 FALSE foo
5 FALSE foo
6 FALSE bar
7 FALSE foo
8 FALSE foo
9 FALSE foo
10 TRUE bar
# Return the second and third rows of all columnsd1[2:3, ]
a b c
2 TRUE bar 0.8966837
3 TRUE foo 0.1961133
Quirks of Matrix-Style Subsetting
In most cases, matrix style subsetting behaves the same was as the [] list-style operator. You can select any number of columns, and the selection is returned as a data frame, but there is one important exception. If you select a single column using matrix-style subsetting, the selection will be converted to a vector.
If you want to create a one-column data frame by extracting a single column from another data frame, you should probably use the [] list-style subsetting operator. That being said, if you really want to use matrix-style subsetting, you can achieve the desired effect by adding the drop = FALSE argument inside the brackets.
I wouldn’t recommend using this pattern since it produces non-idiomatic R code (i.e., by convention, we should only specify function arguments inside the parentheses of a function call). You might come across this syntax in the wild, however, so it’s worth understanding what’s happening.
Convenience Functions
Calling the head() function on a data frame returns the first n rows of the data frame.
head(d1, 3)
a b c
1 FALSE bar 0.05953239
2 TRUE bar 0.89668374
3 TRUE foo 0.19611331
Likewise, the tail() function returns the final n rows of the data frame.
tail(d1, 3)
a b c
8 FALSE foo 0.5196561
9 FALSE foo 0.9868501
10 TRUE bar 0.3935425
As with vectors, providing a negative value for the n argument returns the complementary number of rows from either the top or bottom of the data frame.
head(d1, -3)
a b c
1 FALSE bar 0.05953239
2 TRUE bar 0.89668374
3 TRUE foo 0.19611331
4 FALSE foo 0.36490166
5 FALSE foo 0.09832523
6 FALSE bar 0.90508043
7 FALSE foo 0.92559116
tail(d1, -3)
a b c
4 FALSE foo 0.36490166
5 FALSE foo 0.09832523
6 FALSE bar 0.90508043
7 FALSE foo 0.92559116
8 FALSE foo 0.51965613
9 FALSE foo 0.98685013
10 TRUE bar 0.39354245
Practice
The iris dataset is provided by Base R and available in any R session.
At this point, I hope it comes as no surprise to hear that we can overwrite the columns of a data frame using the same procedures that we use to modify list slots. When modifying one column at a time, we directly apply the intuitive operations.
# View the original data framed1
a b c
1 FALSE bar 0.05953239
2 TRUE bar 0.89668374
3 TRUE foo 0.19611331
4 FALSE foo 0.36490166
5 FALSE foo 0.09832523
6 FALSE bar 0.90508043
7 FALSE foo 0.92559116
8 FALSE foo 0.51965613
9 FALSE foo 0.98685013
10 TRUE bar 0.39354245
# Modify some columnsd1$a <- LETTERS[1:10]d1[[2]] <-rnorm(10)d1["c"] <-rep(c(TRUE, FALSE), each =5)# View the modified data framed1
a b c
1 A -0.5726434 TRUE
2 B 1.0046224 TRUE
3 C 0.6288373 TRUE
4 D -1.9856082 TRUE
5 E 0.5412699 TRUE
6 F 0.9958062 FALSE
7 G -0.3716671 FALSE
8 H -0.1168243 FALSE
9 I 2.2861396 FALSE
10 J -0.2126098 FALSE
When modifying multiple columns with the [], operator, it’s best to supply the replacement values as a data frame or list with the same size as the selected columns. For example, below we replace the first two columns of d1 with an equivalently sized data frame.
If we only want to replace part of a column, we can use matrix-style selection to choose the target cells. Below, we first overwrite the first five rows in column b. We then overwrite rows 3 through 6 in columns a and c.
In the second command, notice the pattern in which the contents of the list are recycled to fill the target cells. The recycling rules for data frames can be a little tricky.
Recycling
When the replacement size doesn’t match the selection size, R will use recycling to resolve the discrepancy, as with vectors and matrices. However, it’s not always easy to predict how the replacement will behave.
If we supply a single vector as the replacement data, the recycling operates the same way it does for matrices. For example, in the following code, we replace the first two columns of d1 by recycling the vector 1:5.
Nothing new here. We simply re-use the elements of the replacement vector, 1:5, until each selected cell is overwritten. Things get more complicated with we supply the replacement data as a list or data frame, though. Below, we want to replace the a and c columns in d1 with the contents of a list, but the vectors therein aren’t long enough, so they will need to be recycled.
d1[c("a", "c")] <-list(c("yes", "no"), 8:12)d1
a b c
1 yes 1 8
2 no 2 9
3 yes 3 10
4 no 4 11
5 yes 5 12
6 no 1 8
7 yes 2 9
8 no 3 10
9 yes 4 11
10 no 5 12
Note that each list slot acts as the replacement for one of the selected columns. So, the recycling happens within columns: the contents of each list slot are recycled independently to resolve the respective size discrepancy. In this case,
The vector c("yes", "no") is recycled five times to match the length of column a.
The vector 8:12 only needs to be recycled twice.
Restrictions
As with matrices, R is oddly specific (in a slightly different way) about the kinds of size discrepancies it will automatically resolved when modifying data frames. When all lengths align cleanly (as in the examples above), R will apply recycling silently (i.e., without any messages or warnings).
Ideal
All replacement lengths \(\leq\) Respective selection lengths
All replacement lengths evenly divide their respective selection lengths
Number of replacement list slots = Number of columns selected
Number of replacement data frame columns = Number of columns selected
For several other types of size discrepancy, R will apply recycling with a warning.
Acceptable
Replacement length > Selection length
Replacement length cleanly divides the selection length
Replacement length exceeds selection length
Replacement list contains more slots than columns selected
Replacement data frame contains more slots than columns selected
In one specific situation, R will throw an error and refuse to implement the replacement.
Invalid
Replacement length does not evenly divide the selection length.
Examples of Acceptable Replacement
When working with data frames, R is quite permissive with abuses of recycling. So, all of the following examples represent legal uses of replacement that R will run without errors, but these examples will all trigger warnings.
Replace the first two columns of d1 by using the first 20 elements from the vector 100:500.
d1[1:2] <-100:500
Warning in matrix(value, n, p): data length [401] is not a sub-multiple or
multiple of the number of rows [10]
Replace the a and c columns in d1 with the first two columns of a three-column data frame.
d1[c("a", "c")] <-data.frame("foo", "bar", "baz")
Warning in `[<-.data.frame`(`*tmp*`, c("a", "c"), value = structure(list(:
provided 3 variables to replace 2 variables
d1
a b c
1 foo 110 bar
2 foo 111 bar
3 foo 112 bar
4 foo 113 bar
5 foo 114 bar
6 foo 115 bar
7 foo 116 bar
8 foo 117 bar
9 foo 118 bar
10 foo 119 bar
Examples of Invalid Replacement
For the most part, R will only complain when the replacement length does not cleanly divide selection length. So, the following two examples represent some of the few situations in which R will completely refuse to apply our requested replacement.
Here, we try to replace the first two columns of d1 (i.e., 20 total cells) with the length-three vector 1:3.
d1[1:2] <-1:3
Error in `[<-.data.frame`:
! replacement has 3 items, need 20
This time, we make essentially the same mistake, but now the incompatible vectors live in a list. We’re trying to replace the a and c columns in d1 with a list containing a length-four vector and a length-eight vector.
d1[c("a", "c")] <-list(letters[4], 1:8)
Error in `[<-.data.frame`:
! replacement element 2 has 8 rows, need 10
Adding Columns
As with lists, we can add new columns to an existing data frame using the $ or [[]] operators.
# Create a baseline data frame(d2 <-data.frame(a =rep(0, 10)))
a
1 0
2 0
3 0
4 0
5 0
6 0
7 0
8 0
9 0
10 0
# Various ways of adding new single columnsd2$b <- letters[1:10]d2[["c"]] <-runif(10)d2[[4]] <-rnorm(10)d2
a b c V4
1 0 a 0.16675152 0.62950639
2 0 b 0.66826056 -0.84161264
3 0 c 0.06672647 0.61689049
4 0 d 0.06259623 1.89699518
5 0 e 0.19566288 -1.55533153
6 0 f 0.37155647 1.03521774
7 0 g 0.52814765 -0.02203947
8 0 h 0.71506695 -0.04771449
9 0 i 0.16562455 -1.61827805
10 0 j 0.87183256 -0.63305394
As you may expect, R will use recycling when we specify too few elements for the new column.
d2["alice"] <-c(TRUE, FALSE)d2
a b c V4 alice
1 0 a 0.16675152 0.62950639 TRUE
2 0 b 0.66826056 -0.84161264 FALSE
3 0 c 0.06672647 0.61689049 TRUE
4 0 d 0.06259623 1.89699518 FALSE
5 0 e 0.19566288 -1.55533153 TRUE
6 0 f 0.37155647 1.03521774 FALSE
7 0 g 0.52814765 -0.02203947 TRUE
8 0 h 0.71506695 -0.04771449 FALSE
9 0 i 0.16562455 -1.61827805 TRUE
10 0 j 0.87183256 -0.63305394 FALSE
We can add multiple columns using the [] operator.
a b c V4 alice V6 V7 foo bar
1 0 a 0.16675152 0.62950639 TRUE -1.1262705 0.9714567 TRUE FALSE
2 0 b 0.66826056 -0.84161264 FALSE 0.4890620 0.1815502 TRUE FALSE
3 0 c 0.06672647 0.61689049 TRUE -0.9121183 -1.4094829 TRUE FALSE
4 0 d 0.06259623 1.89699518 FALSE -0.3190297 -0.5837219 TRUE FALSE
5 0 e 0.19566288 -1.55533153 TRUE -1.4516665 0.7839286 TRUE FALSE
6 0 f 0.37155647 1.03521774 FALSE -1.5835851 -0.6618996 TRUE FALSE
7 0 g 0.52814765 -0.02203947 TRUE -1.6295050 -0.7508775 TRUE FALSE
8 0 h 0.71506695 -0.04771449 FALSE -0.5745959 -0.1194467 TRUE FALSE
9 0 i 0.16562455 -1.61827805 TRUE 0.5779600 0.7589953 TRUE FALSE
10 0 j 0.87183256 -0.63305394 FALSE -0.7166323 -0.5986976 TRUE FALSE
Practice
Run the following code to create an empty data frame containing 10 observations of the 3 variables: a,b, c. Then populate the data frame as described below.
Fill column a with the integer sequence from -9 to 0.
Use the column name to assign the new values.
Fill column b with the even integers between 1 and 20 (inclusive).
Use the numeric column index to assign the new values.
Replace the odd rows in column c with the odd integers between 11 and 20 (inclusive).
Do not overwrite the missing values in the even rows.