Relation to Lists

Important

Data frames are somewhat strange objects that can cause a lot of confusion. To new users, data frames usually feel more similar to matrices than lists, but the opposite is actually true. The underlying implementation of data frames doesn’t match their apparent tabular structure. This page compares data frames to lists and matrices. Understanding these similarities and differences will help you use data frames more effectively and avoid many common mistakes.

Did you notice the similarity between the code that we used to create data frames on the last page and the code that we used to create lists in the last tutorial?

# Create a data frame
someDataFrame <- data.frame(
  a = rnorm(10),
  b = 1:10,
  c = letters[1:10]
)

# Create a list
someDataFrame <- list(
  a = rnorm(10),
  b = 1:10,
  c = letters[1:10]
)

If so, you were right to see some parallels: lists and data frames are actually the same thing!

Data Frames Are Actually Lists

Data frames are actually implemented as a special kind of list. Each column in a data frame is an element of a list, but all of these list elements must be vector-like objects (not necessarily atomic vectors, though) with equal length. In other words, a data frame is just a list where each slot contains a vector representing one column in the data frame, but with additional constraints and attributes that allow the data frame to behave like a 2 dimensional table.

d1 <- data.frame(
  a = sample(c(TRUE, FALSE), 10, replace = TRUE),
  b = sample(c("foo", "bar"), 10, replace = TRUE),
  c = runif(10)
)
d2 <- data.frame(a = 1:10, b = c(-1, 1), c = seq(0.1, 1, 0.1))
d3 <- data.frame(x = -5:4, y = c(1, 10), z = seq(1, 1.5, length.out = 10))

For example, d1, d2, and d3 are all data frames.

is.data.frame(d1)

[1] TRUE

is.data.frame(d2)

[1] TRUE

is.data.frame(d3)

[1] TRUE

But all three are also lists.

is.list(d1)

[1] TRUE

is.list(d2)

[1] TRUE

is.list(d3)

[1] TRUE

Not all Lists are Data Frames

All data frames are lists, but the inverse is not always true. Some lists are not data frames.

l1 <- list()
l2 <- list(
  a = sample(c(TRUE, FALSE), 10, replace = TRUE),
  b = sample(c("foo", "bar"), 10, replace = TRUE),
  c = runif(10)
)
l3 <- list(d2, d3)

As you can see below, l1, l2, and l3 are all lists, but they’re not data frames.

is.list(l1)

[1] TRUE

is.list(l2)

[1] TRUE

is.list(l3)

[1] TRUE

is.data.frame(l1)

[1] FALSE

is.data.frame(l2)

[1] FALSE

is.data.frame(l3)

[1] FALSE

List \(\leftrightarrow\) Data Frame Conversion

We can use the as.list() type conversion function to convert a data frame to a list. For example, in the following code, we convert d1 into a list, l1a.

(l1a <- as.list(d1))

$a
 [1]  TRUE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE

$b
 [1] "bar" "foo" "bar" "foo" "bar" "foo" "foo" "bar" "foo" "foo"

$c
 [1] 0.4395132 0.1226633 0.2210836 0.9823659 0.0998729 0.7353745 0.5083550
 [8] 0.5169476 0.4064910 0.3931368

is.list(l1a)

[1] TRUE

is.data.frame(l1a)

[1] FALSE

Likewise, we can use the as.data.frame() to convert a list to a data frame. Below, we invert the conversion from above.

(d1a <- as.data.frame(l1a))

       a   b         c
1   TRUE bar 0.4395132
2  FALSE foo 0.1226633
3   TRUE bar 0.2210836
4   TRUE foo 0.9823659
5   TRUE bar 0.0998729
6   TRUE foo 0.7353745
7   TRUE foo 0.5083550
8   TRUE bar 0.5169476
9   TRUE foo 0.4064910
10 FALSE foo 0.3931368

is.data.frame(d1a)

[1] TRUE

In most situations, these type conversions do not destroy an information in the underlying data. Below, we see that the converted-and-back-converted version of the data frame, d1a, is identical to the original data frame, d1.

identical(d1, d1a)

[1] TRUE

Practice

Use a single function to convert the data frame you created in the last practice problem to a list.

Interactive Editor
Solution

Recall my solution to the last problem.

df

  text integer complex
1 text      10    1+1i
2 text       8    2+2i
3 text       3    3+3i
4 text       7    4+1i
5 text       1    5+2i
6 text       5    1+3i
7 text       7    2+1i

To convert this data frame to a list, I simply apply the as.list() function.

as.list(df)

$text
[1] "text" "text" "text" "text" "text" "text" "text"

$integer
[1] 10  8  3  7  1  5  7

$complex
[1] 1+1i 2+2i 3+3i 4+1i 5+2i 1+3i 2+1i

Data Frames Are Not Matrices

As you have probably surmised, data frames are not matrices. Although data frames and matrices share many superficial similarities, they’re almost entirely unrelated types of objects.

is.matrix(d1)

[1] FALSE

is.matrix(d2)

[1] FALSE

is.matrix(d3)

[1] FALSE

Consequently, you cannot apply matrix operations (such as matrix multiplication) to a data frame.

d2 %*% t(d3)

Error in `d2 %*% t(d3)`:
! requires numeric/complex matrix/vector arguments

If you wanted to analyze a data frame like a matrix, you could type cast the data frame into a matrix using the as.matrix() function.

as.matrix(d2) %*% t(as.matrix(d3))

       [,1]       [,2]       [,3]       [,4]       [,5]      [,6]       [,7]
 [1,]  -5.9 -13.894444  -3.888889 -11.883333 -1.8777778 -9.872222  0.1333333
 [2,]  -8.8   2.211111  -4.777778   6.233333 -0.7555556 10.255556  3.2666667
 [3,] -15.7 -21.683333  -9.666667 -15.650000 -3.6333333 -9.616667  2.4000000
 [4,] -18.6  -5.577778 -10.555556   2.466667 -2.5111111 10.511111  5.5333333
 [5,] -25.5 -29.472222 -15.444444 -19.416667 -5.3888889 -9.361111  4.6666667
 [6,] -28.4 -13.366667 -16.333333  -1.300000 -4.2666667 10.766667  7.8000000
 [7,] -35.3 -37.261111 -21.222222 -23.183333 -7.1444444 -9.105556  6.9333333
 [8,] -38.2 -21.155556 -22.111111  -5.066667 -6.0222222 11.022222 10.0666667
 [9,] -45.1 -45.050000 -27.000000 -26.950000 -8.9000000 -8.850000  9.2000000
[10,] -48.0 -28.944444 -27.888889  -8.833333 -7.7777778 11.277778 12.3333333
            [,8]      [,9] [,10]
 [1,] -7.8611111  2.144444 -5.85
 [2,] 14.2777778  7.288889 18.30
 [3,] -3.5833333  8.433333  2.45
 [4,] 18.5555556 13.577778 26.60
 [5,]  0.6944444 14.722222 10.75
 [6,] 22.8333333 19.866667 34.90
 [7,]  4.9722222 21.011111 19.05
 [8,] 27.1111111 26.155556 43.20
 [9,]  9.2500000 27.300000 27.35
[10,] 31.3888889 32.444444 51.50

You should be carefully about any type conversions, though. If your data frame contains mixed column types, R will coerce every column into some compatible type by applying the same coercion rule you learned about in the Vectors & Matrices module.

(m1 <- as.matrix(d1))

      a       b     c          
 [1,] "TRUE"  "bar" "0.4395132"
 [2,] "FALSE" "foo" "0.1226633"
 [3,] "TRUE"  "bar" "0.2210836"
 [4,] "TRUE"  "foo" "0.9823659"
 [5,] "TRUE"  "bar" "0.0998729"
 [6,] "TRUE"  "foo" "0.7353745"
 [7,] "TRUE"  "foo" "0.5083550"
 [8,] "TRUE"  "bar" "0.5169476"
 [9,] "TRUE"  "foo" "0.4064910"
[10,] "FALSE" "foo" "0.3931368"

typeof(m1)

[1] "character"

(d4 <- data.frame(x = rep(c(TRUE, FALSE), 3), y = runif(6)))

      x          y
1  TRUE 0.82279816
2 FALSE 0.64067816
3  TRUE 0.65752167
4 FALSE 0.44481012
5  TRUE 0.86688303
6 FALSE 0.06812604

(m4 <- as.matrix(d4))

     x          y
[1,] 1 0.82279816
[2,] 0 0.64067816
[3,] 1 0.65752167
[4,] 0 0.44481012
[5,] 1 0.86688303
[6,] 0 0.06812604

typeof(m4)

[1] "double"

Practice

Consider the following data frame when answering this question.

df <- data.frame(
    x1 = rep(c(TRUE, FALSE), 3),
    x2 = seq(2, 12, 2)
)

df

     x1 x2
1  TRUE  2
2 FALSE  4
3  TRUE  6
4 FALSE  8
5  TRUE 10
6 FALSE 12

Predict what the following code will print.

t(df)

Check your prediction using the interactive editor, and explain why this code prints the observed result.

Recall that the t() function transposes the matrix provided as its argument.

Even though we can’t do matrix operations on data frames, the code doesn’t error. Our original data frame hasn’t made it through the operation intact, though.

t(df)

   [,1] [,2] [,3] [,4] [,5] [,6]
x1    1    0    1    0    1    0
x2    2    4    6    8   10   12

In fact, we truly cannot do matrix operations on data frames, and the apparent success of this command isn’t a counterexample. Before running t(), R is type casting df into a matrix and then transposing that matrix. So, the above is equivalent to the following.

t(as.matrix(df))

   [,1] [,2] [,3] [,4] [,5] [,6]
x1    1    0    1    0    1    0
x2    2    4    6    8   10   12

The type casting explains why the logical values in the x1 column appear as numeric values in the corresponding transposed row. The least destructive type conversion required casting df as a numeric matrix.

as.matrix(df)

     x1 x2
[1,]  1  2
[2,]  0  4
[3,]  1  6
[4,]  0  8
[5,]  1 10
[6,]  0 12

Knowledge Check

In one sentence, explain how a data frame is related to a list.
if a and b are both data frames, why will a %*% t(b) produce an error?
If a data frame has one text column and one numeric column, what happens if you convert it to a matrix with as.matrix()? Why?