R correlation: I'm getting inconsistent correlation results with cor() function - Stack Overflow

admin管理员组

文章数量:1410717

I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id
vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.1631033

Other columns do give consistent results such as serBilir vs serChol (0.39675890). I also coded it myself to test it:

v <- function(x,y=x) mean(x*y) - mean(x)*mean(y)
my_corr <- function(x,y) v(x,y) / sqrt(v(x) * v(y))
my_corr(df$years, df$age) # -0.1631033

So why does cor(df[vars_num1], use = "complete.obs", method="pearson") give different results?

I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id
vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.1631033

Other columns do give consistent results such as serBilir vs serChol (0.39675890). I also coded it myself to test it:

v <- function(x,y=x) mean(x*y) - mean(x)*mean(y)
my_corr <- function(x,y) v(x,y) / sqrt(v(x) * v(y))
my_corr(df$years, df$age) # -0.1631033

So why does cor(df[vars_num1], use = "complete.obs", method="pearson") give different results?

• r
• correlation
• pearson-correlation

Share Improve this question Follow asked Mar 11 at 10:24 HunterHunter 35988 silver badges1515 bronze badges 3

• 2 Because "use=complete.obs" will drop all rows if any variable in the data frame contains NA. There are 28 NA in serChol and 4 in platelets (check with summary(df)) so these will be excluded completely. The second cor command doesn't exclude them, so the results differ. – Edward Commented Mar 11 at 10:31
• 2 Use use = "pairwiseplete.obs" to get the same results for both. – Edward Commented Mar 11 at 10:34
• @Edward thank you, pairwiseplete.obs is working – Hunter Commented Mar 11 at 11:26

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 3

I think the problem comes from your NA values. In the second case, the cor function keeps more rows than in the first case. Use na.omit and you will see that you find the same thing.

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id

vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")

df = na.omit(df)
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.17719866

df[vars_num1]

本文标签： R correlation I39m getting inconsistent correlation results with cor() functionStack Overflow