Dissimilarity Matrix Calculation

Compute all the pairwise dissimilarities (distances) between observations in the data set. The original variables may be of mixed types. In that case, or whenever metric = "gower" is set, a generalization of Gower's formula is used, see ‘Details’ below.

Usage

daisy(x, metric = c("euclidean", "manhattan", "gower"),
      stand = FALSE, type = list(), weights = rep.int(1, p),
      warnBin = warnType, warnAsym = warnType, warnConst = warnType,
      warnType = TRUE)

Arguments

x

numeric matrix or data frame, of dimension $n\times p$, say. Dissimilarities will be computed between the rows of x. Columns of mode numeric (i.e. all columns when x is a matrix) will be recognized as interval scaled variables, columns of class factor will be recognized as nominal variables, and columns of class ordered will be recognized as ordinal variables. Other variable types should be specified with the type argument. Missing values (NAs) are allowed.

metric

character string specifying the metric to be used. The currently available options are "euclidean" (the default), "manhattan" and "gower".
Euclidean distances are root sum-of-squares of differences, and manhattan distances are the sum of absolute differences.

“Gower's distance” is chosen by metric "gower" or automatically if some columns of x are not numeric. Also known as Gower's coefficient (1971), expressed as a dissimilarity, this implies that a particular standardisation will be applied to each variable, and the “distance” between two units is the sum of all the variable-specific distances, see the details section.

stand

logical flag: if TRUE, then the measurements in x are standardized before calculating the dissimilarities. Measurements are standardized for each variable (column), by subtracting the variable's mean value and dividing by the variable's mean absolute deviation.

If not all columns of x are numeric, stand will be ignored and Gower's standardization (based on the range) will be applied in any case, see argument metric, above, and the details section.

type

list for specifying some (or all) of the types of the variables (columns) in x. The list may contain the following components:

"asymm": Asymmetric binary variable, aka "A" in result Types, see dissimilarity.object.
"symm": Symmetric binary variable, aka "S".
"factor": Nominal – the default for factor variables, aka "N". When the factor has 2 levels, this is equivalent to type = "S" for a (symmetric) binary variable.
"ordered": Ordinal – the default for ordered (factor) variables, aka "O", see dissimilarity.object.
"logratio": ratio scaled numeric variables that are to be logarithmically transformed (log10) and then treated as numeric ("I"): must be positive numeric variable.
"ordratio": “raTio”-like variable to be treated as ordered (using the factor codes unclass(as.ordered(x[,j]))), aka "T".
"numeric"/"integer": Interval scaled – the default for all numeric (incl integer) columns of x, aka "I" in result Types, see dissimilarity.object.

Each component is a (character or numeric) vector, containing either the names or the numbers of the corresponding columns of x.

Variables not mentioned in type are interpreted as usual, see argument x, and also ‘default’ above. Consequently, the default type = list() may often be sufficient.

weights

an optional numeric vector of length $p$(=ncol(x)); to be used in “case 2” (mixed variables, or metric = "gower"), specifying a weight for each variable (x[,k]) instead of $1$ in Gower's original formula.

warnBin, warnAsym, warnConst

logicals indicating if the corresponding type checking warnings should be signalled (when found).

warnType

logical indicating if all the type checking warnings should be active or not.

Value

an object of class "dissimilarity" containing the dissimilarities among the rows of x. This is typically the input for the functions pam, fanny, agnes or diana. For more details, see dissimilarity.object.

Details

The original version of daisy is fully described in chapter 1 of Kaufman and Rousseeuw (1990). Compared to dist whose input must be numeric variables, the main feature of daisy is its ability to handle other variable types as well (e.g. nominal, ordinal, (a)symmetric binary) even when different types occur in the same data set.

The handling of nominal, ordinal, and (a)symmetric binary data is achieved by using the general dissimilarity coefficient of Gower (1971). If x contains any columns of these data-types, both arguments metric and stand will be ignored and Gower's coefficient will be used as the metric. This can also be activated for purely numeric data by metric = "gower". With that, each variable (column) is first standardized by dividing each entry by the range of the corresponding variable, after subtracting the minimum value; consequently the rescaled variable has range $[0,1]$, exactly.

Note that setting the type to symm (symmetric binary) gives the same dissimilarities as using nominal (which is chosen for non-ordered factors) only when no missing values are present, and more efficiently.

Note that daisy signals a warning when 2-valued numerical variables do not have an explicit type specified, because the reference authors recommend to consider using "asymm"; the warning may be silenced by warnBin = FALSE.

In the daisy algorithm, missing values in a row of x are not included in the dissimilarities involving that row. There are two main cases,

If all variables are interval scaled (and metric is not "gower"), the metric is "euclidean", and $n_g$ is the number of columns in which neither row i and j have NAs, then the dissimilarity d(i,j) returned is $\sqrt{p/n_g}$ ($p=$ncol(x)) times the Euclidean distance between the two vectors of length $n_g$ shortened to exclude NAs. The rule is similar for the "manhattan" metric, except that the coefficient is $p/n_g$. If $n_g = 0$, the dissimilarity is NA.
When some variables have a type other than interval scaled, or if metric = "gower" is specified, the dissimilarity between two rows is the weighted mean of the contributions of each variable. Specifically, $$d_{ij} = d(i,j) = \frac{\sum_{k=1}^p w_k \delta_{ij}^{(k)} d_{ij}^{(k)}}{ \sum_{k=1}^p w_k \delta_{ij}^{(k)}}. $$ In other words, $d_{ij}$ is a weighted mean of $d_{ij}^{(k)}$ with weights $w_k \delta_{ij}^{(k)}$, where $w_k$= weigths[k], $\delta_{ij}^{(k)}$ is 0 or 1, and $d_{ij}^{(k)}$, the k-th variable contribution to the total distance, is a distance between x[i,k] and x[j,k], see below.

The 0-1 weight $\delta_{ij}^{(k)}$ becomes zero when the variable x[,k] is missing in either or both rows (i and j), or when the variable is asymmetric binary and both values are zero. In all other situations it is 1.

The contribution $d_{ij}^{(k)}$ of a nominal or binary variable to the total dissimilarity is 0 if both values are equal, 1 otherwise. The contribution of other variables is the absolute difference of both values, divided by the total range of that variable. Note that “standard scoring” is applied to ordinal variables, i.e., they are replaced by their integer codes 1:K. Note that this is not the same as using their ranks (since there typically are ties).

As the individual contributions $d_{ij}^{(k)}$ are in $[0,1]$, the dissimilarity $d_{ij}$ will remain in this range. If all weights $w_k \delta_{ij}^{(k)}$ are zero, the dissimilarity is set to NA.

Background

Dissimilarities are used as inputs to cluster analysis and multidimensional scaling. The choice of metric may have a large impact.

References

Gower, J. C. (1971) A general coefficient of similarity and some of its properties, Biometrics 27, 857–874.

Kaufman, L. and Rousseeuw, P.J. (1990) Finding Groups in Data: An Introduction to Cluster Analysis. Wiley, New York.

Struyf, A., Hubert, M. and Rousseeuw, P.J. (1997) Integrating Robust Clustering Techniques in S-PLUS, Computational Statistics and Data Analysis 26, 17–37.

Author

Anja Struyf, Mia Hubert, and Peter and Rousseeuw, for the original version.
Martin Maechler improved the NA handling and type specification checking, and extended functionality to metric = "gower" and the optional weights argument.

Examples


data(agriculture)
## Example 1 in ref:
##  Dissimilarities using Euclidean metric and without standardization
d.agr <- daisy(agriculture, metric = "euclidean", stand = FALSE)
d.agr
#> Dissimilarities :
#>             B        DK         D        GR         E         F       IRL
#> DK   5.408327                                                            
#> D    2.061553  3.405877                                                  
#> GR  22.339651 22.570113 22.661200                                        
#> E    9.818350 11.182576 10.394710 12.567418                              
#> F    3.448188  3.512834  2.657066 20.100995  8.060397                    
#> IRL 12.747549 13.306014 13.080138  9.604166  3.140064 10.564563          
#> I    5.803447  5.470832  5.423099 17.383325  5.727128  2.773085  7.920859
#> L    4.275512  2.220360  2.300000 24.035391 12.121056  4.060788 14.569145
#> NL   1.649242  5.096077  2.435159 20.752349  8.280097  2.202272 11.150785
#> P   17.236299 17.864490 17.664088  5.162364  7.430343 15.164432  4.601087
#> UK   2.828427  8.052950  4.850773 21.485344  8.984431  5.303772 12.103718
#>             I         L        NL         P
#> DK                                         
#> D                                          
#> GR                                         
#> E                                          
#> F                                          
#> IRL                                        
#> I                                          
#> L    6.660330                              
#> NL   4.204759  4.669047                    
#> P   12.515990 19.168985 15.670673          
#> UK   6.723095  7.102112  3.124100 16.323296
#> 
#> Metric :  euclidean 
#> Number of objects : 12
as.matrix(d.agr)[,"DK"] # via as.matrix.dist(.)
#>         B        DK         D        GR         E         F       IRL         I 
#>  5.408327  0.000000  3.405877 22.570113 11.182576  3.512834 13.306014  5.470832 
#>         L        NL         P        UK 
#>  2.220360  5.096077 17.864490  8.052950 
## compare with
as.matrix(daisy(agriculture, metric = "gower"))
#>              B         DK          D        GR          E          F        IRL
#> B   0.00000000 0.22148078 0.08178881 0.8438459 0.38135483 0.11538211 0.47547804
#> DK  0.22148078 0.00000000 0.13969197 0.9145729 0.45208184 0.12117405 0.54620505
#> D   0.08178881 0.13969197 0.00000000 0.8854337 0.42294264 0.09203485 0.51706585
#> GR  0.84384585 0.91457286 0.88543366 0.0000000 0.46249103 0.79339881 0.36836781
#> E   0.38135483 0.45208184 0.42294264 0.4624910 0.00000000 0.33090779 0.09412321
#> F   0.11538211 0.12117405 0.09203485 0.7933988 0.33090779 0.00000000 0.42503100
#> IRL 0.47547804 0.54620505 0.51706585 0.3683678 0.09412321 0.42503100 0.00000000
#> I   0.15222215 0.22294916 0.19380996 0.6916237 0.22913268 0.10177511 0.32325589
#> L   0.15646414 0.06501664 0.07467532 0.9601090 0.49761796 0.16671017 0.59174117
#> NL  0.05318802 0.19426679 0.09477583 0.7906578 0.32816681 0.08816811 0.42229002
#> P   0.66155453 0.73228154 0.70314234 0.1822913 0.28019970 0.61110749 0.18607649
#> UK  0.10095934 0.32244012 0.18274816 0.7629870 0.30049599 0.21634145 0.39461920
#>             I          L         NL         P        UK
#> B   0.1522221 0.15646414 0.05318802 0.6615545 0.1009593
#> DK  0.2229492 0.06501664 0.19426679 0.7322815 0.3224401
#> D   0.1938100 0.07467532 0.09477583 0.7031423 0.1827482
#> GR  0.6916237 0.96010899 0.79065783 0.1822913 0.7629870
#> E   0.2291327 0.49761796 0.32816681 0.2801997 0.3004960
#> F   0.1017751 0.16671017 0.08816811 0.6111075 0.2163414
#> IRL 0.3232559 0.59174117 0.42229002 0.1860765 0.3946192
#> I   0.0000000 0.26848528 0.11202114 0.5093324 0.2401945
#> L   0.2684853 0.00000000 0.16945115 0.7778177 0.2574235
#> NL  0.1120211 0.16945115 0.00000000 0.6083665 0.1281733
#> P   0.5093324 0.77781766 0.60836651 0.0000000 0.5806957
#> UK  0.2401945 0.25742348 0.12817333 0.5806957 0.0000000

## Example 2 in reference, extended  ---  different ways of "mixed" / "gower":

example(flower) # -> data(flower) *and* provide 'flowerN'
#> 
#> flower> data(flower)
#> 
#> flower> str(flower) # factors, ordered, numeric
#> 'data.frame':	18 obs. of  8 variables:
#>  $ V1: Factor w/ 2 levels "0","1": 1 2 1 1 1 1 1 1 2 2 ...
#>  $ V2: Factor w/ 2 levels "0","1": 2 1 2 1 2 2 1 1 2 2 ...
#>  $ V3: Factor w/ 2 levels "0","1": 2 1 1 2 1 1 1 2 1 1 ...
#>  $ V4: Factor w/ 5 levels "1","2","3","4",..: 4 2 3 4 5 4 4 2 3 5 ...
#>  $ V5: Ord.factor w/ 3 levels "1"<"2"<"3": 3 1 3 2 2 3 3 2 1 2 ...
#>  $ V6: Ord.factor w/ 18 levels "1"<"2"<"3"<"4"<..: 15 3 1 16 2 12 13 7 4 14 ...
#>  $ V7: num  25 150 150 125 20 50 40 100 25 100 ...
#>  $ V8: num  15 50 50 50 15 40 20 15 15 60 ...
#> 
#> flower> ## "Nicer" version (less numeric more self explainable) of 'flower':
#> flower> flowerN <- flower
#> 
#> flower> colnames(flowerN) <- c("winters", "shadow", "tubers", "color",
#> flower+                        "soil", "preference", "height", "distance")
#> 
#> flower> for(j in 1:3) flowerN[,j] <- (flowerN[,j] == "1")
#> 
#> flower> levels(flowerN$color) <- c("1" = "white", "2" = "yellow", "3" = "pink",
#> flower+                            "4" = "red", "5" = "blue")[levels(flowerN$color)]
#> 
#> flower> levels(flowerN$soil)  <- c("1" = "dry", "2" = "normal", "3" = "wet")[levels(flowerN$soil)]
#> 
#> flower> flowerN
#>    winters shadow tubers  color   soil preference height distance
#> 1    FALSE   TRUE   TRUE    red    wet         15     25       15
#> 2     TRUE  FALSE  FALSE yellow    dry          3    150       50
#> 3    FALSE   TRUE  FALSE   pink    wet          1    150       50
#> 4    FALSE  FALSE   TRUE    red normal         16    125       50
#> 5    FALSE   TRUE  FALSE   blue normal          2     20       15
#> 6    FALSE   TRUE  FALSE    red    wet         12     50       40
#> 7    FALSE  FALSE  FALSE    red    wet         13     40       20
#> 8    FALSE  FALSE   TRUE yellow normal          7    100       15
#> 9     TRUE   TRUE  FALSE   pink    dry          4     25       15
#> 10    TRUE   TRUE  FALSE   blue normal         14    100       60
#> 11    TRUE   TRUE   TRUE   blue    wet          8     45       10
#> 12    TRUE   TRUE   TRUE  white normal          9     90       25
#> 13    TRUE   TRUE  FALSE  white normal          6     20       10
#> 14    TRUE   TRUE   TRUE    red normal         11     80       30
#> 15    TRUE  FALSE  FALSE   pink normal         10     40       20
#> 16    TRUE  FALSE  FALSE    red normal         18    200       60
#> 17    TRUE  FALSE  FALSE yellow normal         17    150       60
#> 18   FALSE  FALSE   TRUE yellow    dry          5     25       10
#> 
#> flower> ## ==> example(daisy)  on how it is used
#> flower> 
#> flower> 
#> flower> 

summary(d0    <- daisy(flower))  # -> the first 3 {0,1} treated as *N*ominal
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1418  0.3904  0.4829  0.4865  0.5865  0.8875 
#> Metric :  mixed ;  Types = N, N, N, N, O, O, I, I 
#> Number of objects : 18
summary(dS123 <- daisy(flower,  type = list(symm = 1:3))) # first 3 treated as *S*ymmetric
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1418  0.3904  0.4829  0.4865  0.5865  0.8875 
#> Metric :  mixed ;  Types = S, S, S, N, O, O, I, I 
#> Number of objects : 18
stopifnot(dS123 == d0) # i.e.,  *S*ymmetric <==> *N*ominal {for 2-level factor}
summary(dNS123<- daisy(flowerN, type = list(symm = 1:3)))
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1418  0.3904  0.4829  0.4865  0.5865  0.8875 
#> Metric :  mixed ;  Types = S, S, S, N, O, O, I, I 
#> Number of objects : 18
stopifnot(dS123 == d0)
## by default, however ...
summary(dA123 <- daisy(flowerN)) # .. all 3 logicals treated *A*symmetric binary (w/ warning)
#> Warning: setting 'logical' variables 1, 2, 3 to type 'asymm'
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1592  0.4358  0.5341  0.5347  0.6291  0.8910 
#> Metric :  mixed ;  Types = A, A, A, N, O, O, I, I 
#> Number of objects : 18
summary(dA3  <- daisy(flower, type = list(asymm = 3)))
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1418  0.4164  0.5101  0.5098  0.6051  0.8875 
#> Metric :  mixed ;  Types = N, N, A, N, O, O, I, I 
#> Number of objects : 18
summary(dA13 <- daisy(flower, type = list(asymm = c(1, 3), ordratio = 7)))
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1647  0.4387  0.5265  0.5293  0.6252  0.9007 
#> Metric :  mixed ;  Types = A, N, A, N, O, O, T, I 
#> Number of objects : 18
## Mixing variable *names* and column numbers (failed in the past):
summary(dfl3 <- daisy(flower, type = list(asymm = c("V1", "V3"), symm= 2,
                                          ordratio= 7, logratio= 8)))
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1649  0.4378  0.5350  0.5288  0.6318  0.8972 
#> Metric :  mixed ;  Types = A, S, A, N, O, O, T, I 
#> Number of objects : 18

## If we'd treat the first 3 as simple {0,1}
Nflow <- flower
Nflow[,1:3] <- lapply(flower[,1:3], function(f) as.integer(as.character(f)))
summary(dN <- daisy(Nflow)) # w/ warning: treated binary .. 1:3 as interval
#> Warning: binary variable(s) 1, 2, 3 treated as interval scaled
#> 153 dissimilarities, summarized :
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>  0.1418  0.3904  0.4829  0.4865  0.5865  0.8875 
#> Metric :  mixed ;  Types = I, I, I, N, O, O, I, I 
#> Number of objects : 18
## Still, using Euclidean/Manhattan distance for {0-1} *is* identical to treating them as "N" :
stopifnot(dN == d0)
stopifnot(dN == daisy(Nflow, type = list(symm = 1:3))) # or as "S"