| Title: | Distance-Based Learning for Mixed-Type Data |
|---|---|
| Description: | Provides tools for constructing, computing, and using distance measures for numerical, categorical, and mixed-type data. The package implements a flexible framework in which continuous and categorical components can be combined under additive, commensurable, and association-aware specifications. Supported methods include classical distances such as Gower, Euclidean, Manhattan, and Mahalanobis-type distances; categorical dissimilarities such as simple matching, occurrence-frequency, and association-based measures; and mixed-type presets designed to reduce biases due to variable type, scale, distribution, redundancy, and number of categories. The package also provides scaling options, supervised and unsupervised distance constructions, leave-one-variable-out tools for distance-based variable importance, and integration with distance-based learning workflows such as nearest-neighbour prediction, partitioning around medoids, and spectral clustering. Methods are motivated by van de Velden, Iodice D'Enza, Markos, and Cavicchia (2026) <doi:10.1080/10618600.2026.2680181> and related work on categorical and mixed-type dissimilarities. |
| Authors: | Alfonso Iodice D'Enza [aut, cre], Angelos Markos [aut], Michel van de Velden [aut], Carlo Cavicchia [aut] |
| Maintainer: | Alfonso Iodice D'Enza <[email protected]> |
| License: | GPL-3 |
| Version: | 0.5.2 |
| Built: | 2026-10-08 16:52:46 UTC |
| Source: | https://github.com/alfonsoiodicede/manydist_package |
Construct a tibble of valid method specifications that can be passed to [benchmark_mdist()] or iterated over manually to benchmark [mdist()] across multiple configurations.
all_dist_method_specs( mode = c("full", "presets_only", "response_aware_only"), method_cat = NULL, method_num = NULL, preset = NULL, commensurable = NULL )all_dist_method_specs( mode = c("full", "presets_only", "response_aware_only"), method_cat = NULL, method_num = NULL, preset = NULL, commensurable = NULL )
mode |
Character string controlling the initial pool of specifications. Supported values are '"full"', '"presets_only"', and '"response_aware_only"'. |
method_cat |
Optional character vector restricting the categorical methods used in custom specifications. |
method_num |
Optional character vector restricting the numerical preprocessing methods used in custom specifications. |
preset |
Optional character vector restricting preset-based specifications. |
commensurable |
Optional logical vector restricting the commensurable values used in custom specifications. |
The resulting tibble combines preset-based specifications and custom custom specifications built from the currently available categorical methods and numerical preprocessing options listed in [dist_methods_tbl()].
Rows with 'preset = "custom"' use the explicit categorical, numerical, and commensurability options. Named presets determine those settings internally.
'mode' defines the initial candidate pool. Any explicit argument filters supplied by the user are applied afterwards and therefore restrict the selected pool further.
With 'mode = "response_aware_only"', preset specifications and categorical methods are restricted to response-aware methods.
A tibble where each row represents one valid 'mdist()' specification. The tibble contains 'preset', 'method_cat', 'method_num', and 'commensurable'.
all_dist_method_specs() all_dist_method_specs(mode = "presets_only") all_dist_method_specs(mode = "response_aware_only") all_dist_method_specs(mode = "full", method_cat = c("tvd", "le_and_ho"))all_dist_method_specs() all_dist_method_specs(mode = "presets_only") all_dist_method_specs(mode = "response_aware_only") all_dist_method_specs(mode = "full", method_cat = c("tvd", "le_and_ho"))
Draws an annotated triangular heatmap of the pairwise diagnostics from [benchmark_mdist()].
## S3 method for class 'MDistBenchmark' autoplot( object, metric = c("relative_distance", "mad", "alienation", "mds_congruence", "ari"), cluster_method = NULL, digits = 2, ... )## S3 method for class 'MDistBenchmark' autoplot( object, metric = c("relative_distance", "mad", "alienation", "mds_congruence", "ari"), cluster_method = NULL, digits = 2, ... )
object |
An object returned by [benchmark_mdist()]. |
metric |
Character string selecting '"mad"', '"relative_distance"', '"mds_congruence"', '"alienation"', or '"ari"'. A specific ARI column, such as '"ari_pam"', can also be supplied. |
cluster_method |
Optional clustering method used when 'metric = "ari"'. If 'NULL', all available ARI metrics are shown, with one facet per clustering method. |
digits |
Number of decimal places used for cell labels. |
... |
Currently unused. |
The default fill gradient runs from coral ('#E76F51') for lower values to blue ('#008CFF') for higher values, with regular-weight white cell labels. To use another palette, add a fill scale to the returned plot. Self-comparisons on the main diagonal are left blank. ARI plots use fixed colour limits of 0 and 1, so the same ARI has the same colour across plots. Negative ARIs retain their numerical labels but use the low-end colour.
A 'ggplot' object.
Applies [mdist()] repeatedly over a tibble of distance-method specifications, typically generated with [all_dist_method_specs()]. It then compares every pair of successful distance specifications using distance, configuration, and optional clustering diagnostics.
benchmark_mdist( x, response = NULL, specs = all_dist_method_specs(), dims = 2, cluster_k = NULL, cluster_methods = c("pam", "hclust", "spectral"), hclust_method = "average", spectral_sigma = NULL, spectral_nstart = 50 )benchmark_mdist( x, response = NULL, specs = all_dist_method_specs(), dims = 2, cluster_k = NULL, cluster_methods = c("pam", "hclust", "spectral"), hclust_method = "average", spectral_sigma = NULL, spectral_nstart = 50 )
x |
A data frame or tibble of predictors, optionally including the response column. |
response |
Optional response column inside 'x', supplied either unquoted or as a character string. |
specs |
A tibble of method specifications. By default, this is generated with [all_dist_method_specs()]. It must contain the columns 'preset', 'method_cat', 'method_num', and 'commensurable'. An optional 'label' column supplies display labels for comparisons and plots. |
dims |
Integer. Number of dimensions used by classical multidimensional scaling when computing congruence and alienation coefficients. |
cluster_k |
Optional integer. Number of clusters used for pairwise adjusted Rand indices. If 'NULL', no clustering is performed. |
cluster_methods |
Character vector specifying the clustering methods used when 'cluster_k' is supplied. Possible values are '"pam"', '"hclust"', and '"spectral"'. |
hclust_method |
Character string specifying the linkage method passed to [stats::hclust()] when '"hclust"' is requested. |
spectral_sigma |
Optional numeric value for the Gaussian affinity bandwidth used by spectral clustering. If 'NULL', the default used by [spectral_dist()] is applied. |
spectral_nstart |
Integer. Number of random starts used by the k-means step in spectral clustering. |
Each row of 'specs' is interpreted as one valid 'mdist()' configuration. Preset-based and custom specifications are both supported. Failed specifications are caught and returned in the output rather than stopping the full benchmark.
Preset specifications use the 'preset' column and ignore 'method_cat', 'method_num', and 'commensurable'. Component specifications are evaluated as 'preset = "custom"' and use 'method_cat', 'method_num', and 'commensurable'.
Pairwise mean absolute difference ('mad') is computed from the lower triangle of each dissimilarity matrix. The symmetric relative distance is defined as
Classical multidimensional scaling configurations are compared using
[congruence_coeff()]. The corresponding alienation coefficient is
.
When 'cluster_k' is supplied, each requested clustering method is applied once to every successful distance specification. Partitions produced by the same clustering method are then compared pairwise using the adjusted Rand index. When 'cluster_k = NULL', no clustering is performed and no ARI columns are included in the comparisons.
An object of class '"MDistBenchmark"', which is also a tibble. It contains the supplied specifications together with:
The corresponding output of [mdist()], or an error object if the specification failed.
Logical indicator; 'TRUE' if the run completed successfully, 'FALSE' otherwise.
Error message for failed runs, 'NA' otherwise.
Use 'summary()' to display and obtain the complete pairwise-diagnostics tibble, and [ggplot2::autoplot()] to draw an annotated triangular heatmap. The default print method displays a compact run overview; use 'tibble::as_tibble()' to inspect the complete underlying run table.
if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() specs <- all_dist_method_specs( mode = "presets_only", preset = c("gower", "u_indep", "u_dep") ) res <- benchmark_mdist( penguins_small, response = species, specs = specs ) res pairwise_results <- summary(res) ggplot2::autoplot(res, metric = "relative_distance") }if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() specs <- all_dist_method_specs( mode = "presets_only", preset = c("gower", "u_indep", "u_dep") ) res <- benchmark_mdist( penguins_small, response = species, specs = specs ) res pairwise_results <- summary(res) ggplot2::autoplot(res, metric = "relative_distance") }
This function compares **leave-one-variable-out (LOVO)** diagnostics
across multiple distance definitions supported by manydist.
compare_lovo_mdist( x, methods, mds = FALSE, dims = 2, keep_dist = FALSE, .progress = FALSE, ... )compare_lovo_mdist( x, methods, mds = FALSE, dims = 2, keep_dist = FALSE, .progress = FALSE, ... )
x |
A data frame or tibble containing the predictors. |
methods |
A **named list** describing the distance specifications
to compare. Each element must be a list of arguments passed to
For example:
methods = list(
gower = list(preset = "gower"),
u_dep = list(preset = "unbiased_dependent"),
custom = list(
preset = "custom",
method_cat = "matching",
method_num = "std",
commensurable = TRUE
)
)
|
mds |
Logical. If 'TRUE', compute MDS-based congruence and alienation diagnostics for every distance specification. The default is 'FALSE'. |
dims |
Number of dimensions used for the MDS configuration when 'mds = TRUE'. Ignored when 'mds = FALSE'. |
keep_dist |
Logical; if |
.progress |
Logical; if |
... |
Additional arguments passed to These may include optional clustering diagnostics, for example
|
For each distance specification, the function:
Computes the full mixed-type distance using mdist().
Recomputes the distance repeatedly leaving out one variable at a time.
Measures the impact of each variable using mean absolute deviation (MAD) and, when requested, MDS- or clustering-based diagnostics.
The results are combined across methods and returned as an
MDistLOVOCompare object, which supports
print(), summary(), and ggplot2::autoplot().
In autoplot(), reorder = TRUE ranks variables by the selected
metric averaged across methods, separately within each variable type.
Categorical variables precede numerical variables, preserving the blue/red
background bands. top_n still selects variables globally before
ordering. If type information is absent, ordering is global.
An object of class MDistLOVOCompare containing:
A tibble with one row per method-variable combination.
The list of distance specifications used.
Whether MDS diagnostics were computed.
Number of MDS dimensions used, or 'NULL' when 'mds = FALSE'.
Number of observations in the dataset.
## Not run: library(manydist) library(palmerpenguins) data <- penguins |> dplyr::select(-species) |> tidyr::drop_na() cmp <- compare_lovo_mdist( x = data, methods = list( gower = list(preset = "gower"), u_dep = list(preset = "unbiased_dependent") ), cluster_k = 3 ) summary(cmp) autoplot(cmp, metric = "mad_importance") autoplot(cmp, metric = "pam_importance") ## End(Not run)## Not run: library(manydist) library(palmerpenguins) data <- penguins |> dplyr::select(-species) |> tidyr::drop_na() cmp <- compare_lovo_mdist( x = data, methods = list( gower = list(preset = "gower"), u_dep = list(preset = "unbiased_dependent") ), cluster_k = 3 ) summary(cmp) autoplot(cmp, metric = "mad_importance") autoplot(cmp, metric = "pam_importance") ## End(Not run)
Computes the congruence coefficient between two data configurations using the Frobenius inner product of their pairwise distance matrices.
congruence_coeff(L1, L2)congruence_coeff(L1, L2)
L1 |
A numeric matrix or data frame (rows = observations) |
L2 |
A numeric matrix or data frame with the same number of rows as |
The congruence coefficient is defined as
where and are the pairwise distance matrices derived from
L1 and L2.
A scalar in measuring similarity between the two configurations.
Returns a tibble describing the categorical methods, numerical preprocessing options, and preset specifications currently available in 'manydist'.
dist_methods_tbl()dist_methods_tbl()
The table is used by helpers such as [all_dist_method_specs()] and [benchmark_mdist()] to construct valid 'mdist()' specifications.
A tibble with one row per available method. The columns are:
Name of the method or preset.
The 'mdist()' argument to which the method belongs: '"method_cat"', '"method_num"', or '"preset"'.
Type of variables targeted by the method.
Broad methodological family.
Logical; whether the method can use a response variable.
Implementation source.
dist_methods_tbl() dist_methods_tbl() |> dplyr::count(argument)dist_methods_tbl() dist_methods_tbl() |> dplyr::count(argument)
Convenience wrapper around [dist_methods_tbl()] returning only methods that can be used as categorical dissimilarities through 'method_cat'.
dist_methods_tbl_cat()dist_methods_tbl_cat()
A tibble containing the rows of [dist_methods_tbl()] with 'argument == "method_cat"'.
dist_methods_tbl_cat()dist_methods_tbl_cat()
The fifa_nl dataset contains information on players in the Dutch League from the FIFA 21 video game. This dataset includes various attributes of players, such as demographics, club details, skill ratings, and physical characteristics.
data("fifa_nl")data("fifa_nl")
A data frame with observations on various attributes describing the players.
player_positionsPrimary playing positions of the player.
nationalityThe country the player represents.
team_positionPlayer's assigned position within their club.
club_nameName of the club the player is part of.
work_rateThe player's work rate, describing defensive and attacking intensity.
weak_footSkill rating for the player's non-dominant foot, ranging from 1 to 5.
skill_movesSkill moves rating, indicating technical skill and ability to perform complex moves, on a scale of 1 to 5.
international_reputationPlayer's reputation on an international scale, from 1=local to 3=global star.
body_typeBody type of the player ( Lean, Normal, Stocky.
preferred_footDominant foot of the player, either Left or Right.
ageAge of the player in years.
height_cmHeight of the player in centimeters.
weight_kgWeight of the player in kilograms.
overallOverall skill rating of the player out of 100.
potentialPotential skill rating the player may achieve in the future.
value_eurEstimated market value of the player in Euros.
wage_eurPlayer's weekly wage in Euros.
release_clause_eurRelease clause value in Euros, which other clubs must pay to buy out the player's contract.
paceSpeed rating of the player out of 100.
shootingShooting skill rating out of 100.
passingPassing skill rating out of 100.
dribblingDribbling skill rating out of 100.
defendingDefending skill rating out of 100.
physicPhysicality rating out of 100.
This dataset provides a snapshot of player attributes and performance indicators as represented in FIFA 21 for players in the Dutch League. It can be used to analyze player characteristics, compare skills across players, and explore potential relationships among variables such as age, position, and various skill ratings.
Stefano Leone. (2021). FIFA 21 Complete Player Dataset. Retrieved from https://www.kaggle.com/datasets/stefanoleone992/fifa-21-complete-player-dataset.
data(fifa_nl) summary(fifa_nl)data(fifa_nl) summary(fifa_nl)
Generates synthetic mixed datasets with controllable numerical and categorical signal/noise structure and balanced cluster sizes.
gen_mixed( k_true, clustSizeEq = 50, numsignal = 2, numnoise = 2, catsignal = 2, catnoise = 2, q = 5, q_err = 9, numsep = 0.1, catsep = 0.5, seed = NULL, error_type = c("normal", "chisq"), error_df = 2, error_scale = 1 )gen_mixed( k_true, clustSizeEq = 50, numsignal = 2, numnoise = 2, catsignal = 2, catnoise = 2, q = 5, q_err = 9, numsep = 0.1, catsep = 0.5, seed = NULL, error_type = c("normal", "chisq"), error_df = 2, error_scale = 1 )
k_true |
Number of clusters |
clustSizeEq |
Observations per cluster |
numsignal |
Number of numerical signal variables |
numnoise |
Number of numerical noise variables |
catsignal |
Number of categorical signal variables |
catnoise |
Number of categorical noise variables |
q |
Number of categories for signal categorical variables |
q_err |
Number of categories for categorical noise |
numsep |
Separation for numerical signal |
catsep |
Separation for categorical signal |
seed |
Optional seed |
error_type |
Error distribution ("normal" or "chisq") |
error_df |
Degrees of freedom for chi-square noise |
error_scale |
Scale of noise |
A list with elements df, X_num, X_cat, y,
num_cols, cat_cols
generate toy mixed datasets for the supplementary material: figures 1 to 3
generate_dataset( n, porig, pn, pnnoise, pcnoise, sigma, qoptions, seed = NULL, mode = "per_variable" )generate_dataset( n, porig, pn, pnnoise, pcnoise, sigma, qoptions, seed = NULL, mode = "per_variable" )
n |
number of observations |
porig |
number of original informative continuous variables |
pn |
number of continuous variables (total, before adding noise variables) |
pnnoise |
number of extra numeric noise variables |
pcnoise |
number of extra categorical noise variables |
sigma |
sd for noise added to informative variables |
qoptions |
number of bins for categorization (vector if per_variable, scalar if shared) |
seed |
optional seed |
mode |
either "per_variable" or "shared" |
Predict outcomes from precomputed test-to-training distances, or compute those distances from tabular training and test predictors before predicting.
knn_dist( x, y = NULL, new_data = NULL, k = 5, dist_fun = NULL, dist_args = list(), type = c("class", "prob", "numeric"), response = NULL )knn_dist( x, y = NULL, new_data = NULL, k = 5, dist_fun = NULL, dist_args = list(), type = c("class", "prob", "numeric"), response = NULL )
x |
With |
y |
Training outcomes, in the same order as the distance columns or
training rows. Use a factor for classification and numeric outcomes for
regression. Missing outcomes are not supported. May be omitted when
|
new_data |
Test predictors when |
k |
Integer number of neighbours, between one and the number of training observations. |
dist_fun |
Optional function accepting training predictors as its first
argument and test predictors as |
dist_args |
Named list of additional arguments passed to |
type |
Prediction type: |
response |
Optional outcome column name (a string or unquoted column
name) in tabular |
Each distance row corresponds to a test observation and each column to a training observation. Distances must be finite and nonnegative. Classification uses an unweighted majority vote; regression uses the mean of the neighbours' outcomes. Distance ties are resolved in training order and class-vote ties in factor-level order. Class probabilities are neighbour class proportions.
No training-to-training matrix is required. With dist_fun = mdist, distance
construction uses x as training data and new_data as test data. The
function computes distances on each call; it does not return a fitted model.
Other distance functions are responsible for their own preprocessing rules.
For fitted tidymodels workflows, use nearest_neighbor_dist() with
step_mdist() instead.
A factor for type = "class", a data frame with one column per
outcome level for type = "prob", or a numeric vector for
type = "numeric". Each result has one row or element per test observation.
mdist(), nearest_neighbor_dist(), step_mdist()
train <- data.frame(value = c(0, 2, 8, 10)) test <- data.frame(value = c(1, 9)) labels <- factor(c("low", "low", "high", "high")) d <- mdist(train, new_data = test, preset = "gower") knn_dist(d, labels, k = 1) knn_dist(train, labels, new_data = test, k = 1, dist_fun = mdist, dist_args = list(preset = "gower")) train$label <- labels knn_dist(train, response = "label", new_data = test, k = 1, dist_fun = mdist, dist_args = list(preset = "gower")) knn_dist(d, labels, k = 2, type = "prob") knn_dist(d, c(0, 2, 8, 10), k = 2, type = "numeric")train <- data.frame(value = c(0, 2, 8, 10)) test <- data.frame(value = c(1, 9)) labels <- factor(c("low", "low", "high", "high")) d <- mdist(train, new_data = test, preset = "gower") knn_dist(d, labels, k = 1) knn_dist(train, labels, new_data = test, k = 1, dist_fun = mdist, dist_args = list(preset = "gower")) train$label <- labels knn_dist(train, response = "label", new_data = test, k = 1, dist_fun = mdist, dist_args = list(preset = "gower")) knn_dist(d, labels, k = 2, type = "prob") knn_dist(d, c(0, 2, 8, 10), k = 2, type = "numeric")
Computes leave-one-variable-out (LOVO) diagnostics for a distance specification. The function first computes the full dissimilarity matrix using [mdist()]. It then removes one predictor at a time, recomputes the dissimilarity matrix, and compares each leave-one-variable-out matrix with the full one.
lovo_mdist( x, response = NULL, ..., mds = FALSE, dims = 2, keep_dist = FALSE, cluster_k = NULL, cluster_methods = c("pam", "hclust", "spectral"), hclust_method = "average", spectral_sigma = NULL, spectral_nstart = 50, response_used = TRUE )lovo_mdist( x, response = NULL, ..., mds = FALSE, dims = 2, keep_dist = FALSE, cluster_k = NULL, cluster_methods = c("pam", "hclust", "spectral"), hclust_method = "average", spectral_sigma = NULL, spectral_nstart = 50, response_used = TRUE )
x |
A data frame or object coercible to a tibble. Rows are observations and columns are variables used to compute the dissimilarity. |
response |
Optional response variable. It can be supplied as an unquoted column name or as a character string. When supplied and 'response_used = TRUE', it is passed to [mdist()] for response-aware distance construction. The response column is not treated as a predictor in the leave-one-variable-out loop. |
... |
Additional arguments passed to [mdist()], such as 'preset', 'method_cat', 'method_num', 'commensurable', or 'interaction'. |
mds |
Logical. If 'TRUE', compute classical multidimensional scaling diagnostics for the full and leave-one-variable-out dissimilarities. The default is 'FALSE'. |
dims |
Integer. Number of dimensions used by classical multidimensional scaling when 'mds = TRUE'. Ignored when 'mds = FALSE'. |
keep_dist |
Logical. If 'TRUE', store the full dissimilarity matrix and all leave-one-variable-out dissimilarity matrices in the returned object. |
cluster_k |
Optional integer. Number of clusters used when computing clustering-based LOVO diagnostics. If 'NULL', clustering diagnostics are not computed. |
cluster_methods |
Character vector specifying the clustering methods used for clustering-based diagnostics. Possible values are '"pam"', '"hclust"', and '"spectral"'. |
hclust_method |
Character string specifying the linkage method passed to [stats::hclust()] when '"hclust"' is included in 'cluster_methods'. |
spectral_sigma |
Optional numeric value for the Gaussian affinity bandwidth used by spectral clustering. If 'NULL', the default used by [spectral_dist()] is applied. |
spectral_nstart |
Integer. Number of random starts used by the k-means step in spectral clustering. |
response_used |
Logical. If 'TRUE', the response variable, when supplied, is used in the distance construction. If 'FALSE', the response column is removed before computing distances. |
'lovo_mdist()' is useful for assessing how strongly each predictor contributes to a distance-based representation. A predictor is considered influential when removing it produces a large change in the dissimilarity matrix or an optional downstream representation, such as a multidimensional scaling configuration or clustering partition.
In 'autoplot()', 'reorder = TRUE' ranks variables by the selected metric within each variable type, with categorical variables before numerical variables. This preserves the blue/red background bands. 'top_n' selection remains global; without type information, ordering is global.
The returned object contains several LOVO diagnostics. The main distance contribution is measured by the mean absolute difference between the full dissimilarity matrix and each leave-one-variable-out matrix ('mad_importance'). The normalized version is stored as 'relative_distance'.
If 'mds = TRUE', the function also compares classical multidimensional scaling configurations computed from the full and leave-one-variable-out dissimilarities. These diagnostics are stored as 'mds_congruence' / 'cc_importance' and 'ac_importance', the latter corresponding to an alienation coefficient. They are omitted when 'mds = FALSE'.
If 'cluster_k' is supplied, the function additionally computes clustering partitions from the full and leave-one-variable-out dissimilarities and compares them using the adjusted Rand index. The corresponding importance measures are defined as '1 - ARI' and are stored as 'pam_importance', 'hclust_importance', or 'spectral_importance', depending on the selected clustering methods.
Clustering-based diagnostics require the suggested package 'mclust'.
An object of class '"MDistLOVO"'. The main results are stored in the '$results' field as a tibble with one row per left-out variable. The object also has print, summary, and autoplot methods.
[mdist()], [compare_lovo_mdist()], [spectral_dist()]
if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() # Distance-based LOVO diagnostics for a Gower distance res <- lovo_mdist( penguins_small, preset = "gower", response = species, response_used = FALSE ) res summary(res) # Plot the relative distance contribution of each predictor p <- res$autoplot(metric = "relative_distance") p }if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() # Distance-based LOVO diagnostics for a Gower distance res <- lovo_mdist( penguins_small, preset = "gower", response = species, response_used = FALSE ) res summary(res) # Plot the relative distance contribution of each predictor p <- res$autoplot(metric = "relative_distance") p }
Helper to build one method specification for [compare_lovo_mdist()]. This allows tidy-style specification of 'response', e.g. 'response = Name', while storing the method definition as a regular list.
lovo_method_spec(response = NULL, ...)lovo_method_spec(response = NULL, ...)
response |
Optional response column, supplied either unquoted (e.g. 'Name') or quoted (e.g. '"Name"'). |
... |
Additional arguments passed on to [lovo_mdist()] through [compare_lovo_mdist()]. |
A named list of arguments suitable for one element of the 'methods' argument in [compare_lovo_mdist()].
## Not run: methods <- list( tvd_sup = lovo_method_spec( response = species, preset = "custom", method_cat = "tvd", method_num = "std", commensurable = TRUE, response_used = TRUE ), tvd_unsup = lovo_method_spec( response = species, preset = "custom", method_cat = "tvd", method_num = "std", commensurable = TRUE, response_used = FALSE ) ) ## End(Not run)## Not run: methods <- list( tvd_sup = lovo_method_spec( response = species, preset = "custom", method_cat = "tvd", method_num = "std", commensurable = TRUE, response_used = TRUE ), tvd_unsup = lovo_method_spec( response = species, preset = "custom", method_cat = "tvd", method_num = "std", commensurable = TRUE, response_used = FALSE ) ) ## End(Not run)
This helper creates a tidymodels recipe using step_mdist(). It supports both mdist presets and custom param sets.
make_mdist_recipe(df, mdist_type, mdist_preset, param_set, outcome)make_mdist_recipe(df, mdist_type, mdist_preset, param_set, outcome)
df |
A data frame. |
mdist_type |
"preset" or "custom". |
mdist_preset |
Name of the preset (if mdist_type == "preset"). |
param_set |
A list of custom mdist arguments (if mdist_type == "custom"). |
outcome |
Name of the outcome variable. |
A 'recipes::recipe()' object.
Computes a dissimilarity object for numerical, categorical, or mixed-type data. The function combines continuous and categorical components according to either a predefined 'preset' or a user-defined custom specification.
mdist( x, new_data = NULL, response = NULL, method_cat = "tvd", method_num = "std", commensurable = TRUE, ncomp = NULL, threshold = NULL, preset = "custom", interaction = FALSE, prop_nn = 0.1, score = "ba", decision = "prior_corrected", gower_average = TRUE )mdist( x, new_data = NULL, response = NULL, method_cat = "tvd", method_num = "std", commensurable = TRUE, ncomp = NULL, threshold = NULL, preset = "custom", interaction = FALSE, prop_nn = 0.1, score = "ba", decision = "prior_corrected", gower_average = TRUE )
x |
A data frame or matrix containing the training observations. Columns can be numeric, factors, or a mixture of both. |
new_data |
Optional data frame or matrix containing new observations. If supplied, distances are computed from rows of 'new_data' to rows of 'x', producing a rectangular test-to-training dissimilarity matrix. |
response |
Optional response variable used for response-aware categorical dissimilarities. It can be supplied as an unquoted column name or as a character string. The response column is removed from the predictors before computing distances. |
method_cat |
Character string specifying the categorical-variable dissimilarity used when 'preset = "custom"'. Common values include '"matching"' and '"tvd"'. Use [all_dist_method_specs()] to inspect available methods. |
method_num |
Character string specifying numerical-variable preprocessing. Supported values are '"none"' for no preprocessing, '"std"' for standard-deviation scaling, '"range"' for range scaling, '"robust"' for inter-quartile-range-based scaling, and '"pc_scores"' for principal-component score scaling. This argument is used directly when 'preset = "custom"'. When 'preset = "euclidean"' and all predictors are numeric, it can also override the default '"std"' preprocessing; in particular, '"none"' gives ordinary Euclidean distance on the original variables. |
commensurable |
Logical. If 'TRUE', dissimilarities are scaled so that the average contribution of each variable to the overall distance is equal to 1 on the training data. Variable-specific means are computed from training pairs, before aggregation, and never from 'new_data'. The current variable-wise convention includes the zero diagonal of the training square. |
ncomp |
Integer or 'NULL'. Number of principal components to retain when 'method_num = "pc_scores"' or 'preset = "u_dep_bw"'. If 'NULL', all available components are used unless 'threshold' is supplied and supported by the underlying method. |
threshold |
Numeric or 'NULL'. Optional cumulative variance threshold used when 'method_num = "pc_scores"' or 'preset = "u_dep_bw"'. |
preset |
Character string specifying a predefined distance specification. Available values include '"custom"', '"gower"', '"unbiased_dependent"', '"u_dep"', '"u_dep_bw"', '"u_indep"', '"u_mix"', '"hl"', '"gudmm"', '"dkss"', '"mod_gower"', and '"euclidean"'. When 'preset' is not '"custom"', arguments such as 'method_cat', 'method_num', 'commensurable', and 'interaction' are normally handled by the preset and user-supplied values are ignored. The exception is 'method_num' for 'preset = "euclidean"' when all predictors are numeric. |
interaction |
Logical. If 'TRUE', adds an interaction-aware continuous-categorical component based on local predictive separability. |
prop_nn |
Numeric. Proportion of nearest neighbours used when 'interaction = TRUE'. |
score |
Character string specifying the score used when 'interaction = TRUE'. Available values include '"ba"' for balanced accuracy and '"logloss"'. |
decision |
Character string specifying the decision rule used when 'score = "ba"'. The default is '"prior_corrected"'. |
gower_average |
Logical; only used when 'preset = "gower"'. If 'TRUE', returns the standard Gower dissimilarity averaged over variables, matching the scale of [cluster::daisy()] with 'metric = "gower"'. If 'FALSE', returns the sum of per-variable Gower contributions, equivalent to multiplying the averaged Gower dissimilarity by the number of active variables. |
'mdist()' is the main distance-construction function in 'manydist'. It can return ordinary train-train dissimilarities or rectangular test-to-training dissimilarities when 'new_data' is supplied. The resulting object stores both the dissimilarity matrix and metadata about the distance specification that was used.
With 'preset = "custom"', users manually choose the numerical preprocessing, categorical dissimilarity, commensurability, and optional interaction term.
The '"gower"' preset follows the usual Gower construction based on range scaling for continuous variables and matching dissimilarities for categorical variables. The 'gower_average' argument controls whether the result is averaged over variables or returned as a sum of variable-wise contributions.
The '"u_dep"', '"unbiased_dependent"', '"u_indep"', and '"u_mix"' presets are convenience specifications for unbiased or commensurable mixed-variable dissimilarities. The '"u_dep_bw"' preset uses the same categorical construction as '"u_dep"', but whitens the retained numerical principal components and scales their complete Manhattan distance by the number of original numerical variables divided by its mean over distinct training pairs. The '"euclidean"' preset computes Euclidean distance after standardizing numerical variables and one-hot encoding and standardizing categorical variables. For numerical-only inputs, '"std"' remains the default, but 'method_num' can be overridden. Setting 'method_num = "none"' produces the same numerical distances as [stats::dist()] applied directly to the original variables. The '"gudmm"', '"dkss"', and '"mod_gower"' presets provide additional mixed-type distance constructions. Some presets currently support only train-train distances and will stop if 'new_data' is supplied.
Use [all_dist_method_specs()] to inspect the available distance components and method specifications.
An object of class '"MDist"'. The object contains the computed dissimilarity in its '$distance' field, the selected 'preset', the training data, and a list of parameters describing the fitted distance specification. Square train-train dissimilarities are stored as '"dissimilarity"'/'"dist"' objects; rectangular test-to-training dissimilarities are stored as '"dissimilarity"'/'"matrix"' objects.
[step_mdist()], [all_dist_method_specs()]
if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, species, island, sex ) |> tidyr::drop_na() # Gower distance on mixed-type data d_gower <- mdist(penguins_small, preset = "gower") d_gower # Custom mixed-type specification d_custom <- mdist( penguins_small, preset = "custom", method_cat = "matching", method_num = "std", commensurable = TRUE ) d_custom # Train-to-new-data distances penguin_split <- rsample::initial_split(penguins_small, prop = 0.75) penguin_train <- rsample::training(penguin_split) penguin_test <- rsample::testing(penguin_split) d_new <- mdist( penguin_train, new_data = penguin_test, preset = "gower" ) d_new }if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, species, island, sex ) |> tidyr::drop_na() # Gower distance on mixed-type data d_gower <- mdist(penguins_small, preset = "gower") d_gower # Custom mixed-type specification d_custom <- mdist( penguins_small, preset = "custom", method_cat = "matching", method_num = "std", commensurable = TRUE ) d_custom # Train-to-new-data distances penguin_split <- rsample::initial_split(penguins_small, prop = 0.75) penguin_train <- rsample::training(penguin_split) penguin_test <- rsample::testing(penguin_split) d_new <- mdist( penguin_train, new_data = penguin_test, preset = "gower" ) d_new }
Internal: summary method implementation for MDist R6 objects
mdist_summary_impl(object, ...)mdist_summary_impl(object, ...)
object |
An 'MDist' object to summarize. |
... |
Additional arguments currently not used. |
Invisibly returns 'object'.
PAM clustering specification based on manydist dissimilarities
pam_dist(num_clusters = NULL)pam_dist(num_clusters = NULL)
num_clusters |
Number of clusters. |
A 'pam_dist_spec' object.
Returns the methods in [dist_methods_tbl()] that can use a response variable. The output can optionally be restricted by data type or by 'mdist()' argument.
response_aware_methods(data_type = NULL, argument = NULL)response_aware_methods(data_type = NULL, argument = NULL)
data_type |
Optional character vector used to restrict the returned methods by data type, for example '"categorical"' or '"mixed"'. |
argument |
Optional character vector used to restrict the returned methods by argument, for example '"method_cat"' or '"preset"'. |
A character vector of response-aware method names.
response_aware_methods() response_aware_methods(argument = "method_cat") response_aware_methods(argument = "preset")response_aware_methods() response_aware_methods(argument = "method_cat") response_aware_methods(argument = "preset")
Spectral clustering specification based on manydist dissimilarities
spectral_dist(num_clusters = NULL, sigma = NULL, nstart = 50)spectral_dist(num_clusters = NULL, sigma = NULL, nstart = 50)
num_clusters |
Number of clusters. |
sigma |
Optional bandwidth for the Gaussian affinity. If 'NULL', the median pairwise distance is used. |
nstart |
Number of random starts for k-means. |
A 'spectral_dist_spec' object.
## Not run: library(manydist) library(palmerpenguins) library(recipes) library(generics) data <- penguins |> dplyr::select(-species) |> tidyr::drop_na() rec <- recipes::recipe(~ ., data = data) |> step_mdist(all_predictors(), preset = "gower", output = "pairwise") spec <- spectral_dist(num_clusters = 3) fit_obj <- generics::fit(spec, recipe = rec, data = data) print(fit_obj) predict(fit_obj) predict(fit_obj, type = "embed") ## End(Not run)## Not run: library(manydist) library(palmerpenguins) library(recipes) library(generics) data <- penguins |> dplyr::select(-species) |> tidyr::drop_na() rec <- recipes::recipe(~ ., data = data) |> step_mdist(all_predictors(), preset = "gower", output = "pairwise") spec <- spectral_dist(num_clusters = 3) fit_obj <- generics::fit(spec, recipe = rec, data = data) print(fit_obj) predict(fit_obj) predict(fit_obj, type = "embed") ## End(Not run)
Spectral clustering from a distance matrix
spectral_from_dist(D, k, affinity_method = "selftune")spectral_from_dist(D, k, affinity_method = "selftune")
D |
A distance matrix |
k |
Number of clusters |
affinity_method |
Method to build affinity |
A vector of cluster labels
'step_mdist()' is a [recipes::recipe()] step that replaces selected predictors by a distance-based representation computed with [mdist()]. It is designed for distance-based learning workflows, especially nearest-neighbour prediction and clustering models that operate on dissimilarity matrices.
step_mdist( recipe, ..., role = "predictor", trained = FALSE, output = "distance_to_training", response_used = TRUE, preset = "custom", method_cat = "tot_var_dist", method_num = NULL, commensurable = FALSE, ncomp = NULL, threshold = NULL, columns = NULL, response_col = NULL, train_predictors = NULL, preprocessor = NULL, skip = FALSE, id = recipes::rand_id("mdist") )step_mdist( recipe, ..., role = "predictor", trained = FALSE, output = "distance_to_training", response_used = TRUE, preset = "custom", method_cat = "tot_var_dist", method_num = NULL, commensurable = FALSE, ncomp = NULL, threshold = NULL, columns = NULL, response_col = NULL, train_predictors = NULL, preprocessor = NULL, skip = FALSE, id = recipes::rand_id("mdist") )
recipe |
A recipe object. |
... |
Selector(s) for the predictor columns to be used in [mdist()]. These are passed to [recipes::recipes_eval_select()] during preparation. |
role |
Role for the new distance columns. The default is '"predictor"'. |
trained |
Logical for recipes internals. Do not set manually. |
output |
Character string specifying the type of distance output. '"distance_to_training"' returns distances from the baked data to the training observations and is the usual choice for prediction workflows. '"pairwise"' returns the within-training pairwise dissimilarity matrix and is intended for training-only distance-based clustering workflows. |
response_used |
Logical. If 'TRUE' (the default) and the recipe has exactly one outcome, response-aware distance specifications use that outcome when the step is prepared. Set to 'FALSE' to construct the distance from predictors only. Specifications that are not response-aware never use the outcome. |
preset |
Character string specifying the distance preset passed to [mdist()]. Available values include '"custom"', '"gower"', '"unbiased_dependent"', '"u_dep"', '"u_dep_bw"', '"u_indep"', '"u_mix"', '"hl"', '"gudmm"', '"dkss"', '"mod_gower"', and '"euclidean"'. Preset parameters are normally fixed by the selected preset. The exception is 'method_num' for 'preset = "euclidean"' when all predictors selected by the step are numeric. |
method_cat |
Character string specifying the categorical-variable dissimilarity passed to [mdist()] when 'preset = "custom"'. Common values include '"matching"' and '"tvd"'. Use [all_dist_method_specs()] to inspect available methods. |
method_num |
Character string or 'NULL' specifying numerical-variable preprocessing. Supported values are '"none"' for no preprocessing, '"std"' for standard-deviation scaling, '"range"' for range scaling, '"robust"' for inter-quartile-range-based scaling, and '"pc_scores"' for principal-component score scaling. With 'preset = "custom"', 'NULL' defaults to '"none"'. With 'preset = "euclidean"', the default is '"std"'; when all predictors selected by the step are numeric, an explicitly supplied value can override that default. |
commensurable |
Logical. If 'TRUE', dissimilarities are scaled so that the average contribution of each variable to the overall distance is equal to 1, when supported by the selected distance specification. |
ncomp |
Integer or 'NULL'. Number of principal components to retain when 'method_num = "pc_scores"' or 'preset = "u_dep_bw"'. If 'NULL', all available components are used unless 'threshold' is supplied and supported by the underlying method. |
threshold |
Numeric or 'NULL'. Optional cumulative variance threshold used when 'method_num = "pc_scores"' or 'preset = "u_dep_bw"'. |
columns |
Names of columns selected at prep time. Used internally by recipes. |
response_col |
Name of the outcome selected at prep time when it is used by the distance specification. Used internally by recipes. |
train_predictors |
Training predictors stored at prep time. Used internally by recipes to compute distances from new observations to the training observations. |
preprocessor |
Internal fitted manydist preprocessor. |
skip |
Logical. Standard recipes argument indicating whether the step should be skipped when baking new data. |
id |
Character string. Unique step identifier. |
The step can produce either distances from new observations to the training observations, or the within-training pairwise dissimilarity matrix. The former is the usual choice for supervised prediction workflows; the latter is useful for distance-based clustering workflows fitted on the training data.
During [recipes::prep()], 'step_mdist()' stores the selected training predictors and fits the internal manydist preprocessor. During [recipes::bake()], the selected predictors are removed and replaced by distance columns named 'dist_1', 'dist_2', and so on.
When 'response_used = TRUE', the step discovers the outcome from the recipe variable roles. For response-aware presets and custom categorical methods, the outcome from the current analysis set is supplied to [mdist()] during preparation. The fitted response-aware profiles are then reused when new data are baked; assessment and test outcomes are never required or used.
With 'output = "distance_to_training"', baking the training data returns the training pairwise distances, while baking new data returns distances from each new observation to each training observation. This rectangular representation is suitable for nearest-neighbour prediction models.
With 'output = "pairwise"', the step returns the within-training pairwise dissimilarity matrix. Baking genuinely new data is not supported in this mode, because the output is intended for training-only clustering workflows such as [pam_dist()] or [spectral_dist()].
An updated recipe with a manydist step.
[mdist()], [nearest_neighbor_dist()], [pam_dist()], [spectral_dist()], [all_dist_method_specs()]
if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() # Distance-to-training representation for prediction workflows rec <- recipes::recipe(species ~ ., data = penguins_small) |> step_mdist( recipes::all_predictors(), preset = "gower", output = "distance_to_training" ) rec_prep <- recipes::prep(rec, training = penguins_small) baked <- recipes::bake(rec_prep, new_data = penguins_small) baked |> dplyr::slice_head(n=5) # Pairwise representation for clustering workflows rec_pairwise <- recipes::recipe(~ ., data = penguins_small) |> step_mdist( recipes::all_predictors(), preset = "gower", output = "pairwise" ) rec_pairwise_prep <- recipes::prep(rec_pairwise, training = penguins_small) pairwise_dist <- recipes::bake(rec_pairwise_prep, new_data = penguins_small) pairwise_dist }if (requireNamespace("palmerpenguins", quietly = TRUE)) { data("penguins", package = "palmerpenguins") penguins_small <- palmerpenguins::penguins |> dplyr::select( species, bill_length_mm, bill_depth_mm, flipper_length_mm, body_mass_g, island, sex ) |> tidyr::drop_na() # Distance-to-training representation for prediction workflows rec <- recipes::recipe(species ~ ., data = penguins_small) |> step_mdist( recipes::all_predictors(), preset = "gower", output = "distance_to_training" ) rec_prep <- recipes::prep(rec, training = penguins_small) baked <- recipes::bake(rec_prep, new_data = penguins_small) baked |> dplyr::slice_head(n=5) # Pairwise representation for clustering workflows rec_pairwise <- recipes::recipe(~ ., data = penguins_small) |> step_mdist( recipes::all_predictors(), preset = "gower", output = "pairwise" ) rec_pairwise_prep <- recipes::prep(rec_pairwise, training = penguins_small) pairwise_dist <- recipes::bake(rec_pairwise_prep, new_data = penguins_small) pairwise_dist }
Displays the pairwise diagnostics already computed by benchmark_mdist(),
without refitting distances or recomputing diagnostics.
## S3 method for class 'MDistBenchmark' summary(object, ..., n = 10L, width = NULL)## S3 method for class 'MDistBenchmark' summary(object, ..., n = 10L, width = NULL)
object |
An object returned by |
... |
Additional arguments passed to the tibble print method. |
n |
Number of rows to display. Use |
width |
Width of the printed output; |
Invisibly returns the complete tibble with one row per unique pair
of successful specifications, regardless of the display limit n.
Columns include the method labels, MAD, symmetric relative distance,
MDS congruence, alienation, and any requested clustering ARIs.
If fewer than two specifications succeed, the tibble has zero rows.
## Not run: pairwise_results <- summary(benchmark_result) ## End(Not run)## Not run: pairwise_results <- summary(benchmark_result) ## End(Not run)
A fixed snapshot of selected economic, demographic, and labour-market indicators for 181 countries, together with World Bank regional, income, and lending classifications. It provides a compact mixed-type dataset for illustrating distance construction and distance-based learning.
A data frame with 181 rows and 9 variables:
Country or economy name.
World Bank region, as a factor with 7 levels.
World Bank income group, as a factor with 5 levels.
World Bank lending category, as a factor with 4 levels.
GDP per capita in thousands of current US dollars.
Life expectancy at birth, in years.
Unemployment as a percentage of the total labour force, based on the modelled ILO estimate.
Urban population as a percentage of the total population.
Annual population growth, in percent.
The reference year is 2022 and the snapshot was retrieved from the World Bank Indicators API on 2026-05-05. Aggregate regions and observations with missing values in any selected field were removed. GDP per capita was divided by 1,000 and rounded to one decimal place; life expectancy, unemployment, and urban population were rounded to one decimal place; and population growth was rounded to two decimal places.
The numerical indicators and their World Bank codes are:
GDP per capita: NY.GDP.PCAP.CD;
life expectancy: SP.DYN.LE00.IN;
unemployment: SL.UEM.TOTL.ZS;
urban population: SP.URB.TOTL.IN.ZS;
population growth: SP.POP.GROW.
The original World Development Indicators data are distributed by the World Bank under the Creative Commons Attribution 4.0 International license, subject to the World Bank Dataset Terms. Attribution: The World Bank: World Development Indicators.
The World Bank, World Development Indicators, https://datacatalog.worldbank.org/search/dataset/0037712/world-development-indicators.
World Bank Dataset Terms, https://www.worldbank.org/ext/en/legal/terms-conditions/datasets.
data("wdi_2022", package = "manydist") dplyr::glimpse(wdi_2022)data("wdi_2022", package = "manydist") dplyr::glimpse(wdi_2022)