arrow
Return

Toolbox for Distance Estimation and Cluster Validation on Data With Missing Values

delete2022-01-01
delete4
delete
OA
AI
M
Marko Niemelä *
S
Sami Äyrämö
T
Tommi Kärkkäinen
DOI:10.1109/ACCESS.2021.3136435delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
Missing data are unavoidable in the real-world application of unsupervised machine learning, and their nonoptimal processing may decrease the quality of data-driven models. Imputation is a common remedy for missing values, but directly estimating expected distances have also emerged. Because treatment of missing values is rarely considered in clustering related tasks and distance metrics have a central role both in clustering and cluster validation, we developed a new toolbox that provides a wide range of algorithms for data preprocessing, distance estimation, clustering, and cluster validation in the presence of missing values. All these are core elements in any comprehensive cluster analysis methodology. We describe the methodological background of the implemented algorithms and present multiple illustrations of their use. The experiments include validating distance estimation methods against selected reference methods and demonstrating the performance of internal cluster validation indices. The experimental results demonstrate the general usability of the toolbox for the straightforward realization of alternate data processing pipelines. Source code, data sets, results, and example macros are available on GitHub. https://github.com/markoniem/nanclustering_toolbox
Keywords:
Covariance matrices
Estimation
Clustering algorithms
Task analysis
Data preprocessing
Convergence
Clustering methods
Missing values
distance estimation
clustering
cluster validation

Journal

IEEE Access cover
IEEE Access
IF:
3.6
Papers:
9.8W
Citations:
29.4W

Organization

U
university of jyvaskyla
Scholars:
6.3K
Papers: 6.8K
Citations: 12
Cited Papers

Cited Papers

Evaluation of δ13C in Carbonaceous Aerosol Source Apportionment at a Rural Measurement Site
err2017-01-01
err0
errOAAI
errJohan Martinsson; August Andersson; Moa K. Sporre; Johan Friberg; Adam Kristensson; Erik Swietlicki; Pål-Axel Olsson; Kristina Eriksson Stenström
errShare
errSave
Towards energy-autonomous wake-up receiver using Visible Light Communication
err2016-01-01
err0
errOAAI
errJoyce Sariol Ramos; Ilker Demirkol; Josep Paradells; Daniel Vossing; Karim M. Gad; Martin Kasemann
errShare
errSave
Towards a wireless and fully-implantable ECoG system
err2013-06-01
err0
PREAI
errE. Tolstosheeva; J. Hoeffmann; J. Pistor; D. Rotermund; T. Schellenberg; D. Boll; T. Hertzberg; V. Gordillo-Gonzalez; S. Mandon; D. Peters-Drolshagen; M. Schneider; K. Pawelzik; A. Kreiter; S. Paul; W. Lang
errShare
errSave
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
err1998-07-01
err0
errOAAI
errBernhard Schölkopf; Alexander Smola; Klaus-Robert Müller
errShare
errSave
ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
err2024-05-11
err0
errOAAI
errIan Arawjo; Chelse Swoopes; Priyan Vaithilingam; Martin Wattenberg; Elena L. Glassman
errShare
errSave
errShare
errSave
An extensive comparative study of cluster validity indices
err2013-01-01
err920
PREAI
errArbelaitz, Olatz; Gurrutxaga, Ibai; Muguerza, Javier; Perez, Jesus M.; Perona, Inigo
errShare
errSave
researcher View more