arrow
Return

Performance analysis for machine-learning experiments using small data sets

delete2003-01-01
delete35
PRE
AI
R
R. Lacroix
K
K. M. Wade
DOI:10.1016/S0168-1699(02)00104-7delete
deleteOriginal
deleteShare
deleteSave
Abstract

Abstract

En 中文
Machine-learning techniques are increasingly used to deal with a variety of problems in agriculture, However, challenges with the application of machine-learning. such as analyzing the performance achieved through learning from small data sets. still remain. This study focused on using graphical and statistical techniques to analyze the results of machine-learning experiments involving data preprocessing and algorithm tuning. Data consisted of 1428 cases that were classified by a dairy-nutrition specialist as outliers (34 cases) or non-outliers. The performance of classifiers, generated with decision-tree induction, was estimated through tenfold cross validation. Relative operating characteristic (ROC) curves were used to visualize the achieved trade-offs between correctly classifying positive and negative cases. A performance index, representing the mean true positive rate of these curves for a limited range of false positive rate values, was developed to facilitate comparison among classification schemes. Analysis of variance (ANOVA) was used to determine whether real differences existed for the expected performance on new data among the different combinations of data preprocessing and algorithm configurations evaluated in this study. In terms of data preprocessing. randomly assigning herds to the folds of the cross validation did not perform significantly differently from assigning cases to folds, while using a special value to indicate irrelevant attribute values significantly improved the performance over treating these values as unknown. Tuning the configuration of the decision-tree induction algorithm significantly improved the classification performance. The application of ten-fold cross validation in combination with ROC curves and ANOVA was found to he Useful in analyzing the results of machine-learning experiments involving decision-tree induction and small data sets, These methods could also be used with other machine-learning techniques Such as artificial neural networks and instance-based learning. (C) 2002 Elsevier Science B.V. All rights reserved.
Keywords:
machine-learning
decision-tree induction
data preprocessing
algorithm configurations
ROC curves
analysis of variance

Journal

Computers and Electronics in Agriculture cover
Computers and Electronics in Agriculture
IF:
8.9
Papers:
1.0W
Citations:
6.8W

Organization

No organization information available
Cited Papers

Cited Papers

errShare
errSave
Urban Traffic Analysis through an UAV
err2014-02-01
err0
errOAAI
errGiuseppe Salvo; Luigi Caruso; Alessandro Scordo
errShare
errSave
Anomaly driven signatures of extra U(1)’s
err2010-01-01
err0
PREAI
errIgnatios Antoniadis; Alexey Boyarsky; Oleg Ruchayskiy; George Alverson; Pran Nath; Brent Nelson
errShare
errSave
APPLYING MACHINE LEARNING TO AGRICULTURAL DATA
err1995-06-01
err83
errOAAI
errMCQUEEN, RJ; GARNER, SR; NEVILLMANNING, CG; WITTEN, IH
errShare
errSave
errShare
errSave
errShare
errSave
researcher View more