Network generalization differences quantified

doi:10.1016/0893-6080(95)00110-7

Network generalization differences quantified

1996-03-01

49

PRE

AI

D

Derek Partridge *

DOI：10.1016/0893-6080(95)00110-7

原文链接

原文求助

摘要

En 中文

It has long been observed, and frequently noted, by connectionists that small changes in initial conditions, prior to training, can result in networks that generalize very differently. We have performed a systematic study of this phenomenon, using a number of different statistical measures of generalization differences. From these we derive a formal definition of Generalization Diversity. We quantify the relative impacts on generalization of the major parameters used in network initialization as well as extend the formal framework to also encompass the differences in generalization difference from one parameter to another. We reveal, for example, the relative effects of random initialization of the link weights and variation of the number of hidden units, and how similar these two resultant effects are. Finally, examples are presented of how the proposed generalization diversity measure may be exploited in order to improve the performance of neural-net systems. We show how several of these measures can be used to engineer reliability improvements in neural-net systems.

Keyword:

generalization diversity

network initialization

neural-net systems

Network generalization differences quantified

摘要

AI总结

期刊

机构

引用论文