.png){kind=icon}
Return
Network generalization differences quantified
.png){kind=icon}
PRE
AI
DOI：10.1016/0893-6080(95)00110-7.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
It has long been observed, and frequently noted, by connectionists that small changes in initial conditions, prior to training, can result in networks that generalize very differently. We have performed a systematic study of this phenomenon, using a number of different statistical measures of generalization differences. From these we derive a formal definition of Generalization Diversity. We quantify the relative impacts on generalization of the major parameters used in network initialization as well as extend the formal framework to also encompass the differences in generalization difference from one parameter to another. We reveal, for example, the relative effects of random initialization of the link weights and variation of the number of hidden units, and how similar these two resultant effects are. Finally, examples are presented of how the proposed generalization diversity measure may be exploited in order to improve the performance of neural-net systems. We show how several of these measures can be used to engineer reliability improvements in neural-net systems.
Keywords:
generalization diversity
network initialization
neural-net systems
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
6.3
Papers:
8.2K
Citations:
3.0W
Organization
No organization information available
.png){kind=icon}
