logo
科言猫学术研究的AI总结
language
手机端手机端
  • Phone Feature微信小程序
简体中文简体中文
notice消息通知
博客博客
delete
arrow
返回

How to Explain Grokking

delete2026-05-01
delete0
PRE
AI
K
Kozyrev, S. V. *
DOI：10.1134/s0965542526700181delete
delete原文链接
delete原文求助
delete分享
delete收藏
摘要

摘要

En 中文
格罗金现象（学习理论中的延迟泛化）是指一个系统最初通过过拟合（缺乏泛化性质）来记忆训练样本，然后在经过一段显著的时间后，过拟合之后发生泛化。本文通过使用随机梯度（布朗运动）的朗之万动力学来建模学习，并应用热力学思想来解释格罗金现象。研究表明，格罗金现象的性质可以解释为布朗运动的性质，特别是动力学理论中的艾林公式。
Keyword:
grokking
overfitting
generalization

期刊

C
Computational Mathematics and Mathematical Physics
IF:
0
论文数:
86
被引数:
0

机构

R
russian academy of sciences
学者数:
9.1W
论文数: 6.0W
被引数: 60
引用论文

引用论文

Correlation functions and computer simulations
errNuclear Physics B
IF0
err1981-05-01
err0
PREAI
errG. Parisi
err分享
err收藏
SOME METHODS OF CONTROL FOR COMPLEX SYSTEMS
errRussian Mathematical Surveys
IF0
err2007-10-16
err0
PREAI
errI M Gel'fand; M L Tsetlin
err分享
err收藏
Control of Overfitting with Physics
errEntropy
IF0
err
err0
PREAI
errKozyrev,Sergei V.; Lopatin,Ilya A.; Pechen,Alexander N.
err分享
err收藏
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks
errApplied and Computational Harmonic Analysis
IF0
err2022-07-01
err0
errOAAI
errChaoyue Liu; Libin Zhu; Mikhail Belkin
err分享
err收藏
General conditions for predictivity in learning theory
errNature
IF0
err2004-03-25
err0
PREAI
errTomaso Poggio; Ryan Rifkin; Sayan Mukherjee; Partha Niyogi
err分享
err收藏
Towards Understanding Grokking: An Effective Theory of Representation Learning
errAdvances in Neural Information Processing Systems 35
IF0
err2022-01-01
err0
PREAI
errLiu,Ziming; Kitouni,Ouail; Nolte,Niklas S; Michaud,Eric; Tegmark,Max; Williams,Mike
err分享
err收藏
Correlation functions and computer simulations (II)
errNuclear Physics B
IF0
err1982-06-01
err0
PREAI
errG. Parisi
err分享
err收藏
Flat Minima
errNeural Computation
IF0
err1997-01-01
err0
PREAI
errSepp Hochreiter; Jürgen Schmidhuber
err分享
err收藏
没有更多内容