Machine Learning and Deep Learning Optimization Algorithms for Unconstrained Convex Optimization Problem

Naeem, Kainat

doi:10.1109/ACCESS.2024.3522361

Machine Learning and Deep Learning Optimization Algorithms for Unconstrained Convex Optimization Problem

2025-01-01

0

OA

AI

DOI：10.1109/ACCESS.2024.3522361

Original

Original request for help

Save

Abstract

En 中文

This paper conducts a thorough comparative analysis of optimization algorithms for an unconstrained convex optimization problem. It contrasts traditional methods like Gradient Descent (GD) and Nesterov Accelerated Gradient (NAG) with modern techniques such as Adaptive Moment Estimation (Adam), Long Short-Term Memory (LSTM) and Multilayer Perceptron (MLP). Through empirical experiments, convergence speed, solution accuracy and robustness, is evaluated providing insights to aid algorithm selection. The convergence dynamics of convex optimization, is explored analyzing classical algorithms and contemporary neural network (NN) methodologies. The study concludes with a comparative assessment of these algorithms performance metrics and their respective strengths and weaknesses.

Keywords:

Optimization

Convex functions

Convergence

Training

Long short term memory

Heuristic algorithms

Machine learning algorithms

Linear programming

Accuracy

Adaptation models

Adaptive moment estimation (Adam)

convex optimization

deep learning (DL)

gradient descent (GD)

long short term memory (LSTM) networks

machine learning (ML)

mathematical optimization

multilayer perceptron (MLP)

nesterov accelerated gradient (NAG)

Machine Learning and Deep Learning Optimization Algorithms for Unconstrained Convex Optimization Problem

Abstract

Journal

Organization