Hyperbolic Binary Neural Network

Jun Chen

doi:10.1109/TNNLS.2024.3485115

Hyperbolic Binary Neural Network

2024-01-01

0

PRE

AI

DOI：10.1109/TNNLS.2024.3485115

Original

Original request for help

Save

Abstract

En 中文

Binary neural network (BNN) converts full-precision weights and activations into their extreme 1-bit counterparts, making it particularly suitable for deployment on lightweight mobile devices. While BNNs are typically formulated as a constrained optimization problem and optimized in the binarized space, general neural networks are formulated as an unconstrained optimization problem and optimized in the continuous space. This article introduces the hyperbolic BNN (HBNN) by leveraging the framework of hyperbolic geometry to optimize the constrained problem. Specifically, we transform the constrained problem in hyperbolic space into an unconstrained one in Euclidean space using the Riemannian exponential map. On the other hand, we also propose the exponential parametrization cluster (EPC) method, which, compared with the Riemannian exponential map, shrinks the segment domain based on a diffeomorphism. This approach increases the probability of weight flips, thereby maximizing the information gain in BNNs. Experimental results on CIFAR10, CIFAR100, and ImageNet classification datasets with VGGsmall, ResNet18, and ResNet34 models illustrate the superior performance of our HBNN over state-of-the-art methods.

Keywords:

Neural networks

Vectors

Optimization

Geometry

Manifolds

Transforms

Quantization (signal)

Aerospace electronics

Training

Mobile handsets

Binary neural network (BNN)

deep learning

hyperbolic geometry

model compression

Hyperbolic Binary Neural Network

Abstract

Journal

Organization