Progressive Bitwidth Assignment Approaches for Efficient Capsule Networks Quantization

doi:10.1109/ACCESS.2025.3534434

Progressive Bitwidth Assignment Approaches for Efficient Capsule Networks Quantization

2025-01-01

0

OA

AI

M

Mohsen Raji *

A

Amir Ghazizadeh Ahsaei

DOI：10.1109/ACCESS.2025.3534434

原文链接

原文求助

摘要

En 中文

Capsule Networks (CapsNets) are a class of neural network architectures that can be used to more accurately model hierarchical relationships due to their hierarchical structure and dynamic routing algorithms. However, their high accuracy comes at the cost of significant memory and computational resources, making them less feasible for deployment on resource-constrained devices. In this paper, progressive bitwidth assignment approaches are introduced to efficiently quantize the CapsNets. Initially, a comprehensive and detailed analysis of parameter quantization in CapsNets is performed exploring various granularities, such as block-wise quantization and dynamic routing quantization. Then, three quantization approaches are applied to progressively quantize the CapsNet, considering various insights into the susceptibility of layers to quantization. The proposed approaches include Post-Training Quantization (PTQ) strategies that minimize the dependence on floating-point operations and incorporates layer-specific integer bit-widths based on quantization error analysis. PTQ strategies employ Power-of-Two (PoT) scaling factors to simplify computations, effectively utilizing hardware shifts and significantly reducing the computational complexity. This technique not only reduces the memory footprint but also maintains accuracy by introducing a range clipping method tailored to the hardware's capabilities, obviating the need for data preprocessing. Our experimental results on ShallowCaps and DeepCaps networks across multiple datasets (MNIST, Fashion-MNIST, CIFAR-10, and SVHN) demonstrate the efficiency of our approach. Specifically, on the CIFAR-10 dataset using the DeepCaps architecture, we achieved a substantial memory reduction (7.02x for weights and 3.74x for activations) with a minimal accuracy loss of only 0.09%. By using progressive bitwidth assignment and post-training quantization, this work optimizes CapsNets for efficient, real-time visual processing on resource-constrained edge devices, enabling applications in IoT, mobile platforms, and embedded systems.

Keyword:

Capsule networks

deep learning

neural networks

post-training quantization

compression

Capsule networks

deep learning

neural networks

post-training quantization

compression

Progressive Bitwidth Assignment Approaches for Efficient Capsule Networks Quantization

摘要

期刊

机构

引用论文