.png){kind=icon}
.png){kind=icon}
Return
Deep Hypercomplex Networks for Spatiotemporal Data Processing: Parameter efficiency and superior performance
.png){kind=icon}
PRE
AI
DOI：10.1109/MSP.2024.3381808.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Hypercomplex numbers, such as quaternions and octonions, have recently gained attention because of their advantageous properties over real numbers, e.g., in the development of parameter-efficient neural networks. For instance, the 16-component sedenion has the capacity to reduce the number of network parameters by a factor of 16. Moreover, hypercomplex neural networks offer advantages in the processing of spatiotemporal data as they are able to represent variable temporal data divisions through the hypercomplex components. Similarly, they support multimodal learning, with each component representing an individual modality. In this article, the key components of deep learning in the hypercomplex domain are introduced, encompassing concatenation, activation functions, convolution, and batch normalization. The use of the backpropagation algorithm for training hypercomplex networks is discussed in the context of hypercomplex algebra. These concepts are brought together in the design of a ResNet backbone using hypercomplex convolution, which is integrated within a U-Net configuration and applied in weather and traffic forecasting problems. The results demonstrate the superior performance of hypercomplex networks compared to their real-valued counterparts, given a fixed parameter budget, highlighting their potential in spatiotemporal data processing.
Keywords:
Training data
Convolutional neural networks
Algebra
Quaternions
Image processing
Data processing
Spatiotemporal phenomena
Hypercomplex
Neural networks
Backpropagation algorithms
Batch normalization
Journal
IF:
9.6
Papers:
1.1W
Citations:
1.7W
Organization
No organization information available