.png){kind=icon}
返回
Exploring Spherical Autoencoder for Spherical Video Content Processing
.png){kind=icon}
OA
AI
DOI：10.1145/3503161.3548364.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
3D spherical content is increasingly presented in various applications (e.g., AR/MR/VR) for better users' immersiveness experience, yet today processing such spherical 3D content still mainly relies on the traditional 2D approaches after projection, leading to the distortion and/or loss of critical information. This study sets to explore methods to process spherical 3D content directly and more effectively. Using 360-degree videos as an example, we propose a novel approach called Spherical Autoencoder (SAE) for spherical video processing. Instead of projecting to a 2D space, SAE represents the 360-degree video content as a spherical object and employs encoding and decoding on the 360-degree video directly. Furthermore, to support the adoption of SAE on pervasive mobile devices that often have resource constraints, we further propose two optimizations on top of SAE. First, since the FoV (Field of View) prediction is widely studied and leveraged to transport only a portion of the content to the mobile device to save bandwidth and battery consumption, we design p-SAE, a SAE scheme with the partial view support that can utilize such FoV prediction. Second, since machine learning models are often compressed when running on mobile devices in order to reduce the processing load, which usually leads to degradation of output (e.g., video quality in SAE), we propose c-SAE by applying the compressive sensing theory into SAE to maintain the video quality when the model is compressed. Our extensive experiments show that directly incorporating and processing spherical signals is promising, and it outperforms the traditional approaches by a large margin. Both p-SAE and c-SAE show their effectiveness in delivering high quality videos (e.g., PSNR results) when used alone or combined together with model compression.
Keyword:
Spherical Autoencoder
Partial View
Compressive Sensing
期刊
机构
引用论文
N-Doped Graphene-Supported PtRu Direct Methanol Fuel Cell Electrocatalysts氮掺杂石墨烯负载PtRu直接甲醇燃料电池电催化剂OAAI
OAAI分享收藏Normative pediatric visual acuity using single surrounded HOTV optotypes on the Electronic Visual Acuity Tester following the Amblyopia Treatment Study protocol根据弱视治疗研究方案，在电子视力测试仪上使用单个包围的HOTV视标的儿童视力
OAAI
OAAI分享收藏Personal identifiability of user tracking data during observation of 360-degree VR video.png)
SCIENTIFIC REPORTSOAAI
SCIENTIFIC REPORTSIF3.9
OAAI分享收藏
查看更多内容