First person video summarization using different graph representations

Abhimanyu Sahu

doi:10.1016/j.patrec.2021.03.013

Return

First person video summarization using different graph representations

2021-06-01

PRE

Abhimanyu Sahu

Ananda S. Chowdhury *

DOI：10.1016/j.patrec.2021.03.013

Original

Original request for help

Save

Abstract

En 中文

First-person video summarization has emerged as an important research problem for computer vision and multimedia communities. In this paper, we show how different graph representations can be devel-oped for accurately summarizing first-person (egocentric) videos in a computationally efficient manner. Each frame in a video is first represented as a weighted graph. A shot boundary detection method us -ing graph based mutual information is developed. We next construct a weighted graph for each shot. A representative frame from each shot is selected using a graph centrality measure. A new way of charac-terizing egocentric video frames using a graph based center-surround model is shown next. Here, each representative frame is modeled as a union of a center region (graph) and a surround region (graph). By exploiting spectral measures of dissimilarity between the two (center and surround) graphs, optimal cen-ter and surround regions are determined. Optimal regions for all frames within a shot are kept the same as that of the representative frame. Center-surround differences in entropy and optical flow values along with PHOG (Pyramidal HOG) features are extracted from each frame. All frames in a video are finally represented by another weighted graph, termed as a Video Similarity Graph (VSG). The frames are clus-tered by applying a Minimum Spanning Tree (MST) based approach with a new measure for inadmissible edges. Frames closest to the centroid of each cluster are captured to build the summary. Experimental evaluation on two benchmark datasets indicate the advantage of the proposed formulation. (c) 2021 Elsevier B.V. All rights reserved.

Keywords:

First-person video

Center-surround model

Spectral graph dissimilarity

Video similarity graph

Edge inadmissibility measure

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.