Computer Graphics and Multimedia

Content-Aware Video Resizing

Spatial Resizing

Temporal Reduction

Temporal Extension

Abstract

The rapid advance on modern technology has significantly enhanced the procedure of video capture, both in quality and performance, thus the inundation of video data. Video abstraction or video summarization, which focuses on how to quickly and efficiently extract useful information from the raw and huge amount of video data, has therefore become a very important research issue in recent years. Numerous approaches have been developed but each has its drawbacks or limitations.

 

Inspired by the work of Avidan et al. on seam carving for content-aware image resizing, Chen et al. generalize the idea to perform video carving by repeatedly finding and removing the 2D sheet with the minimal cost from the video volume, thus achieving the temporal reduction of a video. However, there are several issues remained in Chen et al.'s approach, which leaves room for further improvement.

 

First, the extracted 2D sheet may not be smooth, that is, two spatially adjacent pixels may sit on two different frames that are temporally adjacent, thus causing periodically fragmented results. Second, the employed graph-cut algorithm entails a lengthy computation time. Finally, the required memory is relatively large, and as a result, the input video has to be broken into subsets so that each subset could be brought into the memory in its entirety and processed accordingly. Worse yet, such a partitioning scheme may lead to undesired sub-optimal results.

 

The main contribution of this paper is to propose a new approach that addresses all the mentioned issues in one shot. First, by modifying the dynamic programming approach originally adopted in Avidan et al.'s seam carving work, we could extract smooth 2D sheets and thus avoid fragmented outlook. Second, unlike Chen et al.'s graph-cut algorithm, our approach is much simpler and efficient, and could perform similar tasks but with a speed that is about two orders of magnitude faster. Finally, the memory consumption is also greatly reduced to be one order of magnitude smaller.

 

To make our work more complete, we also propose methods to extend the length of a video, as well as to resize a video in spatial domain. Results are shown and compared with existing approaches, if applicable, to demonstrate the effectiveness of our proposed approach.

 

Keyword : video summarization , seam carving , graph-cut algorithm , dynamic programming

 

Related publication

Paper

 

 

 

.RAR

 

 

Presentation ppt 

 

 

 

.RAR

 

 

Paper video

 

 

 

.RAR

 

 

ntnu ppt

 

 

 

.RAR

 

 

 

Result video

(all_original)

(all_our_shorten)

(all_our_extension)

(all_our_resize)

(all)

(all)

(all)

(all_graph cut)

(all_video_textures)

(all_seam_carving)

Original

video

Our

approach

Video carving

Fast texture synthesis using tree structured

Texture Mixing and Texture Movie Synthesis

Image and Video Synthesis Using Graph Cuts

Video textures

Seam Carving for Content-Aware Image Resizing

Campus_5(shorten)

350_150

 

 

 

 

 

 

Campus_5

(shorten)

240_190

240_190

 

 

 

 

 

Beach

(shorten)

1163_733

 

 

 

1163_733

 

 

Waterfal2

(extension)

59_109

 

 

59_63

59_109

 

 

Smoke

(extension)

30_90

 

30_64

 

30_90

 

 

River

(extension)

58_68

 

 

 

58_68

 

 

Grass

(extension)

3D_98_449

 

 

 

 

98_449

 

Water can

(extension)

400_570

 

 

 

 

400_570

 

SHOCK_WAVE

(extension)

405_905

 

 

 

 

 

 

Beach_200

(resize)

368x240_246x240

 

 

 

 

 

368x240_246x240

 

368x240_368x160

 

 

 

 

 

368x240_368x160

Waterfall

(resize)

320x240_214x240

 

 

 

 

 

320x240_214x240

Waterfall B

(resize)

384x288_384x216

 

 

 

 

 

384x288_384x216

Balloons

(resize)

224x157_179x157

 

 

 

 

 

 

224x157_179x157

 

224x157_224x105

 

 

 

 

 

224x157_224x105