In this study, we propose a framework to describe the relationship named spatiotemporal correlation between video contents and human gaze dynamics. The spatiotemporal correlation consists of (1) the event-level spatiotemporal gaps between visual events in videos and gaze reactions and (2) the scene-level correlations between video scene structures and corresponding gaze dynamics. Our framework can describe this twofold relationship simply and efficiently by discovering and combining primitive spatiotemporal patterns of visually salient regions in videos and those of gaze. The effectiveness of this framework is confirmed via several practical tasks of gaze behavior analyses in real environments, attentional target identification, attentive state estimation and gaze point prediction.In this study, we propose a framework to describe the relationship named spatiotemporal correlation between video contents and human gaze dynamics. The spatiotemporal correlation consists of (1) the event-level spatiotemporal gaps between visual events in videos and gaze reactions and (2) the scene-level correlations between video scene structures and corresponding gaze dynamics. Our framework can describe this twofold relationship simply and efficiently by discovering and combining primitive spatiotemporal patterns of visually salient regions in videos and those of gaze. The effectiveness of this framework is confirmed via several practical tasks of gaze behavior analyses in real environments, attentional target identification, attentive state estimation and gaze point prediction.