Just got pasted a link about this very neat way to navigate through videos and their playback, Video Browsing By Direct Manipulation. Haven’t read the paper yet (it’s linked on the same page) but it seems that they’re recognizing objects through several frames, and then allow a user to navigate through the video by following those movements. The video says it all.