A video control user interface is provided for use in an interactive television system. The video control user interface includes a remote control handset with a multi-purpose, multi-direction actuation pad and a set-top box configured to operate in different modes, including a movie-on-demand mode. In this mode, the set-top box receives digitally transmitted video data streams of a selected movie from a centralized head end server. During display of a video movie, the set-top box can cause, at the viewer's request, the television to display an icon representing a physical layout of the actuation pad on the remote control handset and one or more symbols arranged at locations relative to the icon. The symbols relate to shuttle controls for controlling viewing of the video movie. This user interface presents an intuitive visual mapping of the shuttle controls about the depicted icon onto physical actuation positions of the multi-direction pad on the remote control handset. When the viewer wishes to change the viewing mode (such as from "play" to "pause"), the viewer simply depresses the pad at an actuation position that corresponds to a desired shuttle control symbol arranged at approximately the same location relative to the pad-resembling icon that is displayed on the screen. This user interface provides intuitive video control using a multi-purpose actuator, thereby eliminating the need for dedicated shuttle control buttons on the remote control handset.