We present techniques for rendering and animation of realistic scenes by analyzing and training on short video sequences. This work extends the new paradigm for computer animation, video tex(cid:173) tures, which uses recorded video to generate novel animations by replaying the video samples in a new order. Here we concentrate on video sprites, which are a special type of video texture. In video sprites, instead of storing whole images, the object of inter(cid:173) est is separated from the background and the video samples are stored as a sequence of alpha-matted sprites with associated veloc(cid:173) ity information. They can be rendered anywhere on the screen to create a novel animation of the object. We present methods to cre(cid:173) ate such animations by finding a sequence of sprite samples that is both visually smooth and follows a desired path. To estimate visual smoothness, we train a linear classifier to estimate visual similarity between video samples. If the motion path is known in advance, we use beam search to find a good sample sequence. We can specify the motion interactively by precomputing the sequence cost function using Q-Iearning.