Stimulus arrays are inevitably presented at different positions on the retina in visual tasks, even those that nominally require fixation. In par(cid:173) ticular, this applies to many perceptual learning tasks. We show that per(cid:173) ceptual inference or discrimination in the face of positional variance has a structurally different quality from inference about fixed position stimuli, involving a particular, quadratic, non-linearity rather than a purely lin(cid:173) ear discrimination. We show the advantage taking this non-linearity into account has for discrimination, and suggest it as a role for recurrent con(cid:173) nections in area VI, by demonstrating the superior discrimination perfor(cid:173) mance of a recurrent network. We propose that learning the feedforward and recurrent neural connections for these tasks corresponds to the fast and slow components of learning observed in perceptual learning tasks.