In this article we study the effects of introducing structure in the input distribution of the data to be learnt by a simple perceptron. We determine the learning curves within the framework of Statis(cid:173) tical Mechanics. Stepwise generalization occurs as a function of the number of examples when the distribution of patterns is highly anisotropic. Although extremely simple, the model seems to cap(cid:173) ture the relevant features of a class of Support Vector Machines which was recently shown to present this behavior.