Face recognition is a K class problem. where K is the number of known individuals; and support vector machines (SVMs) are a binary classi(cid:173) fication method. By reformulating the face recognition problem and re(cid:173) interpreting the output of the SVM classifier. we developed a SVM -based face recognition algorithm. The face recognition problem is formulated as a problem in difference space. which models dissimilarities between two facial images. In difference space we formulate face recognition as a two class problem. The classes are: dissimilarities between faces of the same person. and dissimilarities between faces of different people. By modifying the interpretation of the decision surface generated by SVM. we generated a similarity metric between faces that is learned from ex(cid:173) amples of differences between faces. The SVM-based algorithm is com(cid:173) pared with a principal component analysis (PeA) based algorithm on a difficult set of images from the FEREf database. Performance was mea(cid:173) sured for both verification and identification scenarios. The identification performance for SVM is 77-78% versus 54% for PCA. For verification. the equal error rate is 7% for SVM and 13 % for PCA.