{"title": "Sex with Support Vector Machines", "book": "Advances in Neural Information Processing Systems", "page_first": 960, "page_last": 966, "abstract": null, "full_text": "Sex with Support  Vector  Machines \n\nBaback  Moghaddam \n\nMitsubishi Electric Research Laboratory \n\nCambridge MA  02139,  USA \n\nbaback<amerl.com \n\nMing-Hsuan Yang \n\nUniversity of Illinois at  Urbana-Champaign \n\nUrbana, IL 61801  USA \n\nmhyang<avision.ai.uiuc.edu \n\nAbstract \n\nNonlinear  Support  Vector  Machines  (SVMs)  are  investigated  for \nvisual sex classification with low resolution  \"thumbnail\"  faces  (21-\nby-12  pixels)  processed  from  1,755  images  from  the  FE RET face \ndatabase.  The  performance  of SVMs  is  shown  to  be  superior  to \ntraditional pattern classifiers (Linear, Quadratic, Fisher Linear Dis(cid:173)\ncriminant,  Nearest-Neighbor)  as  well  as  more  modern  techniques \nsuch as Radial Basis Function (RBF) classifiers and large ensemble(cid:173)\nRBF  networks.  Furthermore, the  SVM  performance  (3.4%  error) \nis  currently the best result reported in the open literature. \n\n1 \n\nIntroduction \n\nIn  recent  years,  SVMs  have  been  successfully  applied  to  various  tasks  in  com(cid:173)\nputational  face-processing.  These  include  face  detection  [14],  face  pose  discrim(cid:173)\nination  [12]  and  face  recognition  [16].  Although  facial  sex  classification  has \nattracted much  attention in  the psychological literature [1,  4,  8,  15],  relatively few \ncomputatinal  learning  methods  have  been  proposed.  We  will  briefly  review  and \nsummarize the prior art in facial sex classification. l \n\nGollomb  et at.  [10]  trained a fully  connected two-layer neural network, SEXNET, to \nidentify sex from 30-by-30 face  images.  Their experiments on a set of 90 photos (45 \nmales  and 45  females)  gave an average error rate of 8.1%  compared to  an average \nerror rate  of 11.6%  from  a  study of five  human  subjects.  Brunelli  and  Poggio  [2] \n\nlSex classification  is  also referred to as  gender classification  (for  political  correctness). \nHowever,  given  the  two  distinct  biological  classes,  the  scientifically  correct  term  is  sex \nclassification.  Gender often denotes  a  fuzzy  continuum of feminine ~ masculine  [1). \n\n\fFu.... __  --1-___  ---.J M \n\nF \n\nL-____ ~-+ ________ ~ \n\nM \n\nF \n\nL--___  _ \n\nM \n\nGender \nClassifier \n\nFigure 1:  Sex  classifier \n\ndeveloped  HyperBF  networks  for  sex  classification  in  which  two  competing  RBF \nnetworks,  one for  male  and  the  other for  female,  were  trained  using  16  geometric \nfeatures  as  inputs  (e.g.,  pupil to  eyebrow  separation, eyebrow thickness,  and nose \nwidth).  The  results  on  a  data set  of 168  images  (21  males  and  21  females)  show \nan  average  error rate  of 21%.  Using  similar techniques  as  Golomb  et  al.  [10]  and \nCottrell  and  Metcalfe  [6],  Tamura  et  al. \n[18]  used  multi-layer  neural  networks \nto  classify  sex  from  face  images  at  multiple  resolutions  (from  32-by-32  to  8-by-8 \npixels).  Their experiments on  30  test  images show  that their network  was  able to \ndetermine sex from 8-by-8 images with an average error rate of 7%.  Instead of using \na vector of gray levels to represent faces,  Wiskott  et  al.  [20]  used  labeled graphs of \ntwo-dimensional  views  to describe  faces.  The  nodes  were  represented  by wavelet(cid:173)\nbased local  \"jets\"  and the edges  were  labeled  with  distance  vectors  similar to the \ngeometric features in [3].  They used a small set of controlled model graphs of males \nand females to encode  \"general face  knowledge,\"  in order to generate graphs of new \nfaces  by  elastic  graph  matching.  For  each  new  face,  a  composite  reconstruction \nwas  generated  using  the  nodes  in  the  model  graphs.  The  sex  of the  majority  of \nnodes  used  in  the  composite  graph  was  used  for  classification.  The error  rate  of \ntheir  experiments  on  a  gallery  of  112  face  images  was  9.8%.  Recently,  Gutta  et \nal.  [11]  proposed a hybrid classifier based on neural networks (RBFs)  and inductive \ndecision trees with Quinlan's C4.5 algorithm.  Experiments with 3000 FERET faces \nof size  64-by-72 pixels yielded an error rate of 4%. \n\n2  Sex  Classifiers \n\nA  generic  sex  classifier  is  shown in  Figure  1.  An  input  facial  image  x  generates a \nscalar output f(x) whose polarity - sign of f(x) - determines class membership.  The \nmagnitude II f (x II  can usually be interpreted as a measure of belief or certainty in the \ndecision made.  Nearly all binary classifiers can be viewed in these terms; for density(cid:173)\nbased  classifiers  (Linear,  Quadratic  and  Fisher)  the output function  f(x)  is  a  log \nlikelihood  ratio,  whereas for  kernel-based  classifiers  (Nearest-Neighbor,  RBFs  and \nSVMs)  the output is  a  \"potential field\"  related to the distance from the separating \nboundary. \n\n\f2.1  Support  Vector  Machines \n\nA  Support  Vector  Machine  is  a  learning  algorithm  for  pattern  classification  and \nregression  [19,  5].  The basic training principle behind SVMs is finding  the optimal \nlinear hyperplane such that the expected classification error for unseen test samples \ni.e.,  good generalization performance.  According to the structural \nis  minimized -\nrisk minimization inductive principle [19],  a function that classifies the training data \naccurately  and which  belongs  to a  set  of functions  with  the lowest  VC  dimension \n[5]  will  generalize  best  regardless  of the  dimensionality  of the  input  space.  Based \non  this  principle,  a  linear  SVM  uses  a  systematic  approach  to  finding  a  class  of \nfunctions  with  the  lowest  VC  dimension.  For  linearly  non-separable  data,  SVMs \ncan (nonlinearly)  map the input to a high dimensional feature  space where a  linear \nhyperplane  can be found.  Although  there  is  no  guarantee that  a  linear  separable \nsolution will always exist in the high dimensional space, in practice it is quite feasible \nto construct  a working solution. \n\nGiven  a  labeled  set  of M  training  samples  (Xi , Yi),  where  X i  E  RN  and  Yi  is  the \nassociated label  (Yi  E {-I, I}), a  SVM  classifier finds  the optimal hyperplane that \ncorrectly  separates  (classifies)  the  data  points  while  maximizing  the  distance  of \neither class from  the hyperplane  (the margin).  Vapnik  [19]  shows that maximizing \nthe margin distance is  equivalent to minimizing the VC  dimension  in  constructing \nan optimal hyperplane.  Computing the best  hyperplane is  posed  as  a  constrained \noptimization  problem  and  solved  using  quadratic  programming  techniques.  The \ndiscriminant  hyperplane is  defined  by the level set of \n\nM \n\nf(x) = L  Yi  Q:i  .  k(x, Xi) + b \n\ni=l \n\nwhere  k( \u00b7, \u00b7)  is  a  kernel function  and the sign  of  f(x)  determines  the  membership \nof x.  Constructing  an optimal hyperplane is  equivalent  to finding  all  the nonzero \nQ:i .  Any  vector  Xi  that  corresponds to  a  nonzero  Q:i  is  a  supported  vector  (SV)  of \nthe optimal hyperplane.  A desirable feature of SVMs is that the number of training \npoints which  are retained as support vectors is  usually  quite small,  thus providing \na  compact classifier. \n\nFor  a  linear  SVM,  the  kernel  function  is  just  a  simple  dot  product  in  the  input \nspace while the kernel function in a nonlinear SVM effectively projects the samples \nto  a  feature  space  of higher  (possibly infinite)  dimension  via a  nonlinear  mapping \nfunction: \n\n~ : RN ~ F M ,  M\u00bb N \n\nand  then  constructs  a  hyperplane  in  F.  The  motivation  behind  this  mapping  is \nthat it makes possible a larger class of discriminative functions with which to find  a \nlinear hyperplane in the high dimensional feature space.  Using Mercer's theorem [7], \nthe expensive calculations required in projecting samples into the high dimensional \nfeature  space  can  be  replaced  by  a  much  simpler  kernel  function  satisfying  the \ncondition \n\nk(X,Xi) = ~(x) . ~(Xi) \n\nwhere  ~ is  the  implicit  nonlinear  projection.  Several  kernel  functions,  such  as \npolynomials and radial basis functions, have been shown to satisfy Mercer's theorem \n\n\fI--_L-__ ~_ \n\nFigure 2:  Automatic face  alignment system  [13]. \n\nFigure 3:  Some processed FE RET faces,  at high resolution \n\nand  have  been  used  successfully  in  nonlinear  SVMs.  In  fact,  by  using  different \nkernel  functions,  SVMs  can  implement  a  variety  of  learning  machines,  some  of \nwhich coincide with classical architectures.  Nevertheless, automatic selection of the \n\"right\"  kernel function  and  its  associated  parameters remains  problematic  and  in \npractice one must  resort to trial and error for  model selection. \n\nA radial basis function  (RBF) network is also a kernel-based technique for improved \ngeneralization, but it is based instead on regularization theory [17].  The number of \nradial  bases  and  their  centers  in  a  conventional  RBF  network  is  predetermined, \noften  by  k-means  clustering. \nIn  contrast,  a  SVM  with  the  same  RBF  kernel \nwill  automatically  determine  the  number  and  location  of  the  centers,  as  well  as \nthe  weights  and  thresholds  that  minimize  an  upper  bound  on  the  expected  risk. \n[9]  have  shown  that  both  SVMs  and  RBFs  can  be \nRecently,  Evgeniou  et  al. \nformulated under a unified framework in the context of Vapnik's theory of statistical \nlearning  [19].  As  such, SVMs  provide a  more systematic approach to classification \nthan classical RBF and various other neural networks. \n\n3  Experiments \n\nIn  our  study,  256-by-384  pixel  FERET  \"mug-shots\"  were  pre-processed  using  an \nautomatic face-processing system which  compensates for  translation,  scale,  as  well \nas  slight  rotations.  Shown  in  Figure  2,  this  system  is  described  in  detail  in \n[13]  and  uses  maximum-likelihood  estimation  for  face  detection,  affine  warping \nfor  geometric  shape  alignment  and  contrast  normalization  for  ambient  lighting \nvariations.  The  resulting  output  \"face-prints\"  in  Figure  2  were  standardized  to \n80-by-40  (full)  resolution.  These  \"face-prints\"  were  further  sub-sampled to  21-by-\n12  pixel  \"thumbnails\"  for  our  low  resolution  experiments.  Figure  3  shows  a  few \nexamples  of processed  face-prints  (note  that  these  faces  contain  little  or  no  hair \ninformation).  A total of 1755 thumbnails  (1044  males  and  711  females)  were  used \n\n\fClassifier \n\nSVM with Gaussian RBF kernel \nSVM with Cubic polynomial kernel \nLarge ensemble-RBF \nClassical RBF \nQuadratic classifier \nFisher linear discriminant \nNearest neighbor \nLinear classifier \n\nError Rate \n\nFemale \nOverall  Male \n3.38%  2.05% \n4.79% \n4.88%  4.21%  5.59% \n4.59% \n5.54% \n6.55% \n8.75% \n6.89% \n7.79% \n11.88% \n10.63% \n9.44% \n13.03%  12.31% \n13.78% \n28.04% \n26.53% \n27.16% \n58.95% \n58.47% \n59.45% \n\nTable 1:  Experimental results with thumbnails. \n\nin  our  experiments.  For  each  classifier,  the  average error rate  was  estimated  with \ni. e.,  a  5-way  dataset  split,  with  4/5th  used  for \n5-fold  cross  validation  (CV)  -\ntraining and  1/ 5th used for  testing,  with  4  subsequent  non-overlapping rotations. \nThe average size of the training set  was  1496 (793 males  and 713 females)  and the \naverage size of the test set was 259  (133  males and  126 females) . \n\nSVM wI RBF kernel  \u2022 \n\nSVM wI cubic poly. kernel -\nLarge ensemble of RBF -Classical RBF -\n\nQuadratic classifier \n\nFisher linear discriminant \n\nNearest neighbor \n\nLi near classifier \n\n0 \n\n10 \n\n20 \n\n30 \n\n40 \n\n50 \n\n60 \n\nError Rate \n\nFigure 4:  Error rates of various classifiers \n\nThe SVM classifier was first tested with various kernels in order to explore the space \nof models and performance.  A Gaussian RBF kernel was found to perform the best \n(in terms of error rate), followed by a cubic polynomial kernel as second best.  In the \nlarge ensemble-RBF experiment, the number of radial bases was incremented until \nthe error fell  below a set threshold.  The average number of radial bases in the large \nensemble-RBF was found  to be 1289 which corresponds to 86%  of the training set. \nThe number of radial bases for  classical  RBF networks was  heuristically  set  to  20 \n\n\fprior to actual training and testing.  Quadratic,  Linear  and  Fisher  classifiers  were \nimplemented  using  Gaussian  distributions  and  in  each  case  a  likelihood  ratio test \nwas  used for  classification.  The average error rates of all the classifiers tested with \n21-by-12  pixel thumbnails are reported in  Table  1 and summarized in  Figure 3. \n\nThe  SVMs  out-performed  all  other  classifiers,  although  the  performance  of large \nensemble-RBF  networks was  close  to  SVMs.  However,  nearly  90%  of the training \nset was retained as radial bases by the large ensemble-RBF. In contrast, the number \nof support vectors found  by both SVMs was only about 20% of the training set.  We \nalso  applied SVMs to classification based on high resolution images.  The Gaussian \nand cubic kernel performed equally well  at both low  and high resolutions with only \na  slight  1%  error  rate  difference.  We  note  that  as  indicated  in  Table  1,  all  the \nclassifiers had higher error rates in classifying females,  most likely due to the general \nlack of prominent and distinct facial features in female faces,  as compared to males. \n\n4  Discussion \n\nWe  have  presented  a  comprehensive  evaluation  of various  classification  methods \nfor  determination of sex from  facial  images.  The non-triviality of this task  (made \neven harder by our  \"hairless\" low resolution faces)  is  demonstrated by the fact that \na  linear  classifier  had  an  error  rate  of 60%  (i.e.,  worse  than  a  random  coin  flip). \nFurthermore, an acceptable error rate \u00ab  5%)  for  the large ensemble-RBF network \nrequired storage of 86%  of the training set  (SVMs required about 20%).  Storage of \nthe entire dataset in  the form  of the nearest-neighbor classifier yielded too high an \nerror rate  (30%).  Clearly,  SVMs  succeeded  in  the  difficult  task of finding  a  near(cid:173)\noptimal class partition in face  space with the added economy of a  small number of \nsupport faces. \n\nGiven  the  relative  success  of  previous  studies  with  low  resolution  faces  it  is  re(cid:173)\nassuring  that  21-by-12  faces  can,  in  fact,  be  used  for  reliable  sex  classification. \nHowever,  most  of  the  previous  studies  used  datasets  of  relatively  few  faces, \nconsequently  with  little  statistical  significance  in  the  reported  results.  The  most \ndirectly  comparable  study  to  ours  is  that  of  Gutta  et  al.  [11],  which  also  used \nFERET faces.  With a dataset of 3000 faces at a resolution of 64-by-72, their hybrid \nRBF /Decision-Tree classifier achieved a 4% error rate.  In our study, with 1800 faces \nat a resolution of 21-by-12, a Gaussian kernel SVM was able to achieve a 3.4%  error \nrate.  Both studies use extensive  cross validation to estimate the error rates.  Given \nour results with SVMs, it is  clear that better performance at even lower resolutions \nis  possible with this learning technique. \n\nReferences \n\n[1]  V.  Bruce,  A.  M.  Burton,  N.  Dench,  E.  Hanna,  P.  Healey,  O.  Mason,  A.  Coombes, \nR.  Fright, and A.  Linney.  Sex discrimination:  How  do we  tell the difference  between \nmale and female  faces?  Perception,  22:131- 152,  1993. \n\n[2]  R.  Brunelli and T. Poggio.  Hyperbf networks for gender classification.  In Proceedings \n\nof the  DARPA  Image  Understanding  Workshop,  pages  311-314,  1992. \n\n\f[3]  R.  Brunelli  and  T.  Poggio.  Face  recognition:  Features  vs.  templates. \n\nIEEE \n\nTransactions  on  Pattern  Analysis  and  Machine  Intelligence,  15(10),  October 1993. \n\n[4]  A.  M.  Burton,  V.  Bruce,  and  N.  Dench.  What's  the  difference  between  men  and \n\nwomen?  evidence from facial  measurement.  Perception,  22:153- 176,  1993. \n\n[5]  C.  Cortes and V.  Vapnik.  Support vector networks.  Machine  Learning,  20,  1995. \n\n[6]  Garrison W.  Cottrell.  Empath:  Face,  emotion,  and gender recognition  using  holons. \n\nIn Advances  in Neural  Information Processing  Systems,  pages  564-571,  1991. \n\n[7]  R. Courant and D. Hilbert.  Methods  of Mathematical Physiacs, volume 1.  Interscience, \n\nNew-York,  1953. \n\n[8]  B.  Edelman,  D.  Valentin,  and H.  Abdi.  Sex classification  of face  areas:  how well  can \na  linear  neural  network  predict  human performance.  Journal  of Biological  System, \n6(3):241- 264,  1998. \n\n[9]  Theodoros  Evgeniou,  Massimiliano Pontil,  and Tomaso Poggio.  A unified framework \nfor regularization networks and support vector machines.  Technical Report AI Memo \nNo.  1654, MIT,  1999. \n\n[10]  B.  A.  Golomb,  D.  T.  Lawrence,  and  T.  J.  Sejnowski.  Sexnet:  A  neural  network \nIn  Advances  in  Neural  Information  Processing \n\nidentifies  sex  from  human  faces. \nSystems,  pages  572-577,  1991. \n\n[11]  S.  Gutta,  H.  Wechsler,  and  P.  J.  Phillips.  Gender  and  ethnic  classification. \n\nIn \nProceedings  of the IEEE International Automatic Face  and Gesture Recognition, pages \n194-199,  1998. \n\n[12]  J.  Huang,  X.  Shao,  and H.  Wechsler.  Face  pose  discrimination  using support vector \nmachines.  In Proc.  of 14th Int'l Conf.  on Pattern Recognition  (ICPR '98),  pages  154-\n156,  August 1998. \n\n[13]  B.  Moghaddam  and  A.  Pentland.  Probabilistic  visual  learning  for  object  represen(cid:173)\ntation.  IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence,  PAMI-\n19(7):696- 710,  July  1997. \n\n[14]  E. Osuna, R. Freund, and F. Girosi.  Training support vector machines:  an application \nIn  Proceedings  of  the  IEEE  Computer  Society  Conference  on \n\nto  face  detection. \nComputer  Vision  and Pattern Recognition,  pages  130-136,  1997. \n\n[15]  A.  J.  O'Toole,  T. Vetter,  N.  F.  Troje,  and H.  H.  Bulthoff.  Sex classification  is  better \nwith three-dimensional structure than with image  intensity information.  Perception, \n26:75- 84,  1997. \n\n[16]  P.  J.  Phillips.  Support vector machines applied to face  recognition.  In M.  S.  Kearns, \nS. Solla,  and D. Cohen,  editors,  Advances  in Neural  Information  Processing  Systems \n11,  volume 11,  pages 803- 809.  MIT  Press,  1998. \n\n[17]  T.  Poggio  and  F.  Girosi.  Networks  for  approximation  and learning.  Proceedings  of \n\nthe  IEEE,  78(9):1481- 1497,  1990. \n\n[18]  S.  Tamura, H.  Kawai,  and H.  Mitsumoto.  Male/female identification from 8 x  6 very \nlow  resolution  face  images  by  neural  network.  Pattern  Recognition,  29(2):331- 335, \n1996. \n\n[19]  V.  Vapnik.  The  Nature  of Statistical Learning  Theory.  Springer,  1995. \n\n[20]  Laurenz  Wiskott,  Jean-Marc  Fellous,  Norbert  Kruger,  and  Christoph  von  der \nMalsburg. \nIn  Proceedings  of  the \nInternational  Workshop  on  Automatic  Face  and  Gesture  Recognition,  pages  92- 97, \n1995. \n\nFace  recognition  and  gender  determination. \n\n\f", "award": [], "sourceid": 1933, "authors": [{"given_name": "Baback", "family_name": "Moghaddam", "institution": null}, {"given_name": "Ming-Hsuan", "family_name": "Yang", "institution": null}]}