{"title": "General-Purpose Localization of Textured Image Regions", "book": "Advances in Neural Information Processing Systems", "page_first": 817, "page_last": 823, "abstract": null, "full_text": "General-purpose  localization  of textured \n\n\u2022 \nImage  regions \n\n\u2022 \n\nRutb Rosenboltz\u00b7 \n\nXeroxPARC \n\n3333 Coyote Hill Rd. \nPalo Alto, CA 94304 \n\nAbstract \n\nWe  suggest a working  definition  of texture:  Texture  is  stuff that is \nmore  compactly represented  by  its  statistics  than  by  specifying  the \nconfiguration  of  its  parts.  This  definition  suggests  that  to  fmd \ntexture  we  look  for  outliers  to  the  local  statistics,  and  label  as \ntexture  the  regions  with  no  outliers.  We  present a  method,  based \nupon  this  idea, for  labeling points  in  natural scenes  as  belonging to \ntexture  regions,  while  simultaneously  allowing  us  to  label  low(cid:173)\nlevel,  bottom-up  cues  for  visual  attention.  This  method  is  based \nupon  recent  psychophysics  results  on  processing  of texture  and \npopout. \n\n1  WHAT  IS  TEXTURE,  AND  WHY  DO  WE  WANT  TO \nFIND  IT? \n\nIn  a  number  of problems  in  computer  VlSlon  and  image  processing,  one  must \ndistinguish  between  image  regions  that  correspond  to  objects  and  those  which \ncorrespond to  texture,  and  perform different processing depending  upon  the  type  of \nregion.  Current  computer  vision  algorithms  assume  one  magically  knows  this \nregion  labeling.  But  what  is  texture?  We  have  the  notion  that texture  involves  a \npattern  that  is  somehow  homogeneous,  or  in  which  signal  changes  are  \"too \ncomplex\"  to  describe,  so  that  aggregate  properties  must  be  used  instead  (Saund, \n1998).  There is  by no means a firm  division between texture and  objects; rather, the \ncharacterization often depends upon  the scale of interest (Saund, 1998). \n\n\u2022 Email: rruth@parc.xerox.com \n\n\f818 \n\nR. Rosenholtz \n\nIdeally  the  defmition  of texture should probably  depend  upon  the  application.  We \ninvestigate  a  definition  that we  believe  will  be  of fairly  general  utility:  Texture  is \nstuff that  seems  to  belong  to  the  local  statistics.  We  propose  extracting  several \ntexture  features,  at  several  different  scales,  and  labeling  as  texture  those  regions \nwhose feature values are likely to have come from the local distribution. \nOutliers  to  the  local  statistics  tend  to  draw  our attention  (Rosenholtz,  1997,  1998). \nThe  phenomenon  is  often  referred  to  as  \"popout.\"  Thus  while  labeling  (locally) \nstatistically  homogeneous  regions  as  texture,  we  can  simultaneously  highlight \nsalient  outliers  to  the  local  statistics.  Our  revised  defmition  is  that  texture  is  the \nabsence of popout. \nIn  Section  2,  we  discuss  previous  work  in  both  human  perception  and  in  fmding \ntexture  and  regions  of interest  in  an  image. \nIn Section  3,  we  describe our method. \nWe present and discuss results on a number of real images in  Section 4. \n\n2  PREVIOUS  WORK \n\nSee  (Wolfe,  1998)  for  a review  of the  visual  search  literature.  Popout  is  typically \nstudied  using  simple  displays,  in  which  an  experimental  subject  searches  for  the \nunusual,  target  item,  among  the  other,  distractor  items.  One  typically  attempts  to \njudge  the  \"saliency,\"  or  degree  to  which  the  target  pops  out,  by  studying  the \nefficiency of search  for  that item.  Typically popout is  modeled by a relatively low(cid:173)\nlevel  operator,  which  operates  independently  on  a  number  of basic  features  of the \nIn this  paper,  we \nimage,  including  orientation,  contrast/color,  depth,  and  motion. \nlook only at the features of contrast and orientation. \nWithin  the  image-processing field,  much  of the work in  fmding  texture has  defmed \nas  texture  any region  with  a high luminance variance, e.g.  Vaisey &  Gersho (1992). \nUnfortunately, the luminance variance in  a region  containing  an  edge can  be as high \nas  that  in  a textured  region.  Won  &  Park  (1997)  use  model  fitting  to  detect  image \nblocks  containing  an  edge,  and  then  label  blocks  with  high  variance  as  containing \ntexture. \nRecently,  several  computer  vision  researchers  have  also  tackled  this  problem. \nLeung  &  Malik  (1996)  found  regions  of completely  deterministic  texture.  Other \nresearchers  have  used  the  defmition  that  if the  luminance  goes  up  and  then  down \nagain  (or vice  versa)  it's  texture  (Forsyth  et  aI,  1996).  However,  this  method  will \ntreat lines  as  if they were texture.  Also, with no notion of similarity within a texture \n(also  lacking  in  the  image-processing  work),  one would  mark  a  \"fault\"  in  a texture \nas  belonging  to  that  texture.  This  would  be  unacceptable  for  a  texture  synthesis \napplication,  in  which  a  routine  that  tried  to  synthesize  such  a  texture  would  most \nlikely  fail  to  reproduce  the  (highly  visible)  fault.  More  recently,  Shi  and  Malik \n(1998) presented a method for segmenting images based upon texture features.  Their \nmethod  performs  extremely  well  at  the  segmentation  task,  dividing  an  image  into \nregions  with  internal  similarity  that  is  high  compared  to  the  similarity  across \nregions.  However,  it  is  difficult  to  compare  with  their  results,  since  they  do  not \nexplicitly  label  a  subset  of  the  resulting  regions  as  texture.  Furthermore,  this \nmethod  may  also  tend  to  mark  a  \"fault\"  in  a  texture  as  belonging  to  that  texture. \nThis  is  both  because  the method  is  biased  against separating  out small regions,  and \nbecause  the  grouping  of  a  patch  with  one  region  depends  as  much  upon  the \ndifference  between  that  patch  and  other  regions  as  it  does  upon  the  similarity \nbetween the patch and the given region. \nVery  little  computer vision  work  has  been  done  on  attentional  cues.  Milanese  et al \n(1993)  found  salient image regions  using both  top-down  information  and  a bottom(cid:173)\nup \"conspicuity\" operator, which marks  a local region as  more salient the greater the \n\n\fGeneral-Purpose Localization o/Textured Image Regions \n\n819 \n\ndifference  between  a  local  feature  value  and  the  mean  feature  value  in  the \nsurrounding  region.  However.  for  the  same  difference  in  means.  a  local  region  is \nless  salient when  there  is  a greater variance  in  the feature  values  in  the surrounding \nregion  (Duncan  &  Humphreys.  1989;  Rosenholtz.  1997).  We  use  as  our  saliency \nmeasure a test for  outliers to  the local distribution. This captures.  in many cases. the \ndependence  of saliency  on  difference  between  a  given  feature  value  and  the  local \nmean. relative to the  local standard deviation.  We will discuss our saliency measure \nin greater detail in the following section. \n\n3  FINDING  TEXTURE  AND  REGIONS  OF  INTEREST \n\nWe  compute multiresolution  feature maps  for orientation and contrast. and then  look \nfor  outliers  in  the  local  orientation  and  contrast  statistics.  We  do  this  by  fast \ncreating  a  3-level  Gaussian  pyramid  representation  of  the  image.  To  extract \ncontrast. we  filter  the  pyramid with  a difference  of circularly  symmetric Gaussians. \nThe  response  of these  filters  will  oscillate.  even  in  a  region  with  constant-contrast \ntexture  (e.g.  a  sinewave pattern).  We  approximate  a  computation  of the  maximum \nresponse  of these  filters  over  a  small  region  by  fast squaring  the  filter  responses. \nand  then  filtering  the  contrast  energy  with  an  appropriate  Gaussian.  Finally.  we \nthreshold  the  contrast  to  eliminate  low-contrast  regions  (\"flat\"  texture).  These \nthresholds  (one  for  each  scale)  were  set  by  examining  the  visibility  of sinewave \npatterns of various spatial frequencies. \nWe  compute orientation in  a simple and biologically plausible way.  using Bergen & \nLandy's (1991) \"back pocket model\" for low-level computations: \n1.  Filter the pyramid with  horizontal.  vertical.  and \u00b145\u00b0 oriented  Gaussian second \n\nderivatives. \n\n2.  Compute  opponent  energy  by  squaring  the  filter  outputs.  pooling  them  over  a \nregion  4  times  the  scale  of the  second  derivative  filters.  and  subtracting  the \nvertical from the horizontal response and the +45 0  from the _45 0  response. \n\n3.  Normalize the opponent energy  at each scale  by  dividing  by  the total  energy  in \n\nthe 4 orientation energy bands at that scale. \n\nThe result is  two  images at each  scale of the pyramid.  To  a good approximation.  in \nregions  which  are  strongly  oriented.  these  images  represent  kcos(26)  and  ksin(26). \nwhere 6 is  the local orientation  at that scale.  and k  is  a value  between 0 and  1 which \nis  related  to  the local orientation  specificity.  Orientation  estimates from  points with \nlow  specificity  tend  to  be  very  noisy. \nIn  images  of  white  noise.  80%  of  the \nestimates  of  k  fall  below  0.5.  therefore  with  80%  confidence.  an  orientation \nspecificity  of k>0.5  did not occur due to chance.  We use this  value to  threshold out \norientation estimates with low \"orientedness.\u00b7\u00b7 \nWe  then  estimate D,  the  local  feature  distribution,  for  each  feature  and  scale,  using \nthe  method  of Parzen  windows.  The  blurring  of the  distribution  estimate  by  the \nParzen  window  mimics  uncertainty  in  estimates  of  feature  values  by  the  visual \nsystem.  We  collect statistics over a local integration region.  For texture processing. \nthe  size  of this  region  is  ind.ependent  of viewing  distance,  and  is  roughly  lOS  in \ndiameter, where S is  the support of the Gaussian 2nd  derivative filters  used to  extract \nthe texture features (Kingdom & Keeble,  1997; Kingdom et ai,  1995). \nWe next compute a non-parametric measure of saliency: \n\nsaliency  =  -IO~ \n\n% \n\n(  P(v ID) \n\n, \n) \nmaxP(x ID) \n\n(1) \n\n\f820 \n\nNote that if D were Gaussian  N(~,a2), this simplifies to \n\n(X_tt)2 \n'1a2 \n\nR.  Rosenholtz \n\n(2) \n\nwhich  should  be  compared  to  the  standard  parametric  test  for  outliers,  which  uses \nthe  measure (x - tt)/ a.  Our  saliency  measure  is  essentially  a  more  general,  non-\nparametric form of this measure (i.e. it does not assume a Gaussian distribution). \nPoints with saliency less  than  0.5  are labeled  as  candidate texture points.  If D were \nGaussian,  this  would  correspond  to  feature  estimates  within  one  standard  deviation \nof the  mean.  Points  with  saliency  greater  than  3.1  are  labeled  as  candidates  for \nbottom-up  attentional  cues.  If D  were  Gaussian,  this  would  correspond  to  feature \nestimates  more  than  2.50  from  the  mean,  a  standard  parametric  test  for  outliers. \nOne  could,  of course,  keep  the  raw  saliency values,  as  a measure  of the  likelihood \nthat a region  contained texture,  rather than  setting  a hard  threshold.  We  use  a hard \nthreshold  in  our  examples  to  better  display  the results.  Both the  texture  images  and \nthe region of interest images are median-filtered to remove extraneous points. \n\n4  EXPERIMENTAL  RESULTS \n\nFigure 3 shows  several  example images.  Figures  2,  3,  and 4  show  texture  found  at \neach scale of processing.  The striped and checkered patterns represent oriented and \nhomogeneous  contrast  texture,  respectively.  The  absence  of an  image  in  any  of \nthese figures  means that no texture  of the given  type was  found  in  that image at the \ngiven scale.  Note that we perform no segmentation of one texture from another. \nFor the building image, the algorithm labeled bricks  and window panes  as  fme-scale \ntexture,  and  windows  and  shutters  as  coarser-scale  texture.  The  leopard  skin  and \nlow-frequency  stripes  in  the  lower right comer of the  leopard  image  were correctly \nlabeled  as  texture.  In the  desk  image,  the  \"wood\" texture was  correctly  identified. \nThe  regular  pattern  of windows  were  marked  as  texture  in  the  hotel  image. \nIn the \nhouse  image,  the  wood  siding,  trees,  and  part  of the  grass  were  labeled  as  texture \n(much  of the  grass  was  low  contrast  and  labeled  as  \"flat\"  texture).  One  of the \nbushes  is  correctly  identified  as  having  coarser  texture  than  the  other  has. \nIn  the \nlighthouse image,  the  house sans window,  fence,  and tower were marked,  as  well as \na low-frequency oriented pattern in the clouds. \nFigure 5 shows the regions of interest that were  found (the striped and plaid patterns \nhere  have  no  meaning  but  were  chosen  for  maximum  visibility).  Most  complex \nnatural  scenes  had  few  interesting  low-level  attentional  areas.  In  the  lighthouse \nimage,  the  life  preserver  is  marked. \nIn  the  hotel,  curved  or  unusual  angular \nwindows  are  identified  as  attentional  cues,  as  well  as  the top  of the  building.  Both \nof these results are in  agreement with psychophysical results showing that observers \nquickly  identify  curved  or  bent  lines  among  straight  lines  (reviewed  in  Wolfe, \n1998).  The  simpler  desk  scene  yields  more  intuitive  results,  with  each  of the  3 \nobjects labeled, as well as the phone cord. \nBottom-up  attentional  cues  are  outliers  to  the  local  distribution  of features,  and  we \nhave  suggested that texture is  the  absence of such  outliers.  This  definition captures \nsome  of the  intuition  that  texture  is  homogeneous  and  statistical  in  nature.  We \npresented a method  for  fmding  contrast and  orientation  outliers, and results  both  on \nlocalizing  texture  and  on  finding  popout  in  natural  images.  For  the  simple  desk \nimage,  the  algorithm highlights  salient regions  that correspond to  our notions of the \nimportant  objects  in  the  scene.  On  complicated  natural  scenes,  its  results  are  less \nintuitive;  suggesting  that  search  in  natural  scenes  makes  use  of  higher-level \n\n\fGeneral-Purpose Localization o/Textured Image Regions \n\n821 \n\nprocessing  such  as  grouping  into  objects.  This  result  should  not  be  terribly \nsurprising,  but  serves  as  a  useful  check  on  simple  low-level  models  of  visual \nattention.  The  algorithm  does  a  good  job  of  identifying  textured  regions  at  a \nnumber of different scales, with the results perhaps more intuitive at finer scales. \n\nAcknowledgments \n\nThis  work was  partially  supported  by  an  NRC postdoctoral  award  at NASA  Ames. \nMany thanks to David Marimont and Eric Saund for useful discussions. \n\nReferences \n\nJ.  R.  Bergen  and  M.  S.  Landy  (1991),  \"Computational  modeling  of visual  texture \nsegmentation,\"  Computational  Models  of Visual  Processing,  Landy  and  Movshon \n(eds.), pp. 252-271, MIT Press, Cambridge, MA. \nJ.  Duncan  and  G.  Humphreys  (1989),  \"Visual  search  and  stimulus  similarity,\" \nPsych. Review 96, pp. 433-458. \nD.  Forsyth,  J.  Malik,  M.  Fleck,  H.  Greenspan,  T.  Leung,  S.  Belongie,  C.  Carson, \nand C.  Bregler (1996), \"Finding pictures of objects in collections  of images,\" ECCV \nWorkshop on  Object Representation, Cambridge. \nF.  A.  A.  Kingdom,  D.  Keeble,  D.,  and  B.  Moulden  (1995),  \"Sensitivity  to \norientation modulation in micropattern-based textures,\" Vis.  Res. 35, 1, pp. 79-91. \nF.  A.  A.  Kingdom  and  D.  Keeble  (1997),  \"The  mechanism  for  scale  invariance  in \norientation-defined textures.\" Invest.  Ophthal. and Vis. Sci.  (Suppl.) 38, 4, p. 636. \nT.  K.  Leung  and  J.  Malik  (1996),  \"Detecting,  localizing,  and  grouping  repeated \nscene  elements  from  an  image,\"  Proc.  4th  European  Con!  On  Computer  Vision, \n1064, 1, pp. 546-555, Springer-Verlag, Cambridge. \nR.  Milanese,  H.  Wechsler,  S.  Gil,  J.  -M.  Bost,  and  T.  Pun  (1993),  \"Integration  of \nbottom-up  and  top-down  cues  for  visual  attention  using  non-linear  relaxation,\" \nProc.  IEEE CVPR, pp. 781-785, IEEE Computer Society Press, Seattle. \nR.  Rosenholtz (1997),  \"Basic signal detection  theory model  does  not explain  search \namong  heterogeneous  distractors.\"  Invest.  Ophthal.  and  Vis.  Sci.  (Suppl.)  38,  4,  p. \n687. \nR.  Rosenholtz  (1998),  \"A  simple  saliency  model  explains  a  number  of  motion \npopout phenomena.\" Invest.  Ophthal. and Vis.  Sci. (Suppl.) 39,4, p. 629. \nE.  Saund  (1998),  \"Scale  and  the  ShapelTexture  Continuum,\"  Xerox  Internal \nTechnical Memorandum. \nJ.  Shi and J. Malik (1998), \"SelfInducing Relational Distance and  its Application to \nImage  Segmentation,\" Proc.  jth  European  Con!  on  Computer Vision,  Burkhardt and \nNeumann (eds.), 1406, 1, pp. 528-543, Springer, Freiburg. \nJ.  Vaisey  and  A.  Gersho  (1992),  \"Image  compression  with  variable  block  size \nsegmentation.\" IEEE Trans.  Signal Processing 40,8, pp. 2040-2060. \nJ.  M.  Wolfe  (1998),  \"Visual search:  a  review,\" Attention,  H.  Pashler  (ed.),  pp.  13-\n74, Psychology Press Ltd., Hove, East Sussex, UK. \nC.  S.  Won  and  D.  K.  Park  (1997),  \"Image  block  classification  and  variable  block \nsize segmentation using a model-fitting criterion,\" Opt.  Eng.  36, 8, pp. 2204-2209. \n\n\f822 \n\nR.  Rosenholtz \n\nFigure  I: Original images. \n\n(a) \n\n(b) \n\nFigure 2:  Fine-scale texture. (a) oriented texture, (b) homogeneous contrast \n\ntexture. \n\n- . ~\" \n\n. \n-\n\n~ \n\n1\". \n\n,  .~ \u2022\n\n. .  ' \n\n& \n\n.... \n\u2022 \u2022 I. , . \n-.. \n.. '  .. \n\n\u2022 \n\n(a) \n\n(b) \n\nFigure 3:  Medium-scale texture. (a)  oriented texture, (b) homogeneous contrast \n\ntexture. \n\n\fGeneral-Purpose Localization of Textured Image Regions \n\n823 \n\nFigure 4:  Coarse-scale texture. (a) oriented texture, (b) homogeneous contrast \n\ntexture . \n\n(a) \n\n(b) \n\nI, \n\n..  . \n\u2022 \u2022 \n\n,\n\nFigure 5:  Regions of interest. \n\n\f", "award": [], "sourceid": 1540, "authors": [{"given_name": "Ruth", "family_name": "Rosenholtz", "institution": null}]}