{"title": "Grouping Contours by Iterated Pairing Network", "book": "Advances in Neural Information Processing Systems", "page_first": 335, "page_last": 341, "abstract": "", "full_text": "Grouping Contours by Iterated Pairing Network \n\nAmnon Shashua \nM.I.T. Artificial Intelligence Lab., NE43-737 \nand Department of Brain and Cognitive Science \nCambridge, MA 02139 \n\nAbstract \n\nShimon Ullman \n\nWe describe in this paper a network that performs grouping of image con(cid:173)\ntours. The input to the net are fragments of image contours, and the \noutput is the partitioning of the fragments into groups, together with a \nsaliency measure for each group. The grouping is based on a measure of \noverall length and curvature. The network decomposes the overall opti(cid:173)\nmization problem into independent optimal pairing problems performed \nat each node. The resulting computation maps into a uniform locally \nconnected network of simple computing elements. \n\n1 The Problenl: Contour Grouping \n\nA problem that often arises in visual information processing is the linking of con(cid:173)\ntour fragments into optimal groups. For example, certain subsets of contours spon(cid:173)\ntaneously form perceptual groups, as illustrated in Fig. 1, and are often detected \nimmediately without scanning the image in a systematic manner. Grouping process \nof this type are likely to play an important role in object recognition by segmenting \nthe image and selecting image structures that are likely to correspond to objects of \ninterest in the scene. \n\n'Ve propose that some form of autonomous grouping is performed at an early stage \nbased on geometrical characteristics, that are independent of the identity of objects \nto be selected. The grouping process is governed by the notion of saliency in a way \nthat priority is given to forming salient groups at the expense of potentially less \nsalient ones. This general notion can again be illustrated by Fig. 1; it appears that \ncertain groups spontaneously emerge, while grouping decisions concerning the less \nsalient parts of the image may remain unresolved. As we shall see, the computation \nbelow exhibits a similar behavior. \n\nWe define a grouping of the image contours as the formation of a set of disjoint \n\n335 \n\n\f336 \n\nShashua and Ullman \n\n\" \n\n'....,-- ... .,.,\"''''4.-.... \n\n~ \\..-\n...... ,' \",-:'--.., ... ' \nI':: \n\n.... :. '\"/...,,) \n\n'''\" '- '\".-1 ;' 'I )_' \n'>,,\" I'........ I'~ '\\ \nI \n\\. ..... ' \n' \n\n' \nA \n\nI \n\nI \n\nl \n\n.,.; \" ' : ; , \" \" ' \" \n\n.... \n~\"\"\\ \n_I \n\n\" ..... ' \n-\",(, - ~ ,~ \" ~/ -\nI \n... \" \n-\n)' , \n\" \nt:' \n~ .''''' \n.... I ' \"\"J I \n... \n,~.... \nr\" / f \nI \n-\nh \nll..' ... :\" \n\n.... ' \n\n\" \n\" \n\\ \\ , -\n.... \n\n\\ \n\n.......... \n\n\" \n,,(I \n~ .... I \n\nJI\" \n\n'-.... _ ~ \nI,. .... ,~,,,.' '''} ... - 1 .... \"( ,,1-\\ \n.....)' \n... / 1 \n- I .... \n\n~ ..... , \n~ \nI \n\n......... X3Wl, (ii) \nX2W3 > XlW2 and (iii) XIWI > X2W3. From (ii) and (iii) we get an inequality that \ncontradicts (i). For the induction hypothesis, assume the claim holds for arbitrary \nk - 1. We must show that the claim holds for k . Given the induction hypothesis \nwe must show that there is no selection pattern that will give rise to a cycle of \nsize k. Assume in contradiction that such a cycle exists. For any given cycle of \nsize k we can renumber t.he indecis such that bi = i + 1 and bk = 1 which implies \nthat XiWi > XjWij for all j =/; i. In particular we have the following k inequalities: \nXiWi > Xi-2Wi-l where i = 1, ... , k. From the k - 1 inequalities corresponding \nto i = 2, ... , k we get, by transitivity, that XIWI < Xk-lWk which contradicts the \nremaining inequality that corresponds to i = 1. 0 \n3.3 Summary of Computation \n\nThe optimization is ma.pped onto a locally connected network with a simple uniform \ncomputation. The computation consists of the following steps. (i) Compute the \nsaliency S~ of each line element using the computation defined in (1). (ii) At each \nnode perform a pairing of the line elements at the node. The pairing is performed \nby repeatedly selecting mutual neighbors. (iii) Update at each node the values Sn \nbased on the newly formed pa.iring (eq. 2). (iv) Go back to step 2. \n\nThese iterated pairings allow pairing decisions to propagate along maximally salient \ncurves and influence other pairing decisions . In the implementation, the number of \niterations n is equal in both stages and as n increases, the finer the pairing would \nbe, resulting in a finer discrimination between groups. During the computation, the \nmore salient groups emerge first, the less salient groups require additional iterations. \nAlthough the process is not guaranteed to converge to an optimal solution, it is a \nvery simple computation that yields in practice good results. Some examples are \nshown in the next section. \n\n\fGrouping Contours by Iterated Pairing Network \n\n341 \n\no \n\nFigure 2: Results after 30 iterations of saliency and pairing on a net of size 128 X 128 with \n16 elements per node. Images from left to right display the saliency map following the \nsaliency and pairing stages and a number of strongest groups. The saliency of elements in \nthe display is represented in terms of brightness and width -\nincreased saliency measure \ncorresponds to increase in brightness and width of element in display. \n\n3.4 Examples \n\nFig. 2 shows the results of the network applied to the images in Fig. 1. The \nsaliency values following the saliency and pairing stages illustrate that perceptually \nsalient curves are also associated with high saliency values (see also [3]). Finally, \nin these examples, the highest saliency value of each group has been propagated \nalong all elements of the group such that each group is now associated with a single \nsaliency value. A number of strongest groups has been pulled out showing the close \ncorrespondence of these groups to objects of interest in the images. \n\nAcknowledgments \n\nThis work was supported by NSF grant IRI-8900267. Part of the work was done \nwhile A.S. was visiting the exploratory vision group at IBM research center, York(cid:173)\ntown Heights. \n\nReferences \n\n[1] J. Edmonds. Path trees and flowers. Can. J. Math., 1:263-271, 1965. \n[2] C.H. Papadimitriou and K. Steiglitz. Combinatorial Optimization: Algorithms \n\nand Complexity. Prentice-Hall, New Jersey, 1982. \n\n[3] A. Shashua and S. Ullman. Structural saliency: The detection of globally salient \n\nstructures using a locally connected network. In Proceedings of the 2nd Inter(cid:173)\nnational Conference on Computer Vision, pages 321-327, 1988. \n\n\f", "award": [], "sourceid": 438, "authors": [{"given_name": "Amnon", "family_name": "Shashua", "institution": null}, {"given_name": "Shimon", "family_name": "Ullman", "institution": null}]}