{"title": "Normalized Spectral Map Synchronization", "book": "Advances in Neural Information Processing Systems", "page_first": 4925, "page_last": 4933, "abstract": "The algorithmic advancement of synchronizing maps is important in order to solve a wide range of practice problems  with possible large-scale dataset. In this paper, we provide theoretical justifications for spectral techniques for the map synchronization problem, i.e., it takes as input a collection of objects and noisy maps estimated between pairs of objects, and outputs clean maps between all pairs of objects. We show that a simple normalized spectral method that projects the blocks of the top eigenvectors of a data matrix to the map space leads to surprisingly good results. As the noise is modelled naturally as random permutation matrix, this algorithm NormSpecSync leads to competing theoretical guarantees as state-of-the-art convex optimization techniques, yet it is much more efficient. We demonstrate the usefulness of our algorithm in a couple of applications, where it is optimal in both complexity and exactness among existing methods.", "full_text": "Normalized Spectral Map Synchronization\n\nYanyao Shen\nUT Austin\n\nAustin, TX 78712\n\nQixing Huang\n\nshenyanyao@utexas.edu\n\nhuangqx@cs.utexas.edu\n\nTTI Chicago and UT Austin\n\nAustin, TX 78712\n\nNathan Srebro\nTTI Chicago\n\nChicago, IL 60637\nnati@ttic.edu\n\nSujay Sanghavi\n\nUT Austin\n\nAustin, TX 78712\n\nsanghavi@mail.utexas.edu\n\nAbstract\n\nEstimating maps among large collections of objects (e.g., dense correspondences\nacross images and 3D shapes) is a fundamental problem across a wide range of\ndomains. In this paper, we provide theoretical justi\ufb01cations of spectral techniques\nfor the map synchronization problem, i.e., it takes as input a collection of objects\nand noisy maps estimated between pairs of objects along a connected object graph,\nand outputs clean maps between all pairs of objects. We show that a simple\nnormalized spectral method (or NormSpecSync) that projects the blocks of the top\neigenvectors of a data matrix to the map space, exhibits surprisingly good behavior\n\u2014 NormSpecSync is much more ef\ufb01cient than state-of-the-art convex optimization\ntechniques, yet still admitting similar exact recovery conditions. We demonstrate\nthe usefulness of NormSpecSync on both synthetic and real datasets.\n\n1\n\nIntroduction\n\nThe problem of establishing maps (e.g., point correspondences or transformations) among a collection\nof objects is connected with a wide range of scienti\ufb01c problems, including fusing partially overlapped\nrange scans [1], multi-view structure from motion [2], re-assembling fractured objects [3], analyzing\nand organizing geometric data collections [4] as well as DNA sequencing and modeling [5]. A\nfundamental problem in this domain is the so-called map synchronization, which takes as input noisy\nmaps computed between pairs of objects, and utilizes the natural constraint that composite maps\nalong cycles are identity maps to obtain improved maps.\nDespite the importance of map synchronization, the algorithmic advancements on this problem remain\nlimited. Earlier works formulate map synchronization as solving combinatorial optimizations [1, 6, 7,\n8]. These formulations are restricted to small-scale problems and are susceptible to local minimums.\nRecent works establish the connection between the cycle-consistency constraint and the low-rank\nproperty of the matrix that stores pairwise maps in blocks; they cast map synchronization as low-rank\nmatrix inference [9, 10, 11]. These techniques exhibit improvements on both the theoretical and\npractical sides. In particular, they admit exact recovery conditions (i.e., on the underlying maps can\nbe recovered from noisy input maps). Yet due to the limitations of convex optimization, all of these\nmethods do not scale well to large-scale datasets.\nIn contrast to convex optimizations, we demonstrate that spectral techniques work remarkably well\nfor map synchronization. We focus on the problem of synchronizing permutations and introduce\na robust and ef\ufb01cient algorithm that consists of two simple steps. The \ufb01rst step computes the top\neigenvectors of a data matrix that encodes the input maps, and the second step rounds each block of\n\n30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain.\n\n\fthe top-eigenvector matrix into a permutation matrix. We show that such a simple algorithm possesses\na remarkable denoising ability. In particular, its exact recovery conditions match the state-of-the-art\nconvex optimization techniques. Yet computation-wise, it is much more ef\ufb01cient, and such a property\nenables us to apply the proposed algorithm on large-scale dataset (e.g., many thousands of objects).\nSpectral map synchronization has been considered in [12, 13] for input observations between all\npairs of objects. In contrast to these techniques, we consider incomplete pair-wise observations, and\nprovide theoretical justi\ufb01cations on a much more practical noise model.\n\n2 Algorithm\n\nIn this section, we describe the proposed algorithm for permutation synchronization. We begin with\nthe problem setup in Section 2.1. Then we introduce the algorithmic details in Section 2.2.\n\n2.1 Problem Setup\nSuppose we have n objects S1,\u00b7\u00b7\u00b7 , Sn. Each object is represented by m points (e.g., feature points\non images and shapes). We consider bijective maps \u03c6ij : Si \u2192 Sj, 1 \u2264 i, j \u2264 n between pairs of\nobjects. Following the convention, we encode each such map \u03c6ij as a permutation matrix Xij \u2208 Pm,\nwhere Pm is the space of permutation matrices of dimension m:\n\nPm := {X|X \u2208 [0, 1]m\u00d7m, X1m = 1m, X T 1m = 1m},\n\nwhere 1m = (1,\u00b7\u00b7\u00b7 , 1)T \u2208 Rm is the vector whose elements are 1.\nij \u2208 G along a connected\nThe input permutation synchronization consists of noisy permutations Xin\nobject graph G. As described in [4, 9], a widely used pipeline to generate such input is to 1) establish\nthe object graph G by connecting each object and similar objects using object descriptors (e.g.,\nHOG [14] for images) , and 2) apply off-the-shelf pair-wise object matching methods to compute the\ninput pair-wise maps (e.g., SIFTFlow [15] for images and BIM [16] for 3D shapes).\nThe output consists of improved maps between all of objects\nXij, 1 \u2264 i, j \u2264 n.\n\n2.2 Algorithm\nWe begin with de\ufb01ning a data matrix Xobs \u2208 Rnm\u00d7nm that encodes the initial pairwise maps in\nblocks:\n\n(cid:40) 1\u221a\n\nXobs\n\nij =\n\nXin\nij,\n\ndidj\n0,\n\n(i, j) \u2208 G\notherwise\n\n(1)\n\nwhere di := |{Sj|(Si, Sj) \u2208 G}| is the degree of object Si in graph G.\nRemark 1. Note that the way we encode the data matrix is different from [12, 13] in the sense that\nwe follow the common strategy for handling irregular graphs and use a normalized data matrix.\n\nThe proposed algorithm is motivated from the fact that when the input pair-wise maps are correct, the\ncorrect maps between all pairs of objects can be recovered from the leading eigenvectors of Xobs:\nProposition 2.1. Suppose there exist latent maps (e.g., the ground-truth maps to one object) Xi, 1 \u2264\ni \u2264 n so that X in\nj Xi, (i, j) \u2208 G. Denote W \u2208 Rnm\u00d7m as the matrix that collects the \ufb01rst m\neigenvectors of Xobs in its columns. Then the underlying pair-wise maps can be computed from the\ncorresponding matrix blocks of matrix W W T :\n\nij = X T\n\nX T\n\nj Xi =\n\n(W W T )ij,\n\n1 \u2264 i, j \u2264 n.\n\n(2)\n\n(cid:80)n\ni=1 di(cid:112)didj\n\nThe key insight of the proposed approach is that even when the input maps are noisy (i.e., the blocks\nof X obs are corrupted), the leading eigenvectors of X obs are still stable under these perturbations\n(we will analyze this stability property in Section 3). This motivates us to design a simple two-step\npermutation synchronization approach called NormSpecSync. The \ufb01rst step of NormSpecSync\ncomputes the leading eigenvectors of W ; the second step of NormSpecSync rounds the induced\n\n2\n\n\fAlgorithm 1 NormSpecSync\n\nInput: Xobs based on (1), \u03b4max\nInitialize W0: set W0 as an initial guess for the top-m orthonormal eigenvectors, k \u2190 0\nwhile (cid:107)W (k) \u2212 W (k\u22121)(cid:107) > \u03b4max do\n\n= X obs \u00b7 W (k),\n\nW (k+1)+\nW (k+1)R(k+1) = W (k+1)+,\nk \u2190 k + 1.\n\n(QR factorization),\n\nend while\nSet W = W (k) and X\nspec\nRound each X\ni1\nOutput: Xij = X T\n\nspec\ni1 = (W W T )i1.\ninto the corresponding Xi1 by solving (3).\nj1Xi1, 1 \u2264 i, j \u2264 n.\n\nmatrix blocks (2) into permutations. In the following, we elaborate these two steps and analyze the\ncomplexity. Algorithm 1 provides the pseudo-code.\nLeading eigenvector computation. Since we only need to compute the leading m eigenvectors of\nX obs, we propose to use generalized power method. This is justi\ufb01ed by the observation that usually\nthere exists a gap between \u03bbm and \u03bbm+1. In fact, when the input pair-wise maps are correct, it is\neasy to derive that the leading eigenvectors of X obs are given by:\n\n\u03bb1(X obs) = \u00b7\u00b7\u00b7 = \u03bbm(X obs) = 1, \u03bbm+1(X obs) = \u03bbn\u22121(G),\n\nwhere \u03bbn\u22121(G) is the second largest eigenvalue of the normalized adjacency matrix of G. As we\nwill see later, the eigen-gap \u03bbm(X obs) \u2212 \u03bbm+1(X obs) is still persistent in the presence of corrupted\npair-wise maps, due to the stability of eigenvalues under perturbation.\nProjection onto Pm. Denote X spec\nXij, 1 \u2264 i, j \u2264 n obey Xij = X T\ninto Xik. Without losing generality, we set k = 1 in this paper.\nThe rounding is done by solving the following constrained optimization problem, which projects\nX obs\n\n(W W T )ij. Since the underlying ground-truth maps\njkXik, 1 \u2264 i, j \u2264 n for any \ufb01xed k, we only need to round X spec\n\ni1 onto the space of permutations via the Frobenius norm:\n\n(cid:80)n\ni=1 di\u221a\n\ndidj\n\n:=\n\nik\n\nij\n\n(cid:16)(cid:107)X(cid:107)2\n\nF + (cid:107)X obs\ni1 (cid:107)2\n\nF \u2212 2(cid:104)X, X obs\n\ni1 (cid:105)(cid:17)\n\nXi1 = arg min\nX\u2208Pm\n= arg max\nX\u2208Pm\n\n(cid:107)X \u2212 X obs\ni1 (cid:107)2\n(cid:104)X, X obs\ni1 (cid:105).\n\nF = arg min\nX\u2208Pm\n\n(3)\n\n1\n\nij\n\n(See Algorithm 1).\n\nThe optimization problem described in (3) is the so-called linear assignment problem, which can be\nsolved exactly using the Hungarian algorithm whose complexity is O(m3) (c.f. [17]). Note that the\noptimal solution of (3) is invariant under global scaling and shifting of X obs\nand\nm 11T when generating X obs\nTime complexity of NormSpecSync. Each step of the generalized power method consists of a matrix-\nvector multiplication and a QR factorization. The complexity of the matrix-vector multiplication,\nwhich leverages of the sparsity in X obs, is O(nE \u00b7 m2), where nE is the number of edges in G.\nThe complexity of each QR factorization is O(nm3). As we will analyze laser, generalized power\nmethod converges linearly, and setting \u03b4max = 1/n provides a suf\ufb01ciently accurate estimation\nof the leading eigenvectors. So the total time complexity of the Generalized power method is\n\nO(cid:0)(nEm2 + nm3(cid:1) log(n)). The time complexity of the rounding step is O(nm3). In summary, the\ntotal complexity of NormSpecSync is O(cid:0)(nEm2 + nm3(cid:1) log(n)). In comparison, the complexity of\n\n(cid:80)n\ni=1 di\u221a\n\ni1 , so we omit\n\nthe SDP formulation [9], even when it is solved using the fast ADMM method (alternating direction\nof multiplier method), is at least O(n3m3nadmm. So NormSpecSync exhibits signi\ufb01cant speedups\nwhen compared to SDP formulations.\n\ndidj\n\n3\n\n\f3 Analysis\n\nIn this section, we provide an analysis of NormSpecSync under a generalized Erd\u02ddos-R\u00e9nyi noise\nmodel.\n\n3.1 Noise Model\n\nThe noise model we consider is given by two parameters m and p. Speci\ufb01cally, we assume the\nobservation graph G is \ufb01xed. Then independently for each edge (i, j) \u2208 E,\n\n(cid:26) Im with probability p\n\nPij with probability 1 \u2212 p\n\nXin\n\nij =\n\n(4)\n\nwhere Pij \u2208 Pm is a random permutation.\nRemark 2. The noise model described above assumes the underlying permutations are identity maps.\nIn fact, one can assume a generalized noise model\n\n(cid:26) X T\n\nXin\n\nij =\n\nj1Xi1 with probability p\n\nwith probability 1 \u2212 p\n\nPij\n\nwhere Xi1, 1 \u2264 i \u2264 n are pre-de\ufb01ned underlying permutations from object Si to the \ufb01rst object S1.\nHowever, since Pij are independent of Xi1. It turns out the model described above is equivalent to\n\n(cid:26) Im with probability p\n\nPij with probability 1 \u2212 p\n\nXj1Xin\n\nijX T\n\ni1 =\n\nWhere Pij are independent random permutations. This means it is suf\ufb01cient to consider the model\ndescribed in (4).\nRemark 3. The fundamental difference between our model and the one proposed in [11] or the ones\nused in low-rank matrix recovery [18] is that the observation pattern (i.e., G) is \ufb01xed, while in other\nmodels it also follows a random model. We argue that our assumption is more practical because\nthe observation graph is constructed by comparing object descriptors and it is dependent on the\ndistribution of the input objects. On the other hand, \ufb01xing G signi\ufb01cantly complicates the analysis of\nNormSpecSync, which is the main contribution of this paper.\n\n3.2 Main Theorem\n\nNow we state the main result of the paper.\n\nTheorem 3.1. Let dmin := min1\u2264i\u2264n di, davg :=(cid:80)\n\ni di/n, and denote \u03c1 as the second top eigen-\n\u221a\nvalue of normalized adjacency matrix of G. Assume dmin = \u2126(\nn ln3 n), davg = O(dmin),\n\u03c1 < min{p, 1/2}. Then under the noise model described above, NormSpecSync recovers the\nunderlying pair-wise maps with high probability if\n\nfor some constant C.\n\np > C \u00b7\n\nln3 n\n\u221a\ndmin/\n\n,\n\nn\n\n(5)\n\nProof Roadmap. The proof of Theorem 3.1 combines two stability bounds. The \ufb01rst one considers\nthe projection step:\nProposition 3.1. Consider a permutation matrix X = (xij) \u2208 Pm and another matrix X = (xij) \u2208\nRm\u00d7m. If (cid:107)X \u2212 X(cid:107) < 1\n\n2 , then\n\nX = arg min\nY \u2208Pm\n\nProof. The proof is quite straight-forward. In fact,\n\n(cid:107)Y \u2212 X(cid:107)2\nF .\n\n(cid:107)X \u2212 X(cid:107)\u221e \u2264 (cid:107)X \u2212 X(cid:107) <\n\n1\n2\n\n.\n\n4\n\n\f(a) NormSpecSync (2.25 seconds)\n\n(b) SDP (203.12 seconds)\n\n(c) DiffSync(1.07 seconds)\n\nFigure 1: Comparisons between NormSpecSync, SDP[9], DiffSync[13] on the noise model described in Sec. 2.\n\nThis means the corresponding element xij of each non-zero element in xij is dominant in its row and\ncolumn, i.e.,\n\nxij (cid:54)= 0 \u2194 xij > max(max\nk(cid:54)=j\n\nxik, max\nk(cid:54)=i\n\nxkj),\n\nwhich ends the proof.\nThe second bound concerns the block-wise stability of the leading eigenvectors of X obs:\nLemma 3.1. Under the assumption of Theorem 3.1, then w.h.p.,\n\n(cid:13)(cid:13)(cid:13)(cid:13)(cid:80)n\n\n\u221a\ni=1 di\ndid1\n\n(cid:13)(cid:13)(cid:13)(cid:13) <\n\n(W W T )i1 \u2212 Im\n\n1 \u2264 i \u2264 n.\n\n1\n3\n\n,\n\n(cid:4)\n\n(6)\n\nIt is easy to see that we can prove Theorem 3.1 by combing Lemma 3.1 and Prop. 3.1. Yet unlike\nProp. 3.1, the proof of Lemma 3.1 is much harder. The major dif\ufb01culty is that (6) requires controlling\neach block of the leading eigenvectors, namely, it requires a L\u221e bound, whereas most stability results\non eigenvectors are based on the L2-norm. Due to space constraint, we defer the proof of Lemma 3.1\n(cid:4)\nto Appendix A and the supplemental material.\nNear-optimality of NormSpecSync. Theorem 3.1 implies that NormSpecSync is near-optimal with\nrespect to the information theoretical bound described in [19]. In fact, when G is a clique, (5) becomes\np > C \u00b7 ln3(n)\u221a\nn , which aligns with the lower bound in [19] up to a polylogarithmic factor. Following\nthe model described in [19], we can also assume that the observation graph G is sampled with a\ndensity factor q, namely, two objects are connected independently with probability q. In this case,\nit is easy to see that dmin > O(nq/ ln n) w.h.p., and (5) becomes p > C \u00b7 ln4 n\u221a\nnq . This bound also\nstays within a polylogarithmic factor from the lower bound in [19], indicating the near-optimality of\nNormSpecSync.\n\n4 Experiments\n\nIn this section, we perform quantitative evaluations of NormSpecSync on both synthetic and real\nexamples. Experimental results show that NormSpecSync is superior to state-of-the-art map syn-\nchronization methods in the literature. We organize the remainder of this section as follows. In\nSection 4.1, we evaluate NormSpecSync on synthetic examples. Then in section 4.2, we evaluate\nNormSpecSync on real examples.\n\n4.1 Quantitative Evaluations on Synthetic Examples\n\nWe generate synthetic data by following the same procedure described in Section 2. Speci\ufb01cally,\neach synthetic example is controlled by three parameters G, m, and p. Here G speci\ufb01es the input\ngraph; m describes the size of each permutation matrix; p controls the noise level of the input maps.\nThe input maps follow a generalized Erdos-Renyi model, i.e., independently for each edge (i, j) \u2208 G\nin the input graph, with probability p the input map X in\nij is a random\npermutation. To simplify the discussion, we \ufb01x m = 10, n = 200 and vary the observation graph G\nand p to evaluate NormSpecSync and existing algorithms.\n\nij = Im, and otherwise X in\n\n5\n\nVarying graph densityGraph density0.10.30.50.70.9p-true0.10.20.30.40.5Varying graph densityGraph density0.10.30.50.70.9p-true0.10.20.30.40.5Varying graph densityGraph density0.10.30.50.70.9p-true0.10.20.30.40.5\f(a) NormSpecSync\n\n(b) SpecSync\n\nFigure 2: Comparison between NorSpecSync and SpecSync on irregular observation graphs.\n\n2 \u2212 q)n and ( 1\n\nDense graph versus sparse graph. We \ufb01rst study the performance of NormSpecSync with respect\nto the density of the graph. In this experiment, we control the density of G by following a standard\nErd\u02ddos-R\u00e9nyi model with parameter q, namely independently, each edge is connected with probability\nq. For each pair of \ufb01xed p and q, we generate 10 examples. We then apply NormSpecSync and count\nthe ratio that the underlying permutations are recovered. Figure 1(a) illustrates the success rate of\nNormSpecSync on a grid of samples for p and q. Blue and yellow colors indicate it succeeded and\nfailed on all the examples, respectively, and the colors in between indicate a mixture of success and\nfailure. We can see that NormSpecSync tolerates more noise when the graph becomes denser. This\naligns with our theoretical analysis result.\nNormSpecSync versus SpecSync. We also compare NormSpecSync with SpecSync [12], and\nshow the advantage of NormSpecSync on irregular observation graphs. To this end, we generate\nG using a different model. Speci\ufb01cally, we let the degree of the vertex to be uniformly distribute\nbetween ( 1\n2 + q)n. As illustrated in Figure 2, when q is small, i.e., all the vertices have\nsimilar degrees, the performance of NormSpecSync and SpecSync are similar. When q is large, i.e.,\nG is irregular, NormSpecSync tend to tolerate more noise than SpecSync. This shows the advantage\nof utilizing a normalized data matrix.\nNormSpecSync versus DiffSync. We proceed to compare NormSpecSync with DiffSync [13],\nwhich is a permutation synchronization method based on diffusion distances. NormSpecSync and\nDiffSync exhibit similar computation ef\ufb01ciency. However, NormSpecSync can tolerate signi\ufb01cantly\nmore noise than DiffSync, as illustrated in Figure 1(c).\nNormSpecSync versus SDP.\nFinally, we compare NormSpecSync with SDP [9], which formulates\npermutation synchronization as solving a semide\ufb01nite program. As illustrated in Figure 1(b), the\nexact recovery ability of NormSpecSync and SDP are similar. This aligns with our theoretical analysis\nresult, which shows the near-optimality of NormSpecSync under the noise model of consideration.\nYet computationally, NormSpecSync is much more ef\ufb01cient than SDP. The averaged running time for\nSpecSync is 2.25 second. In contrast, SDP takes 203.12 seconds in average.\n\n4.2 Quantitative Evaluations on Real Examples\n\nIn this section, we present quantitative evaluation of NormSpecSync on real datasets.\nCMU Hotel/House.\nWe \ufb01rst evaluate NormSpecSync on CMU Hotel and CMU House\ndatasets [20]. The CMU Hotel dataset contains 110 images, where each image has 30 marked\nfeature points. In our experiment, we estimate the initial map between a pair of images using\nRANSAC [21]. We consider two observation graphs: a clique observation graph Gf ull, where we\nhave initial maps computed between all pairs of images, and a sparse observation graph Gsparse.\nGsparse is constructed to only connect similar images. In this experiment, we connect an edge between\ntwo images if the difference in their HOG descriptors [22] is smaller than 1\n2 of the average descriptor\ndifferences among all pairs of images. Note that Gsparse shows high variance in terms of vertex\n\n6\n\nVarying vertex degreesIrregularty0.00.10.20.30.4p-true0.10.20.30.40.5Varying vertex degreesIrregularty0.00.10.20.30.4p-true0.10.20.30.40.5\fFigure 3: Comparison between NorSpecSync, SpecSync, DiffSync and SDP on CMU Hotel/House and SCAPE.\nIn each dataset, we consider a full observation graph and a sparse observation graph that only connects potentially\nsimilar objects.\n\ndegree. The CMU House dataset is similar to CMU Hotel, containing 100 images and exhibiting\nslightly bigger intra-cluster variability than CMU Hotel. We construct the observation graphs and the\ninitial maps in a similar fashion. For quantitative evaluation, we measure the cumulative distribution\nof distances between the predicted target points and the ground-truth target points.\nFigure 3(Left) compares NormSpecSync with the SDP formulation, SpecSync, and DiffSync. On\nboth full and sparse observation graphs, we can see that NormSpecSync, SDP and SpecSync are\nsuperior to DiffSync. The performance of NormSpecSync and SpecSync on Gf ull is similar, while\non Gsparse, NormSpecSync shows a slight advantage, due to its ability to handle irregular graphs.\nMoreover, although the performance of NormSpecSync and SDP are similar, SDP is much slower\nthan NormSpecSync. For example, on Gsparse, SDP took 1002.4 seconds, while NormSpecSync\nonly took 3.4 seconds.\nSCAPE. Next we evaluate NormSpecSync on the SCAPE dataset. SCAPE consists of 71 different\nposes of a human subject. We uniformly sample 128 points on each model. Again we consider a\nfull observation graph Gf ull and a sparse observation graph Gsparse. Gsparse is constructed in the\nsame way as above, except we use the shape context descriptor [4] for measuring the similarity\nbetween 3D models. In addition, the initial maps are computed from blended-intrinsic-map [16],\nwhich is the state-of-the-art technique for computing dense correspondences between organic shapes.\nFor quantitative evaluation, we measure the cumulative distribution of geodesic distances between\nthe predicted target points and the ground-truth target points. As illustrated in Figure 3(Right), the\nrelative performance between NormSpecSync and the other three algorithms is similar to CMU Hotel\nand CMU House. In particular, NormSpecSync shows an advantage over SpecSync on Gsparse. Yet\nin terms of computational ef\ufb01ciency, NormSpecSync is far better than SDP.\n\n5 Conclusions\nIn this paper, we propose an ef\ufb01cient algorithm named NormSpecSync towards solving the permuta-\ntion synchronization problem. The algorithm adopts a spectral view of the mapping problem and\nexhibits surprising behavior both in terms of computation complexity and exact recovery conditions.\nThe theoretical result improves upon existing methods from several aspects, including a \ufb01xed obser-\n\n7\n\nEuclidean distance (pixels)0123456% correspondences0102030405060708090100CMU-G-FullRANSACDiffSyncSpecSyncNonSpecSyncSDPEuclidean distance (pixels)0123456% correspondences0102030405060708090100CMU-G-SparseRANSACDiffSyncSpecSyncNonSpecSyncSDPGeodesic distance (diameter)00.050.10.150.2% correspondences0102030405060708090100SCAPE-G-FullRANSACDiffSyncSpecSyncNonSpecSyncSDPGeodesic distance (diameter)00.050.10.150.2% correspondences0102030405060708090100SCAPE-G-SparseRANSACDiffSyncSpecSyncNonSpecSyncSDP\fvation graph and a practical noise method. Experimental results demonstrate the usefulness of the\nproposed approach.\nThere are multiple opportunities for future research. For example, we would like to extend NormSpec-\nSync to handle the case where input objects only partially overlap with each other. In this scenario,\ndeveloping and analyzing suitable rounding procedures become subtle. Another example is to extend\nNormSpecSync for rotation synchronization, e.g., by applying Spectral decomposition and rounding\nin an iterative manner.\nAcknowledgement. We would like to thank the anonymous reviewers for detailed comments on how\nto improve the paper. The authors would like to thank the support of DMS-1700234, CCF-1302435,\nCCF-1320175, CCF-1564000, CNS-0954059, IIS-1302662, and IIS-1546500.\n\nA Proof Architecture of Lemma 3.1\n\n2 AD\u2212 1\n\nThe normalized adjacency matrix \u00afA = D\u2212 1\n\nIn this section, we provide a roadmap for the proof of Lemma 3.1. The detailed proofs are deferred to\nthe supplemental material.\nReformulate the observation matrix.\n2 can\nbe decomposed as \u00afA = ssT + V \u039bV T , where the dominant eigenvalue is 1 and corresponding\np M = \u00afA \u2297 Im + \u02dcN, and it is clear\neigenvector is s. We reformulate the observation matrix as 1\nto see that the ground truth result relates to the term (ssT ) \u2297 Im, while the noise comes from two\nterms: (V \u039bV T ) \u2297 Im and \u02dcN. More speci\ufb01cally, the noise not only comes from the randomness of\nuncertainty of the measurements, but also from the graph structure, and we use \u03c1 to represent the\nspectral norm of \u039b. When the graph is disconnected or near disconnected, \u03c1 is close to 1 and it is\nimpossible to recover the ground truth.\nBound the spectral norm of \u02dcN.\n\u221a\nin each block. In a complete graph, the spectral norm is bounded by O( 1\np\nconsidering the graph structure, we give a O(\n) bound.\nMeasure the block-wise distance between U and s \u2297 Im.\n2 , we\nwant to show the distance between U and s \u2297 1m is small, where the distance function dist(\u00b7) is\nde\ufb01ned as:\n\nThe noise term \u02dcN consists of random matrices with mean zero\nn ), however, when\n\nLet M = U \u03a3U T + U2\u03a32U T\n\n1\ndmin\n\n\u221a\n\np\n\n,\n\nR:RRT =I\n\ndist(U, V ) = min\n\n1 ,\u00b7\u00b7\u00b7 , X T\n\n(cid:107)X(cid:107)B = max\n\nand this B\u2212norm for any matrix X represented in the form X = [X T\nde\ufb01ned as\n\n(7)\nn ]T \u2208 Rmn\u00d7m is\n(8)\nMore speci\ufb01cally, we bound the distance between U and s \u2297 Im by constructing a series of matrix\n{Ak}, and we can show for some k = O(log n), the distances from s \u2297 Ak to both U and s \u2297 Im\nare small. Therefore, by using the triangle inequality, we can show that U and s \u2297 Im is close.\nSketch proof of Lemma 3.1. Once we are able to show that there exists some rotation matrix R,\nsuch that dist(U, s \u2297 Im) is in the order of o( 1\u221a\nn ), then it is straightforward to prove Lemma 3.1.\nIntuitively, this is because the measurements from the eigenvectors is close enough to the ground\ntruth, hence their second moment will still be close. Formally speaking,\n\n(cid:107)Xi(cid:107)F .\n\ni\n\n(cid:13)(cid:13)(cid:13)U \u2212 V R\n\n(cid:13)(cid:13)(cid:13)B\n\n(cid:13)(cid:13)UiU T\nj \u2212 (si \u00b7 Im)(sj \u00b7 Im)(cid:13)(cid:13)\n=(cid:13)(cid:13)UiRRT U T\nj \u2212 (si \u00b7 Im)(sj \u00b7 Im)(cid:13)(cid:13)\n=(cid:13)(cid:13)UiR(RT U T\nj \u2212 (sj \u00b7 Im)T ) + (UiR \u2212 si \u00b7 Im)(sj \u00b7 Im)T(cid:13)(cid:13)\n\u2264(cid:13)(cid:13)Ui\n(cid:13)(cid:13) \u00b7 dist(U, s \u2297 Im) + dist(U, s \u2297 Im) \u00b7(cid:13)(cid:13)sj \u00b7 Im\n(cid:13)(cid:13)(cid:13)(cid:80)n\n(cid:13)(cid:13)(cid:13)UiU T\n(cid:13)(cid:13)(cid:13) =\ni=1 di(cid:112)didj\n\n(cid:80)n\ni=1 di(cid:112)didj\n\nj \u2212 (si \u00b7 Im)(sj \u00b7 Im)\n\nj \u2212 Im\n\nUiU T\n\n(cid:13)(cid:13)\n\n(9)\n(10)\n(11)\n(12)\n\n(13)\n\n(cid:13)(cid:13)(cid:13),\n\nOn the other hand, notice that\n\nand we only need to show that (13) is in the order of o(1). The details are included in the supplemental\nmaterial.\n\n8\n\n\fReferences\n[1] D. F. Huber, \u201cAutomatic three-dimensional modeling from reality,\u201d Tech. Rep., 2002.\n\n[2] D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, \u201cDiscrete-continuous optimization for large-scale\nstructure from motion,\u201d in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011,\npp. 3001\u20133008.\n\n[3] Q.-X. Huang, S. Fl\u00f6ry, N. Gelfand, M. Hofer, and H. Pottmann, \u201cReassembling fractured objects by\n\ngeometric matching,\u201d in ACM SIGGRAPH 2006 Papers, 2006, pp. 569\u2013578.\n\n[4] V. G. Kim, W. Li, N. Mitra, S. DiVerdi, and T. Funkhouser, \u201cExploring collections of 3D models using\nfuzzy correspondences,\u201d Transactions on Graphics (Proc. of SIGGRAPH 2012), vol. 31, no. 4, Aug. 2012.\n\n[5] W. Marande and G. Burger, \u201cMitochondrial dna as a genomic jigsaw puzzle,\u201d Science, vol. 318, Jul. 2007.\n\n[6] C. Zach, M. Klopschitz, and M. Pollefeys, \u201cDisambiguating visual relations using loop constraints.\u201d in\n\nIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 1426\u20131433.\n\n[7] D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, \u201cDiscrete-continuous optimization for large-scale\nIEEE,\n\nstructure from motion,\u201d in IEEE Conference on Computer Vision and Pattern Recognition (CVPR).\n2011, pp. 3001\u20133008.\n\n[8] A. Nguyen, M. Ben-Chen, K. Welnicka, Y. Ye, and L. Guibas, \u201cAn optimization approach to improving\ncollections of shape maps,\u201d in Eurographics Symposium on Geometry Processing (SGP), 2011, pp. 1481\u2013\n1491.\n\n[9] Q. Huang and L. Guibas, \u201cConsistent shape maps via semide\ufb01nite programming,\u201d Computer Graphics\nForum, Proc. Eurographics Symposium on Geometry Processing (SGP), vol. 32, no. 5, pp. 177\u2013186, 2013.\n\n[10] L. Wang and A. Singer, \u201cExact and stable recovery of rotations for robust synchronization,\u201d CoRR, vol.\n\nabs/1211.2441, 2012.\n\n[11] Y. Chen, L. J. Guibas, and Q. Huang, \u201cNear-optimal joint object matching via convex relaxation,\u201d 2014.\n\n[Online]. Available: http://arxiv.org/abs/1402.1473\n\n[12] D. Pachauri, R. Kondor, and V. Singh, \u201cSolving the multi-way matching problem by permutation synchro-\n\nnization,\u201d in Advances in Neural Information Processing Systems, 2013, pp. 1860\u20131868.\n\n[13] D. Pachauri, R. Kondor, G. Sargur, and V. Singh, \u201cPermutation diffusion maps (pdm) with application to\nthe image association problem in computer vision,\u201d in Advances in Neural Information Processing Systems,\n2014, pp. 541\u2013549.\n\n[14] N. Dalal and B. Triggs, \u201cHistograms of oriented gradients for human detection,\u201d in Proceedings of the 2005\nIEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905) - Volume 1 -\nVolume 01, ser. CVPR \u201905, 2005, pp. 886\u2013893.\n\n[15] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. T. Freeman, \u201cSift \ufb02ow: Dense correspondence across different\nscenes,\u201d in Proceedings of the 10th European Conference on Computer Vision: Part III, ser. ECCV \u201908,\n2008, pp. 28\u201342.\n\n[16] V. G. Kim, Y. Lipman, and T. Funkhouser, \u201cBlended intrinsic maps,\u201d in ACM Transactions on Graphics\n\n(TOG), vol. 30, no. 4. ACM, 2011, p. 79.\n\n[17] R. Burkard, M. Dell\u2019Amico, and S. Martello, Assignment Problems. Philadelphia, PA, USA: Society for\n\nIndustrial and Applied Mathematics, 2009.\n\n[18] E. J. Cand\u00e8s, X. Li, Y. Ma, and J. Wright, \u201cRobust principal component analysis?\u201d J. ACM, vol. 58, no. 3,\n\npp. 11:1\u201311:37, Jun. 2011. [Online]. Available: http://doi.acm.org/10.1145/1970392.1970395\n\n[19] Y. Chen, C. Suh, and A. J. Goldsmith, \u201cInformation recovery from pairwise measurements: A shannon-\n\ntheoretic approach,\u201d CoRR, vol. abs/1504.01369, 2015.\n\n[20] T. S. Caetano, L. Cheng, Q. V. Le, and A. J. Smola, \u201cLearning graph matching,\u201d in Computer Vision, 2007.\n\nICCV 2007. IEEE 11th International Conference on.\n\nIEEE, 2007, pp. 1\u20138.\n\n[21] M. A. Fischler and R. C. Bolles, \u201cRandom sample consensus: A paradigm for model \ufb01tting with applications\n\nto image analysis and automated cartography,\u201d Commun. ACM, vol. 24, no. 6, pp. 381\u2013395, Jun. 1981.\n\n[22] R. Osada, T. Funkhouser, B. Chazelle, and D. Dobkin, ACM Trans. Graph., vol. 21, no. 4, pp. 807\u2013832,\n\n2002.\n\n9\n\n\f", "award": [], "sourceid": 2486, "authors": [{"given_name": "Yanyao", "family_name": "Shen", "institution": "UT Austin"}, {"given_name": "Qixing", "family_name": "Huang", "institution": "Toyota Technological Institute at Chicago"}, {"given_name": "Nati", "family_name": "Srebro", "institution": "TTI-Chicago"}, {"given_name": "Sujay", "family_name": "Sanghavi", "institution": "UT-Austin"}]}